This job offer is not available in your country.

AI Agent Quality Specialist - AI Trainer

MindriftWorkFromHome, Mpumalanga, South Africa

3 days ago

Job description

Overview

AI Agent Quality Specialist - AI Trainer at Mindrift. We connect domain experts with cutting-edge AI projects to ethically shape the future of AI. The Mindrift platform, launched and powered by Toloka, aims to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

We are looking for curious and intellectually proactive contributors who double-check assumptions, play devil’s advocate, and thrive in an asynchronous, remote, flexible environment. This is a flexible, project-based opportunity suitable for individuals who want to learn how modern AI systems are tested and evaluated.

Responsibilities

Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Annotate cause-effect relationships, reasoning paths, and plausible alternatives
Think through complex systems and policies to ensure agents are tested properly
Collaborate with QA, writers, or developers to suggest refinements or edge-case coverage

Qualifications and desirable attributes

Excellent analytical thinking : ability to reason about complex systems, scenarios, and logical implications

Strong attention to detail : ability to spot contradictions, ambiguities, and vague requirements

Familiarity with structured data formats; ability to read JSON / YAML (writing not required)

Holistic assessment : identify what’s missing, what’s unrealistic, and what might break

Clear written English to document findings

We also value applicants who have experience with policy evaluation, logic puzzles, case studies, or structured scenario design; backgrounds in consulting, academia, olympiads, or research; exposure to LLMs, prompt engineering, or AI-generated content; familiarity with QA or test-case thinking (edge cases, failure modes); and some understanding of evaluation scoring (e.g., precision, coverage).

Benefits

Earn by applying your expertise, with rates up to $20 / hour depending on skills and project needs

Flexible, remote, freelance project that fits around other commitments

Gain experience on an advanced AI project to enhance your portfolio

Influence how future AI models understand and communicate in your field

How to get started

Apply to this post, qualify, and contribute to a project aligned with your skills on your own schedule. Shape the future of AI while building tools that benefit everyone.

#J-18808-Ljbffr

Create a job alert for this search

Quality Specialist • WorkFromHome, Mpumalanga, South Africa