AI Agent Quality Specialist - AI Trainer

Overview

AI Agent Quality Specialist - AI Trainer at Mindrift. We connect domain experts with cutting-edge AI projects to ethically shape the future of AI. The Mindrift platform, launched and powered by Toloka, aims to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

We are looking for curious and intellectually proactive contributors who double-check assumptions, play devil’s advocate, and thrive in an asynchronous, remote, flexible environment. This is a flexible, project-based opportunity suitable for individuals who want to learn how modern AI systems are tested and evaluated.

Responsibilities

• Review evaluation tasks and scenarios for logic, completeness, and realism
• Identify inconsistencies, missing assumptions, or unclear decision points
• Help define clear expected behaviors (gold standards) for AI agents
• Annotate cause-effect relationships, reasoning paths, and plausible alternatives
• Think through complex systems and policies to ensure agents are tested properly
• Collaborate with QA, writers, or developers to suggest refinements or edge-case coverage

Qualifications and desirable attributes

We also value applicants who have experience with policy evaluation, logic puzzles, case studies, or structured scenario design; backgrounds in consulting, academia, olympiads, or research; exposure to LLMs, prompt engineering, or AI-generated content; familiarity with QA or test-case thinking (edge cases, failure modes); and some understanding of evaluation scoring (e.g., precision, coverage).

Benefits

How to get started

Apply to this post, qualify, and contribute to a project aligned with your skills on your own schedule. Shape the future of AI while building tools that benefit everyone.

#J-18808-Ljbffr