AI Agent Evaluation Analyst - AI Trainer

Overview

AI Agent Evaluation Analyst - AI Trainer role at Mindrift. We are looking for curious, intellectually proactive contributors who double-check assumptions and think through scenarios and edge cases. This is a flexible, project-based opportunity for those who enjoy evaluating how modern AI systems are tested and evaluated.

Mindrift connects domain experts with AI projects, powered by Toloka, to unlock the potential of GenAI through real-world expertise from across the globe.

What you'll do

• Review evaluation tasks and scenarios for logic, completeness, and realism
• Identify inconsistencies, missing assumptions, or unclear decision points
• Help define clear expected behaviors (gold standards) for AI agents
• Annotate cause–effect relationships, reasoning paths, and plausible alternatives
• Think through complex systems and policies to ensure agents are tested properly
• Work closely with QA, writers, or developers to suggest refinements or edge case coverage

Requirements

We also value

Benefits

How to get started

Apply to this post, qualify, and contribute to a project aligned with your skills on your own schedule. Shape the future of AI while building tools that benefit everyone.

Seniorities

Employment type

Job function

Industries

Referrals increase your chances of interviewing at Mindrift.

#J-18808-Ljbffr