Agent Evals Specialist

Job Description

REQUIREMENTS

• Strong written English with the ability to read dense technical content for extended periods
• High level of consistency in scoring and evaluation
• Ability to provide clear, specific, and actionable feedback
• Proficiency with Slack and internal platforms
• Familiarity with Markdown

Preferred

• Prior experience as an AI trainer, tutor, or evaluator (e.g., Outlier, DataAnnotation, xAI, Surge, Mercor, Invisible, Toloka)
• Background in technical writing, editing, QA, translation, paralegal work, or research assistance

RESPONSIBILITIES

• Review AI agent outputs side-by-side with source material to verify accuracy
• Evaluate what the AI agent created, changed, or omitted
• Score tasks using a rubric covering accuracy, coverage, organization, and rule adherence
• Write detailed, specific feedback regarding mistakes to drive AI performance improvements

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#CrossChannelJobs #JobSearch
#CareerOpportunities #HiringNow
#Employment #JobOpenings
#JobSeekers
#FacebookLinkedIn