- Company Clera
- Location Remote
- Employment type Full Time
- Stated salary USD 30–60 per year
About the Role
Join a specialist talent pool supporting human-data projects that evaluate AI models. You will help design and run assessments that measure model capabilities and contribute to AI safety and capability research.
What You’ll Do
-
Write questions and tasks that isolate and test specific model capabilities.
-
Develop rubrics that define strong, partial, and poor answers.
-
Create reference answers and edge cases to challenge AI models.
-
Grade model outputs against rubrics and explain scoring decisions.
-
Analyze failed answers, group error patterns, and document findings.
-
Compare multiple models using identical prompts.
-
Validate benchmarks and flag compromised or ineffective questions.
-
Participate in small user studies evaluating ratings or interactions with model outputs.
What We’re Looking For
-
Zero to five years of professional experience.
-
Experience in QA or test engineering, assessment design, psychometrics, UX research, data analysis, ML evaluation, research support, or relevant subject-matter work involving question writing.
-
Experience writing test cases, exam questions, rubrics, or evaluation tasks, and reviewing work consistently against written criteria.
-
Comfort using spreadsheets or Python to organize and analyze results, and identifying and documenting error patterns.
-
Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo is useful.
-
Clear written communication and the ability to compare AI model outputs or participate in user and rating studies.
Compensation & Benefits
Compensation is $30 to $60 per hour, depending on project, field, and experience. Payment is for hours worked.
Location
Fully remote. Projects typically last from one week to several months.
Compensation Range: $62.4K – $124.8K
Source: Himalayas. Always check the employer before you send personal details or pay for anything: real job or scam? checklist.
More Information
- Location Remote, United Kingdom
