Evaluation Specialist/Recent Grad

The work

Design difficult questions and thoroughly researched answers for AI evaluation. Test how models handle questions requiring synthesis and several reasoning steps, then refine your work using results and reviewer feedback. Previous AI experience is not required.

• Create original question and answer pairs across varied subjects.
• Check answers against multiple independent sources and document supporting evidence.
• Build tasks requiring analytical reasoning and synthesis across information.
• Test prompts with AI models, examine outcomes and adjust the difficulty.
• Explain your research method, reasoning and citations clearly.
• Revise material in response to review while following quality standards and project instructions.

Preferred experience and skills

• Preferred candidates demonstrate strong independent research and critical source evaluation.
• Fluent written English with precise language and close attention to detail; English need not be your first language.
• Ability to formulate nuanced, well structured questions that probe deep understanding.
• Reliability, accountability and independent remote work habits.
• Interest in testing the limits of AI capabilities.
• AI training, content creation or evaluation experience is helpful but optional.
• Recent graduates, advanced degree holders and applicants from any background may qualify through demonstrated writing and research skill.

Working arrangements

Payment is made per task accepted against project specifications, and completion time varies with experience and workflow. A weekly minimum submission requirement applies, although the source does not state its numerical amount.

Selection is typically completed within 48 hours. Successful contributors are expected to start tasks within 24 to 48 hours of onboarding.