Legal & Compliance Expert

Build realistic legal and compliance evaluations

Create benchmark tasks that test whether AI systems can complete real legal and compliance work. Translate practical workflows into demanding assignments with clear reference materials and objective grading criteria. Prior AI experience is not required; the emphasis is on direct professional knowledge and sound judgment.

Responsibilities

• Design tasks involving contract review, due diligence, regulatory assessment, and risk analysis, with multiple steps and enough context for a reviewer to evaluate the result.
• Write clear task instructions and prepare or source the supporting agreements, policy documents, regulatory texts, and diligence files.
• Define grading criteria that let two qualified reviewers reach the same conclusion about the same output.
• Test the ability to interpret the supplied materials and make judgments, rather than simply retrieve information.
• Identify ambiguous tasks or outputs that cannot be graded reliably, and flag them instead of submitting them as finished work.
• Deliver approved tasks on schedule and meet the required quality standard.

Relevant experience

• At least five years of direct responsibility in contract review, due diligence, regulatory compliance, or risk. Experience must include doing and owning the work, rather than only supervising it or working alongside it.
• Experience developing or applying structured standards such as review checklists, diligence protocols, compliance frameworks, playbooks, or training assessments.
• Ability to compare multiple documents and identify inconsistencies. A law degree and bar admission, or an equivalent compliance qualification, is helpful but optional.
• Strong written and spoken English, with B2 as the minimum and C1 preferred.
• AI or large language model evaluation experience is welcome but is not a prerequisite.

Contract and selection

The contract is fully remote, with no geographic restrictions. Pay is up to $90 per hour equivalent, and a task is expected to take approximately ten hours. This is project work paid per approved task.

Selection includes screening questions and an AI interview specific to this role, followed by onboarding. The interview presents your domain experience for the client's review. A profile review is also part of the selection process.