Project Manager, Applied AI

The work

You'll lead large-scale multilingual data collection and Large Language Model (LLM) evaluation projects, coordinating global annotators and data specialists. The role sits within Production (Applied AI), reports to the Program Director, and covers Europe, Asia-Pac, Argentina, and Mexico. The rate is $20 per hour.

Project delivery

• Own the full AI data project lifecycle, from scoping and guideline creation through data delivery and post-mortem analysis.
• Manage multilingual audio, text, and image collection pipelines.
• Coordinate LLM evaluation workflows covering RLHF, SFT, ranking, and safety testing.

Quality and performance

• Track throughput KPIs as the volume of data processed per hour or day.
• Monitor accuracy scores, Inter-Annotator Agreement (IAA), and gold-set performance.
• Analyze cost-per-task and worker efficiency to understand productivity.
• Run QA loops, investigate the root causes of quality declines, and provide corrective training for annotator pools.
• Maintain dashboards showing project health and emerging bottlenecks in real time.

Stakeholder coordination

• Manage relationships with data experts and crowd pools, keeping delivery aligned with SLAs, local language nuances, and linguistic accuracy.
• Work with Applied AI Technical Ops teams and translate technical requirements into clear guidelines for non-technical annotators.
• Establish feedback loops so data findings inform annotation instructions and model fine-tuning strategies.

Essential skills and experience

• 3-5+ years of project management experience specifically in AI/ML data operations.
• Strong knowledge of LLM pre-training, SFT, and RLHF, plus human-in-the-loop evaluation and red teaming.
• Advanced Excel/Google Sheets skills and the ability to write SQL queries for extracting and analyzing performance data.
• A proven record using Agile, Scrum, or Kanban to manage complex workflows.
• The ability to write clear, unambiguous instructions for multilingual audiences.

Preferred qualifications

• Fluency in a second language is highly desirable.
• Experience with data annotation tools, including Super Annotate, and project management tools such as Jira.
• Background in ML Engineering, Computer Science, Data Science and Project Management training.

Professional community

The work supports multilingual AI and human-verified services for enterprises, governments, and AI developers. You'll collaborate with language and domain professionals on projects intended to make information more accessible across languages. The source describes varied remote projects, prompt and fair payment, and opportunities to build your professional network. Application steps are tailored to your expertise.