Rubrics Expert

Rubrics Expert

Work as a Rubrics Expert, where you will play a central role in designing and validating robust rubrics for AI training projects Utilize your expertise in performance

Analysis:, scoring methodology, and real-world judgment to ensure the highest quality of assessment outputs

Details

• Job type: Contract
• Location: Remote

Day-to-day

• Design, develop, and refine comprehensive rubrics tailored to AI training and assessment needs
• Analyze task performance and provide structured, actionable feedback to guide improvement and calibration
• Establish and document effective scoring methodologies that ensure consistency, fairness, and high-quality outcomes
• Lead calibration sessions to align evaluators and maintain rubric integrity across projects
• Validate assessment outputs for accuracy, reliability, and alignment with project objectives
• Oversee quality control processes, making data-driven recommendations for rubric enhancements
• Collaborate with cross-functional teams to integrate prompt design principles and grocery shopping scenarios into assessment frameworks

You bring

• Proven expertise in rubric design, performance
• Analysis:, and feedback structuring for complex assessment environments
• Strong background in scoring methodology development and calibration for AI or educational projects
• Demonstrated success in assessment validation and quality control processes
• Exceptional written and verbal communication skills, with the ability to convey nuanced judgment and rationale
• Experience with prompt design and rubric creation for AI-driven initiatives
• Practical, hands-on grocery shopping experience (via apps, delivery, in-store, or pickup) to inform realistic scenario assessments

Bonus

• Previous experience in AI data training or human data evaluation projects
• Background in instructional design, educational assessment, or related field
• Familiarity with agile work environments and cross-functional project collaboration