AI Quality Analyst (Personalization) -Japanese

AI Quality Analyst (Personalization) -Japanese

Job type: Contractor; Engagement Length: 3 months; Our offered

This role requires a unique blend of creativity and analytical rigor. As an AI Quality Analyst, you will evaluate a new personalization feature for .

Day-to-day

• In this role, you will be part of a dynamic team focused on evaluating the quality of personalized AI interactions. Your day-to-day work will involve
• Designing and executing multi-turn conversational prompts (typically 1-5 turns) that require the AI to utilize your personal information and experiences.
• Evaluating model responses based on your intent from the starting prompt, checking if the personalization was appropriately applied.
• Analyzing responses for Grounding issues, ensuring claims about you are supported by evidence and not flawed inferences or hallucinations.
• Assessing Integration quality to ensure personal data is woven naturally into the response without robotic "overnarrating".
• Rigorously evaluating and stack-ranking two model responses side-by-side (SxS) to determine which is overall more helpful, easy to use, and enjoyable.
• Writing clear, defensible rationales for your comparisons, explicitly referencing where issues or positive aspects occurred in the conversation.

You bring

• Japanese Proficiency: Ability to read and write in Japanese with a high degree of comp, as Japanese is the focus language for this project.
• Personal Account Usage: Willingness to use your primary personal Google account (not a testing account) and enable personal data sources for a genuine assessment.

Pay

• Commitments Required: at least 4 hours per day and up to 20 hours per week with 4 hours of overlap with PST.