LLM AI Quality Analyst (Personalization) – Remote

About the role

This short‑term contract position focuses on assessing and improving personalized responses generated by large language models. Working remotely, you will help ensure AI outputs respect user context, maintain data privacy, and deliver helpful, grounded answers.

Key responsibilities

• Design multi‑turn conversational prompts that incorporate personal context.
• Evaluate AI responses for relevance, grounding, and overall helpfulness.
• Check correct and incorrect usage of personal data in model outputs.
• Conduct side‑by‑side (SxS) ranking of AI responses.
• Identify grounding errors, poor inferences, and forced personalization.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data‑source usage.
• Maintain strict data hygiene by deleting evaluation conversations after review.

Required profile

• Fluent in English (reading and writing).
• Strong experience in data annotation, AI quality evaluation, content moderation, or related roles.
• Analytical mindset with high attention to detail.
• Ability to evaluate nuanced and ambiguous AI responses.
• Comfortable using a personal Google account with enabled data sources.
• BS/BA degree or equivalent analytical experience.
• Excellent written communication and structured feedback skills.
• Self‑motivated, able to work independently in a remote setting.
• Reliable desktop/laptop with stable internet connection.

Required skills

What we offer

• Hourly rate of $11.5.
• Flexible remote work schedule (20 hours per week).
• Immediate start and a one‑month engagement.