Data Annotation / Prompt Engineer

Inviting applications for the role of Data Annotation / Prompt Engineer. In this role, you will drive the quality, consistency, and governance of AI and Large Language Model (LLM) outputs while supporting annotation, prompt engineering, AI evaluation, and quality assurance activities.

Responsibilities

• Identify and manage project deliverables.
• Lead the design, optimization, and standardization of annotation processes and prompts across multiple AI use cases.
• Collaborate with the AI team to define and govern annotation schemas, taxonomies, and evaluation rubrics.
• Review complex or high-risk AI outputs and provide authoritative quality judgments.
• Ensure consistency, accuracy, and auditability of annotation and evaluation results.
• Define reusable prompt frameworks and best practices aligned with business and compliance objectives.
• Evaluate prompt performance at scale and guide improvements based on data-driven insights.
• Establish quality benchmarks, calibration processes, and escalation protocols.
• Identify bias, hallucinations, and systemic risks in AI outputs and recommend mitigation strategies.
• Support AI governance, compliance, and responsible AI initiatives.
• Participate in calibration sessions to align on annotation standards and evaluation criteria.
• Work closely with AI team Data Scientists, content reviewers, and product teams.
• Mentor and coach junior and intermediate annotators / prompt engineers.
• Develop and maintain AI evaluation frameworks and quality standards.
• Evaluate AI-generated outputs using Completeness, Groundedness, Precision, Guardrails, and Bias metrics.
• Create and maintain benchmark datasets, golden datasets, and ground-truth references.
• Support RAG evaluation, regression testing, and conversational AI assessments.
• Analyze evaluation results and provide actionable recommendations.
• Maintain evaluation documentation, auditability, and traceability.