How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Product Content Engineering is a horizontal function supporting initiatives across Meta's family of apps. We partner closely with product and technical teams to solve problems by providing content-centered solutions, setting standards of quality, and building the frameworks that ensure AI-powered experiences actually work for people.
We're looking for a Content Engineer to join our team and help define how Meta evaluates and improves AI content experiences. You'll work at the intersection of content quality, AI evaluation, and the search and recommendation systems that power Meta's products: building the frameworks, rubrics, and pipelines that hold AI outputs to a high standard. You'll assess model behavior, identify where it falls short, and work cross-functionally with engineering, product, research, and data science teams to make it better.
If you're energized by building better AI experiences through rigorous evaluation, are comfortable navigating undefined problem spaces by proposing structure and driving clarity, and bring experience applying editorial and analytical judgment to evaluate content quality and inform product decisions, we encourage you to apply.
Responsibilities
- Define content quality standards and use them to systematically evaluate how AI models are performing across our products and content experiences.
- Design golden sets, taxonomies, and guidelines that integrate qualitative and quantitative signals.
- Build repeatable workflows for collecting, annotating, and analyzing AI outputs so that evaluations can run efficiently as models evolve.
- Evaluate successive model releases through structured comparison, documenting what improved, what regressed, and what to prioritize next.
- Develop processes to track content quality and model performance over time and flag regressions.
- Synthesize evaluation results into structured error patterns and concrete recommendations that engineering and product teams can act upon.
- Work cross-functionally with engineering, data science, and product teams to align AI behaviors with real-world user expectations.
- Identify and mitigate operational risks — staffing gaps, timeline conflicts, quality regressions — before they impact delivery.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Minimum Qualifications
- 8+ years of experience working collaboratively with product, engineering, design, and user research teams
- 1+ years of experience working with generative AI products, AI evaluation, prompt engineering, annotation, and/or content labeling and analysis
- Experience designing and implementing evaluation frameworks, annotation guidelines, or quality rubrics for AI/ML systems
- Demonstrated data analysis skills, with experience exploring data, identifying patterns, and producing actionable insights
- Experience applying critical thinking to lead data-driven analyses that inform product or content decisions, and communicating findings to executive leadership
- Proven track record of cross-functional collaboration and delivering results in environments with evolving requirements and competing priorities
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Preferred Qualifications
- Experience with Python, SQL, or other tools for data analysis and evaluation automation
- Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy review)
- Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
- Familiarity with AI evaluation methods such as human evaluation, model-as-judge, A/B testing, or red teaming
- Experience building dashboards, scripts, or workflows that codify evaluation metrics
- Background in content understanding, search quality, recommendation systems, or trust and safety
- B.A. or B.S. in Computer Science, Data Science, Linguistics, or a related field
- Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location