White Circle
Head of Data Labeling
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
About Us
White Circle is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies – simple natural-language rules that define what an AI model should and shouldn’t do. We automatically test, enforce, and continuously improve these policies at scale.
We’ve raised $11M from top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others. We process over 100M+ API calls every month. We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model.
We’re at a multi-million dollar run rate already having signed customers like Lovable and multiple neobanks. You’ll be joining at the most exciting time - early enough that the equity can be life changing but at a point where demand has been proven.
In this role, you will
- Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)
- Define annotation guidelines, quality standards, and evaluation frameworks
- Develop quality assurance processes, calibration sessions, and auditing systems
- Partner with AI researchers and engineers to translate research objectives into labeling workflows
- Prioritise labeling projects based on business and research needs
- Monitor operational metrics including quality, consistency, throughput, and cost
- Improve annotation tooling, automation, and workflow efficiency
- Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation
- Analyse disagreement patterns and edge cases to improve guidelines and model performance
- Manage vendor relationships and ensure consistent quality across distributed teams
- Build reporting dashboards and communicate operational insights to leadership
- Foster a culture of continuous improvement, accountability, and operational excellence
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
We're looking for someone who
- Has experience leading data annotation or AI evaluation teams
- Has strong operational and people management skills
- Understands AI model evaluation, LLM behavior, and modern annotation workflows
- Can design scalable processes without sacrificing quality
- Communicates clearly across technical and non-technical teams
- Thrives in fast-moving startup environments
You might be a great fit if you
- Have managed annotation programs for LLMs, generative AI, or machine learning
- Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation
- Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops
- Have managed distributed or global annotation teams
- Have experience with vendor management and outsourcing operations
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Bonus points
- Familiarity with prompt engineering and AI safety policies
- SQL, Python, or data analysis experience
- Experience building internal annotation platforms or workflow automation
- Background in linguistics, cognitive science, machine learning, or data operations
Important note
This role involves overseeing projects that may include offensive, harmful, violent, sexual, or otherwise disturbing content.
You'll be responsible for ensuring reviewers have the tools, guidance, and support necessary to perform this work safely and consistently.
Why White Circle
- Competitive salary + equity
- Work from Paris (hybrid) with a relocation package available, or work from London (note: we are currently unable to provide relocation support and medical insurance for London-based roles)
- Paid time off in line with your local regulations
- All the hardware, tools, and services you need
- Covered subscriptions for AI agents and IDEs
- Team off-sites twice a year: we’ve recently been to the Alps and Saint-Tropez
How we hire
- Intro call with HR (30 min)
- Take-home exercise
- Final conversation with our CEO (45 min)
Please submit your application in English.
Compensation Range: $60K - $100K
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location