Rodeo
Get started

Jobgether

Evaluations Team Lead

UK
Posted about 10 hours ago
Sign up to applySee more jobs like this
Get notified of more jobs like this · No spam, ever

How your CV stacks up

1Upload CV
2Analyse CV
3Improve CV

Upload your CV to see how well it fits this job role

?%

Evaluations Team Lead

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an Evaluations Team Lead based in United Kingdom.

As an Evaluations Team Lead, you will shape how advanced AI models are evaluated, validated, and improved in production environments. You will lead the development of a shared evaluation platform that enables research, engineering, and customer-facing teams to measure model performance with confidence. Your work will establish rigorous standards for benchmarking, statistical validation, data quality, and reproducibility. Combining hands-on technical leadership with people management, you will build a high-performing team while remaining closely involved in code, experiments, and methodology. You will assess predictive quality, inference latency, and cost efficiency to guide research priorities and product releases. This is an opportunity to define evaluation practices from the ground up and directly influence the reliability and impact of next-generation enterprise AI.

Accountabilities:

  • Build and maintain a shared evaluation platform that supports research, engineering, and Applied AI teams, enabling independent benchmarking and investigation of model performance.
  • Develop and continuously improve benchmark suites, curated datasets, and model comparisons to assess predictive quality across a broad range of business use cases.
  • Establish rigorous evaluation standards covering statistical metrics, validation strategies, data splits, leakage prevention, calibration, uncertainty estimation, and benchmark contamination.
  • Design reproducible evaluation pipelines with version-controlled inputs, datasets, and artifacts, integrating automated regression checks into research and release workflows.
  • Benchmark models against competing approaches using fair and consistent protocols for hyperparameter tuning, data access, computational resources, and end-to-end latency measurement.
  • Evaluate predictive performance, inference latency, and cost efficiency across deployment configurations, task types, and dataset characteristics.
  • Support customer proof-of-concept evaluations by providing reliable tooling, methodological guidance, and data-driven analysis to help demonstrate model value.
  • Translate evaluation findings into actionable research priorities, release recommendations, and evidence-based improvement plans for customer use cases.
  • Recruit, lead, and develop a small team of evaluation specialists, setting priorities, providing technical coaching, and delivering constructive performance feedback.
  • Remain hands-on with Python development, experimental design, statistical analysis, and evaluation methodology while fostering collaboration across research, engineering, and customer-facing teams.

Reasons to use Rodeo

I’m in my final year doing Economics and I don’t know whether to apply for grad schemes now or do a masters first. What do you think?

Honest answer — it depends on where you want to end up. A lot of top grad schemes (Big 4, civil service, banking) don’t need a masters. Let’s look at the ones you’d be competitive for now, and we can decide if a masters actually adds anything.

Also worth knowing: most autumn 2026 applications are open now. Timing matters more than you think.

Start with a chat, not a search bar

Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.

P

Graduate Consultant — 2026 Scheme

PwC·London, UK
£35,000/yr

Why you're a good match

Strong

Your economics background and your summer at a regional bank line up with what PwC looks for on the consulting scheme. Applications close in four weeks.

See breakdown
Save jobNot relevant
View details

It searches the market for you

Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.

Why you're a good match

You’ve got the grades and the economics background, and your bank internship is exactly the experience this scheme looks for. Apply soon — deadlines close within the month.

See breakdown
Strong

Experience fit

Your summer at the bank plus your econometrics coursework map directly to the day-one responsibilities on this scheme — client modelling, market briefings, and deal support.

See breakdown
Strong

Only hits

No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.

Requirements:

  • Proven experience owning the evaluation of tabular machine learning systems used in production environments or high-impact customer decision-making.
  • Strong statistical expertise, including selecting appropriate metrics and validation schemes, estimating uncertainty, comparing models across datasets, and accounting for repeated experimentation.
  • Practical experience identifying and preventing data leakage involving preprocessing, feature engineering, joins, temporal dependencies, and related entities across training and evaluation splits.
  • Strong proficiency in Python and SQL, with familiarity with scikit-learn, gradient-boosted decision trees, and the development of reliable ML tools or platforms used by multiple teams.
  • Demonstrated experience designing fair and reproducible model comparisons, including hyperparameter optimization, resource allocation, and end-to-end inference latency measurement.
  • Previous people management experience, including hiring, technical mentoring, team development, and performance management, combined with a willingness to remain actively involved in technical work.
  • Excellent written and verbal communication skills, with the ability to collaborate effectively with researchers, software engineers, and customer data scientists.
  • A strong analytical mindset and the confidence to challenge unsupported conclusions, communicate uncertainty, and ensure that technical decisions are grounded in reliable evidence.
  • Experience evaluating tabular foundation models or automated machine learning (AutoML) systems is an advantage.
  • Familiarity with measuring the impact of model optimizations on predictive performance and inference efficiency is desirable.
  • Experience working with relational or multi-table datasets and data platforms such as Snowflake or Databricks is a plus.
  • Knowledge of automated or agent-driven machine learning workflows, including identifying failure modes that aggregate metrics may conceal, is beneficial.

Get help with your application

Your very own career expert that helps elevate your application to the next level.

Get help applying for this job

Benefits:

  • Competitive compensation package combining salary and equity.
  • Comprehensive health coverage for employees and their dependents.
  • Paid parental leave for all new parents, including adoptive and surrogacy journeys.
  • Relocation assistance for employees moving to join the team at one of the company's office locations.
  • Remote working opportunities across eligible locations in Europe and Israel.
  • The opportunity to establish evaluation standards and build a new function from the ground up, with substantial technical ownership and influence.
  • A mission-driven, collaborative, and low-ego culture that values diverse perspectives, individual ownership, and proactive decision-making.
  • The opportunity to work on advanced AI research and help improve the reliability, predictive performance, and efficiency of enterprise AI systems.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

Trusted by 25,000+ job seekers

“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”

Jessica, London

Get help applying for this job

Location

United Kingdom

Sign up to applySee more jobs like this