IntelStack
AI Test Lead
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Job Title: AI Test Lead
Job Type: Contract
Location: West Yorkshire — Hybrid
Duration: 6 Months
Role Purpose
The AI Test Lead is responsible for defining and leading testing and quality assurance practices for AI-enabled systems and solutions. Operating within the AI Centre of Enablement and AI Foundry delivery team, the role ensures that AI-powered capabilities are validated, reliable, and safe to operate within enterprise environments. The role addresses the unique challenges of testing AI systems, including probabilistic behavior, variability in outputs, and responsible AI considerations. Through robust testing frameworks and collaboration with delivery and governance teams, the role supports compliant and trustworthy deployment of AI solutions within a regulated financial services environment.
Responsibilities and Accountabilities
- Define and lead AI testing strategies by establishing quality assurance approaches tailored to AI-enabled systems, including machine learning models, large language models, and agent-based solutions.
- Develop AI testing frameworks and methodologies that account for non-deterministic behavior, probabilistic outputs, and variability in AI responses.
- Design and execute model and prompt evaluation by assessing model performance, response quality, retrieval accuracy, and alignment with intended use cases.
- Embed responsible AI and risk testing by defining test cases that assess bias, fairness, transparency, safety, and the effectiveness of AI guardrails.
- Implement automated AI testing capabilities by developing test automation and integrating testing activities into DevOps and MLOps pipelines.
- Support operational testing and monitoring by validating AI behavior in live environments, investigating anomalies, and supporting performance optimisation.
- Collaborate across delivery and governance teams to ensure testing practices align with engineering standards, regulatory expectations, and enterprise assurance requirements.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Key Outcomes and Performance Indicators
Success in this role is demonstrated through:
- AI solutions that meet defined quality, reliability, and safety standards prior to deployment
- Effective testing coverage of AI behavior, including edge cases and failure scenarios
- Early identification and mitigation of AI-related risks
- Automated testing integrated into delivery pipelines
- Clear evidence of compliance with responsible AI and governance expectations
Performance may be measured through test effectiveness, defect detection, reduction of post-deployment issues, quality of assurance artefacts, and stakeholder confidence in AI solution safety.
Skills and Capabilities
Technical Skills
- Strong experience in testing and quality assurance for complex systems
- Working knowledge of AI and machine learning technologies, including large language models and Retrieval-Augmented Generation (RAG)
- Experience designing evaluation frameworks for AI model behavior and performance
- Familiarity with test automation, DevOps pipelines, and MLOps practices
- Experience working with Microsoft Azure, Azure AI services, and Copilot-based solutions
- Working knowledge of MLOps, CI/CD pipelines, monitoring, and operational support of AI systems
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Professional and Behavioral Capabilities
- Analytical and risk-focused mindset
- Attention to detail and structured approach to quality assurance
- Ability to reason about AI uncertainty, variability, and edge cases
- Strong collaboration and communication skills, including engagement with non-technical stakeholders
Key Stakeholders and Interfaces
- AI Centre of Enablement and AI Foundry delivery teams
- AI Product Developers and engineers
- AI Design Analysts and product owners
- Enterprise Architecture and Platform Engineering teams
- Information Security, Risk, Compliance, and Governance teams
The role works closely with both technical and assurance stakeholders to explain AI testing outcomes, risks, and quality considerations.
Legal or Essential Requirements
Legal: None
Essential
- Ongoing development of knowledge in AI testing approaches, responsible AI, and quality assurance practices
- Adherence to organizational AI policy, security standards, and regulator
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London