Ascendion
AI Test Manager/Test Architect
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Job Description
Description
We are seeking an experienced AI Test Manager / Test Architect to lead and scale quality engineering capabilities across enterprise AI initiatives, with a strong focus on Generative AI and Agentic AI solutions. This role is responsible for defining and governing the end-to-end AI testing strategy, architecture, automation framework, and quality standards to ensure delivery of secure, scalable, reliable, and production-ready AI products. The individual will manage teams of AI Test Engineers while collaborating with Product, Engineering, Architecture, Security, and DevSecOps teams to drive quality transformation, Responsible AI adoption, and continuous improvement across the delivery lifecycle.
Your Role
- Lead and manage teams of AI Test Engineers across multiple AI delivery programs.
- Define and govern the AI testing strategy, operating model, quality standards, and delivery processes.
- Establish end-to-end testing approaches across Generative AI, Agentic AI, functional, integration, performance, and security testing.
- Build and scale AI-focused QE capabilities, accelerators, frameworks, and reusable assets.
- Drive automation excellence through scalable and reusable frameworks using Playwright and Python.
- Embed quality across the SDLC through shift-left practices, CI/CD integration, and DevSecOps quality gates.
- Implement shift-right quality engineering using production monitoring, observability, and continuous feedback mechanisms.
- Govern release readiness, defect management, quality metrics, and executive reporting to ensure predictable delivery outcomes.
- Collaborate with cross-functional stakeholders and mentor teams to drive continuous improvement and AI quality transformation.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Your Professional Experience
Must Have
- 12+ years of experience in Testing / Quality Engineering with 5+ years leading testing teams or large delivery programs.
- Proven experience testing in Artificial Intelligence, Machine Learning, Generative AI, and Agentic AI solutions.
- Experience implementing AI-powered QE platforms and enterprise-wide AI testing transformation initiatives.
- Strong expertise in defining and governing enterprise AI testing strategies, operating models, and quality frameworks.
- Hands-on experience in AI validation, including model evaluation, prompt testing, output assessment, and agent behavior validation.
- Strong automation experience with Playwright and Python, including building scalable and reusable automation frameworks.
- Drive innovation through AI-enabled test automation and productivity improvements.
- Deep understanding of QA methodologies including Agile, DevOps, Waterfall, and BDD.
- Experience embedding quality controls into CI/CD and DevSecOps pipelines.
- Strong knowledge of Responsible AI practices, including fairness, explainability, bias, privacy, and security testing.
- Experience managing test governance, defect triage, quality metrics, release readiness, and executive reporting.
- Expertise with test management and delivery tools such as Jira, Azure DevOps, or equivalent platforms.
- Strong stakeholder management, communication, leadership, and mentoring capabilities.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Desirable Experience
- Experience working within large enterprises, regulated industries, or consulting environments.
- Direct experience testing Agentic AI ecosystems, including multi-agent orchestration and interaction validation.
- Exposure to AI observability, production monitoring, and shift-right quality practices.
- Knowledge of data governance, data quality, and AI lifecycle management.
- Experience establishing or scaling Quality Engineering / AI Testing Centers of Excellence (CoE).
- Experience implementing cloud-native testing and enterprise DevSecOps practices.
- Familiarity with AI governance and enterprise operating models.
- Professional certifications such as ISTQB Foundation, ISTQB Agile Tester, ISTQB AI Tester, and cloud or AI/ML certifications.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location