Lorien
Tester - AI Initiatives
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Tester, AI Initiatives
Contract Length: 6 Months
Location: London (hybrid)
About the opportunity
Our client, a leading international events and media group, is looking for an experienced Tester to play a key role in delivering AI initiatives across Sales and Marketing. You'll make sure what ships works, is reliable and can be trusted by the commercial teams who use it every day.
Testing AI is different from testing traditional software. Outputs vary, models can be confidently wrong, and quality depends as much on data and prompts as on code. You'll bring a strong testing foundation and the curiosity to apply it to AI, setting the standard for how quality is assured as the team grows.
About the team
You'll join the Commercial AI team, which builds AI products and workflows across a portfolio of events and media brands. It's an established team that is growing to meet critical demand for AI delivery across the business.
You'll work closely with engineers, a Business Analyst, product owners and commercial stakeholders, helping take AI ideas from concept to live products that teams can rely on.
Key responsibilities
- Own the test approach for each initiative, from early review of requirements through to release
- Review user stories and acceptance criteria with the Business Analyst before build, so they are clear and testable
- Design and run functional, integration, regression and user acceptance testing across web apps, APIs and data flows
- Test AI features for accuracy, consistency and safe behavior, including edge cases, unexpected inputs and how the system handles wrong or low-confidence outputs
- Build and maintain test datasets and evaluation sets that reflect real sales and marketing scenarios
- Develop automated tests where they add value, and help build testing into the team's CI/CD pipeline
- Check that data moving between systems, such as CRM and enrichment tools, is complete and correct
- Log, track and retest defects in Jira, and help engineers get to the root cause quickly
- Plan and support UAT with business users, and give a clear view of release readiness and risk
- Help shape testing standards and practices as the team grows
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What you'll deliver
- A test strategy and plan for each initiative, covering scope, approach, environments and risk
- Test cases and scenarios linked to user stories and acceptance criteria in Jira
- Evaluation sets and quality measures for AI features, with results tracked from release to release
- An automated regression suite that runs as part of each release
- Clear defect reports with steps to reproduce, severity and status
- UAT plans, scripts and business sign-off
- Release readiness reports that give a clear go or no-go view, with known issues and risks
About you
Essential
- Degree in computer science, a technical subject or a related field, or equivalent experience
- ISTQB Foundation or equivalent testing qualification (Advanced level desirable)
- Substantial experience as a Tester or QA Engineer in an agile delivery team
- Experience testing AI or machine learning products, such as LLM-based tools, chatbots, agents or scoring models
- A practical understanding of how AI behaves: variable outputs, hallucination, bias, prompt sensitivity and the role of data quality
- Strong manual testing skills, with a sharp eye for edge cases and how users will really use a product
- Experience with test automation tools and frameworks, such as Playwright, Cypress, Selenium or Pytest
- Confidence testing APIs and integrations, using tools such as Postman
- Experience managing tests and defects in Jira or a similar tool
- Clear communication, with the ability to explain quality risks to both technical and business audiences
- A proactive, hands-on approach: happy to work with incomplete information, switch between priorities and find the answer yourself where one doesn't exist
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Desirable
- Experience with LLM evaluation approaches or tools, such as golden datasets, scoring rubrics or automated evaluation frameworks
- Good SQL and data skills, to check data accuracy across systems
- Experience testing sales and marketing systems, such as Salesforce, marketing automation or data enrichment tools
- Some Python or JavaScript scripting
- Experience with performance or security testing
- Background in B2B, events or media businesses
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location