Steven.com
Senior Product Manager - Tech, Experimentation & Failure
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Product Manager - Tech, Experimentation & Failure
Company: STEVEN.COM
Reporting to: CITO
Location: LONDON
About Steven.com
Steven.com is building the operating system for the creator economy, forecast to pass a trillion dollars by the early 2030's. This industry is currently held back by fragmented tools and a lack of professional infrastructure. Steven.com is the unlock. We are the end-to-end Operating System designed to scale what is irreplaceably human.
We have built proprietary technology and obsessed teams to identify and scale the highest-potential creators across our core pillars:
- Creator Media: Amplifying reach, influence, and trust
- Creator Community: Transforming audiences into connected tribes
- Creator Products: Providing creators with the infrastructure to build and back aligned products and ventures
- Powered by Creator Tech & Intelligence: A proprietary data and technology suite that fuels smarter decisions and drives innovation across the entire flywheel.
Our Experimentation & Failure team has one of the most strategically important and unusual mandates at Steven.com: increase the rate of failure. As Steven Bartlett puts it: "the path to the correct answer is out-failing your competition." This isn't a growth team or an optimisation function. It's the team that exists to make sure we learn faster than anyone else.
Role Mission
Lead the Experimentation & Failure team, reporting to the CITO. You'll out-experiment and out-fail the competition - running high-velocity, rigorous experiments across every show, creator, piece of content, and commercial bet at Steven.com, while building a team and a culture that treats deliberate failure as the primary learning mechanism. This is deeply hands-on: you'll be setting hypotheses, isolating variables, checking statistical power, reading results, and moving to the next test - not directing from a distance.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Key Outcomes
- Own and drive the experimentation agenda across every show, creator, and IP property in the FlightStory portfolio - from podcast topic selection and episode structure through to thumbnail design and social tile copy. No detail too small to test.
- Lead hypothesis formation for every experiment, with clear success, failure, and inconclusive criteria defined in advance - and enforce single-variable discipline across the board.
- Ensure every experiment is adequately powered before launch: sample sizes calculated, measurement windows defined, results interpretable by design.
- Systematically increase experimentation velocity and build the intake process that makes high-volume testing the default across FlightStory and Steven.com.
- Build institutional memory - a searchable, structured record of every experiment run, what was learned, and what was decided - as a compounding organisational asset.
- Partner with the VP of Engineering & Applied AI to apply AI tooling to experiment design, analysis, and reporting, and to ensure infrastructure supports testing at this velocity.
Core Competencies
- Deep, first-principles command of experimentation mathematics: statistical significance, power, sample size calculation, p-values, confidence intervals, Type I/II errors, and the difference between statistical and practical significance.
- Genuine mastery of the scientific method applied to product and content - hypothesis formation, single-variable isolation, measurement design, and result interpretation.
- A track record of building experimentation culture, not just running tests - creating an environment where the whole team experiments and failure is rewarded.
- Comfortable querying data and working shoulder-to-shoulder with engineers and data scientists at implementation depth.
- Strong written and verbal communication - able to write a hypothesis an engineer respects and explain a result a producer will act on.
- Experience operating at pace, in high-volume testing environments where speed of learning is the competitive edge.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
You'll Thrive Here If
- You think in hypotheses, not features.
- You isolate one variable at a time, and understand viscerally why changing five things at once makes a result meaningless.
- You give a null or inconclusive result the same intellectual respect as a win - you know how to extract the signal either way.
- You're obsessive about measurement: an experiment that can't be measured is just a change, not an experiment.
- You're deeply sceptical of your own results, and design experiments to prove yourself wrong rather than confirm what you already believe.
- You move fast, expect others to move fast, and don't wait for perfect conditions to run a test.
- You believe failure is feedback, feedback is knowledge, and knowledge is power - and you build systems to generate that knowledge at the highest possible rate.
Ideal Background
- Demonstrable experience leading (not just participating in) product or content experimentation programs at a technology, media, or creator-economy company - owning the methodology, volume, and culture.
- Strong advantage: experience with algorithmic platforms and how to design experiments against platform-specific metrics; podcasting, video, social, or creator-economy background; familiarity with YouTube/Spotify/social analytics (CTR, retention, watch time, audience behaviour).
- Nice to have: experience building an experimentation platform from scratch; familiarity with causal inference beyond standard A/B testing (holdouts, quasi-experiments, diff-in-diff); experience experimenting on AI/ML systems or prompt variations in production; a background in statistics, maths, CS, economics, or a natural science; exposure to early-stage environments where you had to build the experimentation infrastructure yourself.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location