Prolific
Senior Product Engineer (Agent Training and Evals)
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Engineering
About Prolific
Prolific is not just another player in the AI space – we are the architects of the human data infrastructure that's reshaping the landscape of AI development. In a world where foundational AI technologies are increasingly commoditized, it's the quality and diversity of human-generated data that truly differentiates products and models.
The Role
As a Senior Product Engineer, you'll be one of the early engineers in our Agentic Training & Evals team.
You'll build the infrastructure that lets us evaluate AI agents, capturing agent traces at scale alongside real human expert work in safe and secure environments.
This is a genuine 0→1 role. There isn't a well-established playbook or existing architecture for you to simply maintain. You'll have significant influence and ownership over the technical direction of the team and the products we build.
You'll work closely with engineering, product, and AI research to turn ambiguous problems into scalable systems, and you'll see the impact of what you build directly.
What you’ll bring to the role
- Strong experience building scalable infrastructure or distributed systems in the cloud
- Solid understanding of how systems are deployed, scaled, and secured
- Hands-on experience with integrating and evaluating AI models or agents
- Interest in how AI agents are trained, evaluated, and improved
- Bonus: experience with image, video, or audio processing
- Strong product mindset with a focus on delivering business and customer value
- Comfort with ambiguity, and the ability to move quickly from idea to working system
- Strong communication and collaboration skills
- Python experience is useful, but not essential
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What you’ll be doing in the role
- Build tooling for capturing and processing data from agents and humans performing real-world tasks, at significant scale
- Solve hard problems around compute, orchestration, scaling, security, and reliability
- Help develop approaches for training, benchmarking, and evaluating AI agents
- Work with emerging agentic AI technologies and rapidly evolving models
- Make pragmatic technical decisions in a highly ambiguous environment
- Partner closely with product engineering and AI research to turn customer problems into working products
- Drive direct impact in a fast-moving, high-ownership, cross-functional team
Key Technologies
- Cloud Platforms: Google Cloud Platform
- Programming Languages: Python, JavaScript, and TypeScript
- Frameworks: FastAPI, Vue.js
- Container-based and Serverless architectures
- Databases: Postgres SQL
- DevOps and Monitoring: GitHub Actions, Kubernetes, DataDog
Why Prolific is a great place to work
We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breadth and the best of humanity.
Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research. Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture.
Links to more information on Prolific
- Benefits
- External Handbook
- Website
- Youtube
- Privacy Statement
By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organization planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location