JetBrains
Research Engineer (Agentic Models)
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Agentic Models Team – Research Engineer
About the Role
At JetBrains, code is our passion. Since 2000, we have strived to deliver the strongest, most effective developer tools on earth. Today, AI-powered assistants and agents are becoming integral to how developers work within our IDEs.
The Agentic Models team is building multi-step coding agents capable of understanding large codebases, planning changes, calling external tools, and iterating with users. As a Research Engineer in this team, you will focus on developing the models, training pipelines, and evaluation systems that drive these agents.
You will work at the intersection of supervised fine-tuning (SFT), reinforcement learning (RL) post-training, and product-driven evaluation, leveraging our distributed GPU and MapReduce clusters to ensure models are effectively integrated into JetBrains products.
Key Responsibilities
Here’s what you’ll be doing:
Model & Pipeline Development
- Design, implement, and maintain SFT and RL post-training pipelines for multi-step coding agents.
- Train and adapt large language models (LLMs) for agent workflows, including planning, tool usage, and multi-step interactions within JetBrains IDEs.
Evaluation & Simulation
- Build and develop evaluation and simulation environments where coding agents can execute realistic developer tasks, allowing performance measurement and comparison.
- Develop evaluation frameworks and metrics for agent behaviour, analyse traces and logs, and iterate based on feedback from training, data, and reward design phases.
Performance Analysis & Model Optimisation
- Examine training and evaluation results, proposing and implementing improvements in architectures, training methods, and datasets.
- Collaborate to introduce enhancements in product-driven evaluation methodologies.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Technical & Infrastructure Work
- Work with large-scale infrastructure, including distributed training on GPU clusters and MapReduce-style data processing for pre-training and fine-tuning datasets.
- Handle issues related to data pipelines, batching, mixed precision training, and debugging unstable training runs.
Product & Team Collaboration
- Work closely with research, product, and infrastructure teams to convert high-level product visions into models, experiments, and shipped features.
Requirements & Experience
You’ll fit this role if you have:
- Extensive experience in training LLMs (pre-training, fine-tuning, or post-training) in research or production settings.
- Deep expertise in modern deep learning frameworks, particularly PyTorch, and experience with specialised LLM training stacks (e.g., Megatron, NeMo, Deepspeed).
- A rigorous understanding of LLM fundamentals, including:
- Architectures (Transformer models, attention mechanisms)
- Tokenization approaches
- Data pipelines, batching, mixed precision training
- Distributed training techniques
- Debugging strategies for unstable training runs
- The ability to own end-to-end projects: starting from identifying a product problem, designing a solution, setting up the experimental pipeline, and iterating based on results.
- A product-aware mindset: passion for developer experience and the ability to translate product needs and failure modes into effective modeling and evaluation strategies.
- At least 3 years of Python experience, writing clean, maintainable code in modern ML ecosystems.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Desired Experience
It will strengthen your candidacy if you have:
- Experience with ML orchestrators and workflow tools (e.g., Kubeflow, Dagster, Airflow, ZenML) or core job schedulers (e.g., Kubernetes, SLURM) for managing large-scale experiments.
- Familiarity with building and optimising large-scale data and training pipelines, including:
- MapReduce-style clusters and multi-node GPU training
- Workloads involving significant computing power, e.g., 1M+ CPU/GPU hours
- Experience with LLM or agent evaluation pipelines, including:
- Metrics design and dashboards
- Experiment tracking (e.g., Weights & Biases, MLflow)
- Automated regression checks and log analysis
- AI agent development (e.g., tool-using agents, planners, or multi-step coding workflows) and knowledge of agentic frameworks or design patterns.
- Experience with:
- Inference optimisation and deploying high-performance models in production.
Why JetBrains?
At JetBrains, we believe that amazing developer tools are built by passionate experts who care deeply about the code their products enable. Join a fully remote team that values contribution from everywhere and thrives on creating inclusive environments.
We currently offer no location bias, so you’ll work from where you’re most effective.
We commit to the following for all talented candidates:
- Equal Opportunity: We believe diverse perspectives lead to outstanding ideas. Our workplace understands and celebrates your background and who you are.
- Workplace Inclusivity: We actively value background, identity, age, accessibility needs, and orientation and are committed to fostering an inclusive culture where everyone can contribute their best self.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location