JetBrains
Research Engineer (LLM Training and Performance)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Research Engineer (Mellum LLM Training and Scaling)
At JetBrains, code is our passion. Ever since we started back in 2000, we have been striving to make the strongest, most effective developer tools on earth. By automating routine checks and corrections, our tools speed up production, freeing developers to grow, discover, and create.
We’re looking for a Research Engineer who will own the training stack and model architecture for our Mellum LLM family. Your job is challenging: make training faster, cheaper, and more stable at large scale. You’ll profile, design, and implement changes to the training pipeline—from architecture to custom GPU kernels, as needed.
**### About the Role
As part of our team, you will:
- Be responsible for improving end-to-end performance for multi-node LLM pre-training and post-training pipelines.
- Profile hotspots (Nsight Systems/Compute, NVTX) and fix them using compute/communication overlap, kernel fusion, scheduling, etc.
- Design and evaluate architecture choices:
- Depth/width trade-offs
- Attention variants including GQA (Grouped Query Attention), MQA (Multi-Head Attention variants), MLA, Flash-Style
- RoPE (Rotary Position Embeddings) scaling/NTK (Neural Tangent Kernel)
- Mixture-of-Experts (MoE) routing and load balancing
- Implement custom ops (Triton and/or CUDA C++), integrate via PyTorch extensions, and upstream when possible.
- Push memory/performance levers:
- FSDP (Fully Sharded Data Parallelism)/ZeRO (Zero Redundancy Optimizer)
- Activation checkpointing, FP8/TP (Tensor/Transposed tensor)
- Tensor pipelining, sequence parallelism, expert parallelism
- NCCL (NVIDIA Collective Communications Library) tuning
- Hardening large runs via:
- Elastic and fault-tolerant training setups
- Robust checkpointing and reproducibility
- Preemption resilience
- Keep the data path fast using:
- Streaming and sharded data loaders
- Tokenizer pipeline optimisations
- Improved overall throughput and cache efficiency
- Define the right metrics, build dashboards, and deliver steady improvements.
- Efficiently run both pre-training and post-training (SFT, RLHF, GRPO) across large-scale clusters.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
### Requirements
Essential:
- Strong PyTorch and PyTorch Distributed experience:
- Experience running multi-node jobs with tens to hundreds of GPUs.
- Hands-on experience with:
- Megatron-LM/Megatron-Core/NeMo,
- DeepSpeed, or serious FSDP/ZeRO expertise.
- Real profiling expertise with:
- Nsight (Systems/Compute), NVTX, nvprof.
- GPU programming skills, particularly:
- Triton and/or CUDA C++.
- Ability to write, test, and debug kernels.
- Solid understanding of NCCL collectives, including:
- Topology and fabric effects (InfiniBand, RoCE).
- Profile-aware optimisation (e.g. trace-based adjustments).
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Preferred:
- Experience with:
- FlashAttention-2/3, CUTLASS, CuTe, TransformerEngine, FP8.
- MoE at scale (expert parallelism, router loss handling, capacity management).
- Long-context optimisation (ALiBi (Attention with Linear Biases), YaRN, NTK scaling).
- Kubernetes/SLURM in large environments, including placement and affinity tuning.
- Cloud GPU fleets (AWS, GCP, Azure).
- Web-scale data plumbing (streaming datasets, Parquet, TFRecord, tokeniser performance), benchmarking frameworks.
- Safety and post-training methods (DPO, ORPO, GRPO, reward modelling).
- Inference optimisation (vLLM, paged KV caching).
### Benefits
We value diversity, inclusion, and innovation, fostering an environment where everyone can thrive.
- Equality of opportunity: Everyone’s uniquely valuable contribution is welcomed, regardless of background or identity.
- Privacy-conscious recruitment: Data is processed in accordance with the [JetBrains Recruitment Privacy Policy].
"Great ideas can come from anyone, anywhere."
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location