Luma | Dream Lab
Research Scientist / Engineer — Multimodal Agent
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior ML Research Scientist, Multimodal Foundation Models
About Luma AI
Luma’s mission is to build multimodal AGI. We believe AI needs to be jointly trained across all signal modalities—text, video, audio, and images, mirroring the structure of the human brain.
At Luma, we develop and operate every component of the stack, from foundational models to deployment systems and end products. Our integrative approach drives innovations like Ray3, now adopted at scale by Fortune 500 firms in media, entertainment, and advertising. With a recent $900M Series C and a 2 GW compute supercluster partnership (Project Halo), our Dream Machine platform enables creators globally to produce groundbreaking content.
Where You Come In
This position offers a unique and foundational opportunity to shape the future of multimodal AI. You’ll work at the forefront, designing and training large-scale multimodal models that redefine how users engage with digital and physical worlds.
This role blends cutting-edge research and hands-on engineering to solve novel challenges with no established framework—bringing breakthrough ideas from lab to real-world impact.
What You'll Do
Your responsibilities span four key areas, equally weighted as both expert and innovator:
Modeling
- Build large-scale multimodal agentic models capable of:
- Advanced reasoning and problem-solving
- Multi-step planning
- Coding capabilities integrated into multimodal tasks
- Tool calling, enabling seamless interaction with external systems
- Develop architectures to achieve complex goals using blending AI and user-facing workflows.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Data
- Task development: Redesign existing modalities or introduce new multimodal challenges; hillclimb tasks for model improvement.
- Multimodal data pipelines:
- Design, implement, and optimize robust systems for constructing, enriching, and filtering promasive multimodal datasets (e.g., images, video, text).
- Address scalability, quality, and bias mitigation across modalities.
- Fuel provenance and multi-phase evaluation strategies using diverse synthetic and real-world data.
Systems
- Scale high-efficiency training and inference pipelines for:
- Large-scale foundational models across multiple data modalities, including video, audio, and images.
- Mixed-precision training strategies to maximize compute utilization.
- Advanced distributed training, particularly for data-center ADP procedures.
- Scientifically rigorous balancing of fidelity and performance.
Evaluation
- Design and build novel evaluation frameworks for multimodal agents, including:
- Emergent capabilities in reasoning, coding, and tool-use tasks.
- Context-aware and trajectory-efficient reasoning and problem solving.
- Automated or semi-automated assessments that span vision, audio, and language understanding.
What You Bring
Core Qualifications
We seek professional Machine Learning researchers with:
- A strong foundation in machine learning, foundation models, and agentic systems.
- Deep expertise in advancing:
- Reasoning architectures using LLMs/VLMs.
- Coding models in multimodal contexts.
- Tool-calling for interaction with external systems.
- Hands-on expertise in:
- PyTorch and parallel training frameworks.
- Large-scale training systems, including:
- Cluster-level pipelines (e.g., using PyTorch/DDP).
- Memory management and precision tearing.
- Iteratively adapting models from edge devices to full data centers.
- Distributed programming and load balancing techniques.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Preferred Experience (Bonus)
Hands-on work that accelerates LLMs/VLMs in:
- State-of-the-art reasoning enabling generative multistep reasoning across modalities.
- State-of-the-art coding for execution-focused capabilities.
- State-of-the-art tool-calling modeling unstructured instruction workflows.
- Multimodal agents pushing-forward practical tests, evaluations, and applicability.
Compensation & Benefits
You can expect a competitive compensation package within the range of $250,000 – $450,000 per year.
About Luma
Luma is on a mission to create unified general intelligence—capable of generating, understanding, and operating in the physical world. Vision-based multimodal systems will be critical to surpassing pure language models, enabling AI to understand and explain the seen world and ultimately interact with it.
By joining Luma, you aren’t just choosing a job; you’re designing how technology reshapes human capabilities, shapes the next wave of creativity, and transforms the frontier of AI.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location