RED Global
Databricks Data Engineer
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Job Title: Databricks Data Engineer
Location: London
Work Model: Hybrid (3 days on-site)
Start Date: ASAP
Contract Length: 12 Months
Contract Type: Inside IR35
About the Role
We are looking for an experienced Azure Data Engineer to join a long-term data engineering programme based in London. You will be responsible for designing, developing and optimising scalable data solutions using Azure Databricks, Azure Data Factory and Scala. The role will involve processing large-scale datasets, building reliable ETL pipelines and working closely with technical and business stakeholders to translate requirements into effective data solutions.
This is a hands-on engineering position suited to someone with strong Azure data platform experience and a proven background delivering high-performance data pipelines within complex enterprise environments.
Key Responsibilities
- Design and develop scalable data engineering solutions using Azure Databricks and Azure Data Factory.
- Build and maintain complex ETL and data transformation pipelines.
- Process and transform large datasets containing millions of records.
- Optimise data pipelines and distributed processing workloads for performance, reliability, and scalability.
- Participate in customer discussions, requirement-gathering sessions, and technical solution design.
- Translate business and technical requirements into practical data engineering solutions.
- Develop clean, maintainable, and reusable code in line with engineering best practices.
- Perform code reviews, performance tuning, and technical troubleshooting.
- Monitor delivery progress and communicate project updates to stakeholders.
- Identify and escalate technical risks, blockers, and critical issues.
- Collaborate with engineers, architects, and wider project teams within an Agile delivery environment.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Essential Skills and Experience
Get help with your application
Your very own career expert that helps elevate your application to the next level.
- 6–8 years of commercial experience within data engineering or data software engineering.
- Strong hands-on experience developing solutions using Azure Databricks.
- Proven experience processing and transforming large-scale datasets.
- Strong working knowledge of Azure Data Factory, including the development of complex ETL workflows.
- Experience optimising data pipelines and distributed data-processing environments.
- Hands-on development experience using Scala.
- Strong understanding of relevant Microsoft Azure data and cloud services.
- Experience designing scalable, reliable, and maintainable cloud-based data solutions.
- Strong SQL, data transformation, and troubleshooting skills.
- Excellent analytical and problem-solving ability.
- Strong communication skills and experience working with technical and non-technical stakeholders.
- Previous experience working within Agile delivery teams.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location