Insight International (UK) Ltd
Azure DataBricks Engineer
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Role Title: Azure DataBricks Engineer
Location: London, UK
Employment Type: Inside IR35 Contract
Required skills and experience
- 8-10 years of hands-on experience in data engineering space, with ability to translate working solution into implementable working package and/or built new solution using Azure platform
- High proficiency and experience in designing/developing data analytics and data warehouse solutions with Azure Data Factory (ADF) and Azure Databricks, services related to Azure Analytics, Azure data lake storage Gen2, Azure SQL/Postgres, Azure analytics services and good understanding of Azure identity (SPN, SAMI, UAMI etc.)
- Experience in designing large data distribution, integration with service-oriented architecture, data mesh, data Lake and data analytics solution using Azure data services with large and multi-format data
- Proficient coding experience using Spark (Python/Scala) and SQL
- Hands on experience with one of scripting language: Linux / PowerShell.
- Prior ETL development experience using industry tool e.g. informatica/SSIS/Talend etc., good to have knowledge in Kafka streaming Azure Infrastructure
- A proven track of work experience in cloud (MS Azure platform preferred) in an agile SDLC environment, leveraging modern programming languages, DevOps, and test-driven development
- Exposure to Docker, Git
- Hands on experience with pipeline orchestration tools like Apache Airflow, Autosys, Control-M – Preferred Apache Airflow
- Proficient at working with large and complex code bases using GitLab/Github, Fork/Pull Model and CI/CD
- Knowledge of DevOps tools and technologies (like GitLab CI/CD pipeline creation, YAML, Terraform, ARM, Puppet), Azure function app, logic app is a plus
- Working experience in Agile methodologies (SCRUM, XP, Kanban). Work within an agile SDLC environment to deliver high-quality data solutions
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
You should hold a relevant bachelor’s degree or equivalent. Excellent communication skills, with fluency in English for both presentations and technical writing.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location