Aubay UK
Tech Lead (Azure Databricks Data Platform)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Role Summary
Aubay UK is seeking an experienced Tech Lead (Azure Databricks Data Platform) to lead the design, development, and delivery of enterprise-scale data platform solutions leveraging the Azure Data Platform and Databricks ecosystem.
The successful candidate will be responsible for providing end-to-end technical leadership across data engineering initiatives, driving solution architecture, platform strategy, engineering standards, and delivery excellence. Working closely with business stakeholders, architects, product teams, and engineers, the Tech Lead will translate complex business requirements into scalable, high-performance data solutions that support critical business operations and analytics capabilities.
This role requires deep hands-on expertise in Azure Databricks, Structured Streaming, Delta Live Tables (DLT), PySpark, Python, and SQL, alongside strong stakeholder management and leadership skills. The ideal candidate will have a proven track record of delivering large-scale cloud data platforms, optimising platform performance and costs, and mentoring engineering teams to achieve successful business outcomes.
Required Skills and Experience:
- Strong hands-on experience with Azure Databricks, PySpark, Python, and SQL
- Experience within financial trading, commodities trading, or energy trading environments
- Understanding of trading data models and trading business processes
- Experience building and supporting enterprise-scale data platforms on Microsoft Azure
- Expertise in Structured Streaming and Delta Live Tables (DLT)
- Experience designing scalable, high-performance data solutions and architectures
- Strong understanding of data governance, data quality, and engineering best practices
- Experience leading technical delivery from design through to production support
- Strong stakeholder management and communication skills
- Experience leading and mentoring engineering teams
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Desired Skills and Experience:
- Experience optimising Databricks performance and managing cloud platform costs
- Knowledge of Lakehouse architecture and modern data platform design patterns
- Experience working within Agile delivery environments
- Relevant Azure and/or Databricks certifications
Roles and Responsibilities:
- Lead the design, development, and implementation of scalable data platform solutions using Azure and Databricks
- Provide technical leadership across the full solution lifecycle, from architecture through delivery and production support
- Define and promote best practices for data engineering, governance, security, quality, and platform standards
- Design and optimise real-time data processing solutions using Structured Streaming and Delta Live Tables (DLT)
- Collaborate with business stakeholders to translate business requirements into technical solutions
- Drive technical decision-making and ensure alignment with platform strategy and business objectives
- Monitor and optimise platform performance, reliability, scalability, and cloud costs
- Support the development of data architecture, integration patterns, and engineering roadmaps
- Mentor Data Engineers and provide technical guidance across delivery teams
- Ensure successful delivery of business outcomes through strong ownership, accountability, and leadership
- Work closely with architects, product teams, and stakeholders to manage technical risks, dependencies, and priorities
- Promote continuous improvement and adoption of modern data engineering practices across the platform team
About Aubay UK – Ahead of Innovation!
Aubay UK is a recognised InSourcing Partner for client-side deployment delivered across London. Our team, based in Canary Wharf, specialises in hiring IT professionals within London’s Energy and FinTech sectors, helping our clients to expand their operations with top-tier talent who are experts in their fields. We work exclusively with clients who are globally recognized as Energy Super Majors/Financial Services and innovative FinTech players.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Aubay UK is the most recently started branch of Aubay Group www.aubay.com. Aubay Group is an international Digital Services Company, listed on a Euronext Stock Exchange, who have been operating for 25 years in the European market and working alongside some of the biggest names in the Banking, Finance, Insurance, Energy, IT/Digital, Manufacturing, Transport and Telecoms sectors. With over 7,800 employees across 7 countries and 16 offices in England, France, Belgium, Luxembourg, Italy, Spain, and Portugal, Aubay Group generated revenues of €534 million in 2023.
Our Client
Our client is one of the Super Major global energy companies with around 84,000 employees across 70+ countries who are working to power progress through cleaner energy solutions.
Specialities: Upstream/Downstream, Biofuels, Integrated Gas, New Energies, Chemicals, Energy and Trading.
You will have the opportunity to work in a challenging but rewarding environment that is fast-paced and changing fundamentally and work towards transforming the business of a Super Major energy company to meet the ambition to be a net-zero emissions energy business by 2050 whilst delivering a world-class business case that has a robust societal license to operate. In your role, you will be expected to enact change and deliver value globally across business lines and geographies.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location