X4 Technology
Contract Senior Data Engineer (Databricks)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Role: Senior Data Engineer (Azure Databricks)
Programme: Azure Databricks Platform
Industry: Commodities / Energy Trading
Location: London (Hybrid)
Contract: Initial 6-month contract (long-term transformation programme)
Rate: £850-900/day (Inside IR35)
X4 Technology are partnered with a leading global energy trading organisation delivering a major technology transformation centred on the migration from a legacy ETRM platform to Endur and the modernisation of its enterprise data platform using Azure Databricks.
As part of this multi-year programme, the organisation is migrating Oracle-based data platforms into Azure Databricks to build a scalable, cloud-native data platform supporting Endur, global reporting, analytics and risk management, while reducing critical data latency from 10–15 minutes to near real-time.
They are looking for an experienced Senior Data Engineer to play a key role in designing and building the Azure Databricks platform. This role combines deep hands-on engineering with technical ownership, helping define engineering standards, optimise platform performance and shape the future architecture of a business-critical data platform.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Responsibilities of the Senior Data Engineer (Azure Databricks)
- Design, build and optimise scalable Azure Databricks data pipelines and platforms using PySpark, SQL, Delta Lake and Structured Streaming.
- Lead the migration of legacy Oracle data platforms into Azure Databricks, delivering high-performance data solutions for trading, reporting and analytics.
- Drive engineering best practices across performance, scalability, CI/CD, testing, governance and cloud cost optimisation.
- Collaborate with architects, platform teams and business stakeholders to shape technical solutions, mentor engineers and influence platform direction.
Requirements for the Senior Data Engineer (Azure Databricks)
- Extensive hands-on experience delivering enterprise-scale Azure Databricks data platforms, with strong expertise in PySpark, Python, SQL, Delta Lake and Structured Streaming (Delta Live Tables desirable).
- Strong understanding of modern cloud data architecture, data modelling, performance optimisation, governance and data engineering best practices.
- Experience working on large-scale cloud migration or platform modernisation programmes, with the ability to influence technical direction and mentor engineering teams.
- Experience within energy trading, commodities, capital markets or financial services is advantageous but not essential.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
This is an opportunity to work on one of the largest Azure Databricks transformations in the energy trading sector, helping build a modern cloud data platform that underpins trading, risk, reporting and analytics. You'll have significant influence over technical decisions while remaining deeply involved in hands-on engineering within a high-performing team.
Interviews are being scheduled immediately, with a view to onboarding as soon as possible. Apply now for immediate consideration.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location