X4 Technology
Principal Data Engineer (Databricks)
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Role: Principal Data Engineer (Azure Databricks)
Programme: Data Exchange Platform
Industry: Commodities / Energy Trading
Location: London (Hybrid)
Contract: Initial 6-month contract (long-term transformation programme)
Rate: £850-950/day (Inside IR35)
X4 Technology are partnered with a global energy trading firm on a major transformation of its Data Exchange platform - the technical data backbone supporting physical commodities trading on a global scale.
The platform automates, reconciles and standardises transaction workflows across physical crude oil and refined petroleum products, giving traders a single, trusted source of truth across the entire ecosystem.
Joining as contract Principal Data Engineer, you'll play a pivotal role in the transformation - combining deep Azure Databricks expertise with genuine technical ownership across data architecture, streaming, integration and platform engineering.
Responsibilities for the Principal Data Engineer (Databricks)
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
- Own end-to-end data and platform architecture, shaping scalable, secure and high-performance solutions across multiple workstreams
- Lead design/delivery of the Databricks data platform - batch and real-time ingestion, streaming, data modelling, integration across Azure
- Set technical direction and engineering standards, driving best practice on performance, scalability, governance, security, testing and cost optimisation
- Be the go-to across Architects, Engineers, Programme Managers, Business Analysts and Trading stakeholders
- Spot and land opportunities for AI and intelligent automation - MLflow, Azure AI Foundry, Agent Bricks and Microsoft Copilot
Requirements for the Principal Data Engineer (Databricks)
- Extensive hands-on Azure Databricks expertise with a strong track record designing and delivering enterprise-scale data platforms (PySpark, Python, SQL, Delta Lake, Structured Streaming, DLT)
- Proven ownership of data platform architecture - data modelling, integration, governance, security, performance and cost optimisation
- Experience leading complex, greenfield or large-scale data transformation programmes across multiple workstreams and teams
- Strong real-time data and integration experience, including streaming architectures and complex data flows (Kafka beneficial)
- Excellent technical stakeholder management and influencing
Get help with your application
Your very own career expert that helps elevate your application to the next level.
High-profile seat at the heart of a global trading data ecosystem. You'll combine hands-on data engineering with real technical influence, shaping a highly secure, cloud-native data platform that connects trading desks, physical assets and external partners
Initial 6-month contract with multi-year programme scope, £850-950/day (inside IR35), hybrid in London. Interviews are being scheduled immediately, with a view to onboarding as soon as possible, so please apply now for immediate consideration.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location