X4 Technology
Contract Principal Data Engineer (Databricks)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Role: Principal Data Engineer (Azure Databricks)
Programme: Data Exchange Platform
Industry: Commodities / Energy Trading
Location: London (Hybrid)
Contract: Initial 6-month contract (long-term transformation programme)
Rate: £850-950/day (Inside IR35)
X4 Technology are partnered with a global energy trading firm on a major transformation of its Data Exchange platform - the technical data backbone supporting physical commodities trading globally.
The platform automates, reconciles and standardises transaction workflows across physical crude oil and refined petroleum products, providing a single, trusted source of data across the trading ecosystem.
Joining as contract Principal Data Engineer, you'll play a key role in the continued development of the Data Exchange platform, combining deep Azure Databricks expertise with technical ownership across data architecture, streaming, integration and platform engineering.
Responsibilities for the Principal Data Engineer (Databricks)
- Own the end-to-end data and platform architecture, shaping scalable, secure and high-performance solutions across multiple workstreams
- Lead the design and delivery of the Databricks data platform, covering batch and real-time ingestion, streaming, data modelling and integration across Azure
- Set technical direction and engineering standards, driving best practice across performance, scalability, governance, security, testing and cost optimisation
- Act as the technical focal point across Architects, Engineers, Programme Managers, Business Analysts and Trading stakeholders
- Identify and implement opportunities for AI and intelligent automation, leveraging technologies including MLflow, Azure AI Foundry, Agent Bricks and Microsoft Copilot
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements for the Principal Data Engineer (Databricks)
- Extensive hands-on Azure Databricks expertise, with a strong track record designing and delivering enterprise-scale data platforms (PySpark, Python, SQL, Delta Lake, Structured Streaming, DLT)
- Proven ownership of data platform architecture, with strong knowledge of data modelling, integration, governance, security, performance and cost optimisation
- Experience leading complex, greenfield or large-scale data transformation programmes across multiple workstreams and engineering teams
- Strong real-time data and integration experience, including streaming architectures and complex data flows (Kafka beneficial)
- Excellent technical stakeholder management, with the ability to influence
Get help with your application
Your very own career expert that helps elevate your application to the next level.
High-profile role at the heart of a global trading data ecosystem. You'll combine hands-on engineering with genuine technical influence, helping shape a highly secure, cloud-native data platform that connects trading desks, physical assets and external partners
Initial 6-month contract with long-term programme scope, £850-950/day (inside IR35), hybrid in London. Interviews are being scheduled immediately, with a view to onboarding as soon as possible, so please apply now for immediate consideration.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location