Pioneer Search
Data Engineer
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Data Engineer
Contract Data Engineer - Specialty Insurance
Contract Details
- Contract Length: 6 Months
- Location: Hybrid / London
- Rate: £475 - £500 per day
- IR35 Status: Outside IR35
- Start Date: ASAP
**About the Role
An experienced Data Engineer is required to take ownership of building a data pipeline to consolidate bordreaux reporting data into a standardised, reporting-ready structure. This is a hands-on delivery role within the BI team, where you will act as the sole Data Engineer responsible for the end-to-end development, design, and delivery of the project.
You will be building scalable Azure Databricks-based data pipelines, working independently to define and implement robust, production-ready solutions that support reporting and downstream analytics.
Key Responsibilities
- Take ownership of the end-to-end build of a scalable ETL pipeline in Azure Databricks to ingest and consolidate bordereaux reporting data
- Design and implement data ingestion processes, leveraging existing email-based ingestion tooling for multiple file formats from shared mailboxes
- Develop Databricks notebooks, jobs, and pipelines to support data transformation and orchestration, using a medallion architecture approach
- Build and implement data validation, quality controls, logging, monitoring, and error handling to ensure reliable pipeline performance
- Integrate solutions with Azure DevOps CI/CD pipelines across development, test, and production environments
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Key Requirements
- Strong hands-on experience as a Data Engineer (5+ years required)
- Proven experience with Azure Databricks, including:
- Building ETL pipelines
- Working with Delta Lake / Lakehouse architecture
- Strong proficiency in:
- Python
- PySpark
- Spark SQL
- Knowledge of Unity Catalog and modern Databricks governance and platform features
- Experience implementing CI/CD pipelines using Azure DevOps
- Experience supporting BI/reporting layers, such as Power BI or similar tools
- Insurance experience is advantageous
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Data & Application Notice
We process personal data submitted as part of your application for recruitment purposes in line with UK GDPR. All applications are reviewed by a member of our team; we do not use automated decision-making in our selection process.
Your application will be assessed against the requirements of this role.
By applying to this advert, you consent to receiving future relevant roles and industry news and insights from us. To opt out, please contact: compliance@pioneer-search.com.
For more information on how we use your data, including how long we retain it and your rights, please refer to our Privacy Policy.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location