Serco Plc
Lead Data Engineer
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Lead Data Engineer
Lead Data Engineer Hybrid – Attached to our Solihull, London or Hook office Full Time, Permanent Band 3 / £75,000 - £85,000 (dependent on experience)
Here at Serco, we’re looking for a Lead Data Engineer to act as the technical authority for data engineering,
About the Role
Driving architecture, standards, and delivery across a modern multi-cloud ecosystem spanning Azure, AWS, and Databricks.
You’ll spend around 80% of your time hands-on, designing and building complex, enterprise-scale data solutions. The remaining 20% is just as critical: leading people, influencing stakeholders, and shaping how data engineering operates across the organisation.
Databricks sits at the heart of our platform – this role requires someone who can deeply leverage Databricks to enable scalable analytics, data science, and AI use cases across the business.
Key responsibilities include:
- Define and enforce data engineering standards, patterns, and best practices across the organisation.
- Architect and deliver scalable, resilient data solutions across Azure, AWS, and Databricks, ensuring enterprise-grade performance and reliability.
- Lead complex engineering workstreams – unblocking teams, guiding technical decisions, and ensuring high-quality delivery outcomes.
- Design and govern enterprise data models (conceptual, logical, and physical), including lakehouse zone strategies and schema evolution, approaches.
- Champion Databricks as the core data platform, leveraging:
- Lakehouse architecture,
- Unity Catalog,
- Delta Live Tables,
- Structured Streaming,
- MLflow.
- Drive a consistent multi-cloud Databricks strategy, enabling scalable and interoperable data platforms across environments.
- Optimise performance across pipelines, compute, storage, and orchestration frameworks, embedding cost-efficiency and scalability.
- Define and implement CI/CD for data platforms, supporting:
- Automated testing,
- Secure deployments,
- Multi-environment promotion).
- Embed robust data governance, security, and lifecycle controls, ensuring compliance through:
- Classification,
- Encryption,
- Access management,
- Lineage frameworks (e.g., Unity Catalog, Purview, Lake Formation).
- Act as a senior SME (Subject Matter Expert) for troubleshooting complex issues, leading root-cause analysis, and long-term remediation of data quality and platform challenges.
- Lead, mentor, and grow a community of Data Engineers, driving technical excellence through:
- Coaching,
- Reviews,
- Knowledge sharing.
- Operate effectively across a matrix organisation, influencing:
- Architecture,
- Cyber,
- Product,
- Sensor stakeholders.
- Communicate complex technical concepts clearly, articulating:
- Trade-offs,
- Risks,
- Strategic decisions to both technical and non-technical audiences.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements
To excel in this Lead Data Engineer role, you’ll need:
- Expert-level experience in enterprise-scale data engineering, including:
- Defining standards,
- Patterns,
- Overseeing complex delivery.
- Advanced proficiency in PySpark and SQL, with deep expertise in:
- Performance optimisation,
- Scalable data processing.
- Strong experience in:
- Data modelling, and large-scale data integration design,
- Building robust, maintainable, and testable pipelines.
- Proven ability to embed:
- Testing frameworks,
- Observability,
- Data quality controls across engineering workflows.
- Deep hands-on experience with Databricks (essential) including:
- Lakehouse architecture,
- Delta Lake,
- Unity Catalog,
- Streaming technologies.
- Broad cloud expertise across:
- Azure (Synapse, ADF, Event Hub, ADLS Gen2, Purview, Key Vault),
- AWS (S3, Glue, EMR, Kinesis, Athena, Lake Formation, IAM),
- With strong multi-cloud experience.
- Strong understanding of modern engineering practices, including:
- CI/CD for data platforms,
- Automated deployment pipelines,
- Environment promotion strategies.
- Experience designing:
- Monitoring,
- Observability,
- Cost optimisation frameworks,
- Governance,
- Lineage,
- Catalogueing,
- Compliance at scale.
- Exceptional stakeholder management skills with the ability to:
- Influence senior leaders,
- Work effectively across matrixed organisations.
- Proven leadership capability, including:
- Mentoring,
- Coaching,
- Developing engineering teams.
- Strong communication skills with the ability to translate complex technical concepts into clear business value.
- 8+ years’ experience in data engineering within enterprise environments.
- Relevant certifications (desirable) such as:
- Azure Data Engineer Associate,
- AWS Data Analytics Specialty,
- Databricks Data Engineer (Associate/Professional).
- SC clearance (or willingness to obtain it).
Get help with your application
Your very own career expert that helps elevate your application to the next level.
What We Offer
At Serco, we provide a shop-floor of support, and we put your personnel development front and centre.
- Company car
- 5% on-target bonus scheme
- Private medical insurance
- Flexible working
- 6% Pension
- Chance to contribute to innovation in public services
- A company passionate about diversity and inclusion
- Serco discounts in:
- Cinema,
- Merlin entertainment,
- Online shopping,
- Discounts on mobile phone plans and leisure centre memberships
- Range of benefits for your and your family’s health and wellbeing, including:
- Employee Assistance Programme,
- Simply Health Cash Plans,
- More.
- Wealth of career development training, tailored to your aspirations:
- Role-specific training,
- Leadership coaching,
- Formal study, and beyond, to support your career progression.
- A safe and supportive culture.
A Little About Us
At Serco, we’re: "Technical people building extraordinary human outcomes.".
About 50,000 people across:
- Defence,
- Transport,
- Justice,
- Immigration,
- Healthcare,
- Leisure,
- Citizen Services, A global team united by a shared purpose to improve the services communities rely on.
Guided by our values: Trust, Care, Innovation, Pride, we make meaningful impact every day.
Our Commitments:
- As a Disability Confident Leader,
- Top Great British Employers of Veterans (considered for 2026),
- A Gold Inclusive Employer Standard Holder, we champion equal opportunities for all.
We offer flexible and hybrid working where possible, and make reasonable adjustments throughout recruitment and employment.
Guaranteed Interview Scheme: Applicants who:
- Have disabilities,
- Are veterans,
- Have convictions,
- Meet minimum job criteria, Will be considered for guaranteed interviews – simply opt-in during application.
Ban the Box: We support fair access for those with unspent criminal convictions through our ‘Ban the Box’ pledge (certain roles may have security exemptions).
For support with your application, please contact:
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location