Collage Recruitment
Lead Data Architect
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Lead Data Architect – Insurance Data & Analytics (Databricks Migration)
London (Hybrid) | Permanent, Full-Time
Note that this is an advert created by Collage Recruitment. The full and original job description will be shared with candidates who are selected for a video interview with Collage Recruitment.
About the Role
Our client, a leading provider of benchmark data and analytics for the financial services sector, is hiring the first senior engineering leader for their UK Insurance data business. This is a newly created role, not a backfill.
This is a rare opportunity to own a platform transformation from the ground up: migrating a legacy SQL benchmarking engine onto a modern Databricks Lakehouse, and then building the product tooling on top of it.
Our client provides benchmark data behind pricing, acquisition, renewal, and retention decisions across the UK insurance market, built on a proprietary, behavioural dataset that underpins trusted relationships with senior commercial decision-makers across the industry.
You'll report directly to the Head of Insurance, based in London, and will be the senior technical voice for the UK Insurance business — with the option to line-manage an existing mid-level engineer as the team grows. You'll also work closely with a broader, global engineering function on platform standards, while owning delivery locally.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What You'll Do
- Lead the migration of legacy SQL solutions into Databricks Jobs, Workflows, and PySpark/SQL pipelines, starting with a lift-and-shift of core benchmark data, while maintaining operational SLAs throughout
- Define the target-state data architecture (Lakehouse, Medallion, domain-oriented data products) to support reporting, analytics, and future AI/ML use cases
- Work directly with business leadership and analysts to codify canonical metric logic — replacing spreadsheet calculations with documented, testable code
- Own UK Unity Catalog governance: catalogue, schema, table design, ownership, and access control
- Build automated QA and reconciliation between legacy and new outputs, with the observability to give internal and external stakeholders confidence in the data
- Implement CI/CD and Infrastructure as Code for Databricks assets
- Tune Spark and SQL workloads for performance and cost
- Act as the primary link between UK delivery and the centralised platform engineering function
- Provide technical leadership and mentorship, with the option to line-manage a mid-level engineer
What We're Looking For
Essential:
- Hands-on experience delivering a Databricks migration at production scale — this is the core of the role
- Strong stakeholder and relationship management skills. You'll work closely with business leadership, analysts, and engineers across regions, often with ambiguity to navigate
- Advanced Python, PySpark, and SQL
- A track record migrating SQL Server workloads, stored procedures, and scheduled jobs to modern, cloud-native platforms
- Solid understanding of Unity Catalog governance, security models, and access control
- Experience with CI/CD, automated testing, and Infrastructure as Code
Get help with your application
Your very own career expert that helps elevate your application to the next level.
We'd love if you also have:
- Experience in a smaller, B2B-focused fintech environment — this role suits someone used to close commercial context and hands-on delivery, rather than large enterprise-scale environments with extensive legacy system sprawl
- Insurance, financial services, pricing, or marketing analytics background
- Familiarity with UK GDPR, data residency, and regional governance
- dbt or analytics engineering experience
Why This Role
This isn't a startup. It's an established, well-regarded business in its space, but you'll have the ownership and autonomy of building something from scratch. You'll shape the architecture, the standards, and potentially the team, with the backing and technical standards of a wider global engineering organisation behind you.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location