iXceed Solutions
Data Engineer
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
GCP Data Engineer – Technology Lead
Location: London, UK
Work Mode: Hybrid – 2 days per week onsite
Contract Duration: 11 months
Contract Type: Contract
Role Overview
We are looking for an experienced GCP Data Engineer / Technology Lead to lead the engineering delivery and technical direction of a scalable, multi-tenant Data Integration Platform hosted on Google Cloud Platform (GCP).
The platform uses Apache Airflow to orchestrate ETL pipelines, moving data from source systems through the platform to downstream vendor systems. The successful candidate will provide hands-on technical leadership across data engineering, cloud architecture, automation, CI/CD, data quality, observability and end-to-end data lineage.
This is a hands-on technical leadership role requiring someone who can combine strong engineering capability with architectural thinking and stakeholder management.
Key Responsibilities
Technical Leadership & Architecture
- Own and drive the technical roadmap in line with business priorities and operational resilience.
- Lead solution design for new integrations, platform capabilities and technical improvements.
- Define and maintain engineering standards covering coding, testing, documentation, observability and security.
- Lead technical design reviews, code reviews and engineering discussions.
- Provide hands-on technical support for critical platform components, ETL pipelines, design spikes and performance issues.
- Mentor engineers and raise engineering standards across the team.
Data Engineering & Airflow
- Design, develop and maintain production-grade ETL/ELT pipelines using Apache Airflow.
- Manage pipeline scheduling, dependencies, retries, backfills and failure handling.
- Develop reusable patterns for onboarding new source systems and delivering data to downstream vendors.
- Build reliable and scalable data processing solutions across development, test and production environments.
- Implement data validation, reconciliation and exception-handling mechanisms.
- Develop operational runbooks and processes to support production data pipelines.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
GCP & Cloud Platform
- Design and deliver scalable data solutions using Google Cloud Platform (GCP).
- Work with GCP services including BigQuery, Cloud Storage, Pub/Sub, Dataflow, GKE and/or Cloud Run.
- Apply appropriate GCP IAM, networking, secrets management and environment separation practices.
- Optimise platform performance, reliability and cloud costs.
- Support capacity planning and the delivery of solutions against defined SLAs.
Multi-Tenant Platform Engineering
- Design and maintain scalable multi-tenant platform architecture.
- Implement appropriate tenant isolation and configuration management.
- Develop metadata-driven pipeline and integration frameworks.
- Ensure platform components can scale reliably as new tenants and integrations are onboarded.
CI/CD & Infrastructure
- Design and maintain CI/CD processes for platform services and Airflow DAGs.
- Implement automated testing, static analysis, security scanning and quality gates.
- Support versioning, packaging, deployment promotion and release controls.
- Implement infrastructure as code using Terraform.
- Establish safe deployment, rollout and rollback processes.
Data Lineage, Quality & Governance
- Support end-to-end data lineage from source → platform → downstream vendor.
- Implement metadata capture and visual data lineage capabilities.
- Ensure appropriate data quality, auditability, retention and access-control practices.
- Work with data governance and architecture teams to improve transparency and traceability.
Reliability & Observability
- Design resilient pipeline patterns using retries, idempotency, dead-letter mechanisms and backpressure.
- Implement effective logging, metrics, alerting and operational dashboards.
- Identify and resolve performance and reliability issues.
- Drive improvements to incident recovery, operational resilience and MTTR.
Stakeholder Management
Get help with your application
Your very own career expert that helps elevate your application to the next level.
- Work closely with product owners, data owners, architects, vendors and governance teams.
- Translate complex technical concepts for both technical and non-technical stakeholders.
- Lead technical discussions and drive alignment across multiple teams.
- Participate in Agile ceremonies, planning sessions and technical prioritisation.
Essential Skills & Experience
- Proven experience as a Technical Lead, Lead Data Engineer or Lead Engineer delivering cloud-based data engineering or integration platforms.
- Strong hands-on experience with Apache Airflow and production ETL/ELT pipelines.
- Strong experience with Google Cloud Platform (GCP).
- Practical experience with GCP data services such as BigQuery, Cloud Storage, Pub/Sub, Dataflow, GKE and/or Cloud Run.
- Strong Python development skills.
- Experience working with APIs, configuration management, automated testing and performance tuning.
- Strong experience with CI/CD, DevOps and automated quality gates.
- Practical experience with Terraform / Infrastructure as Code (IaC).
- Strong understanding of cloud security including IAM, networking and secrets management.
- Experience designing reliable data pipelines using patterns such as retries, idempotency, dead-letter handling and backpressure.
- Ability to communicate technical concepts clearly and work effectively with mixed technical and business audiences.
- Proven experience providing technical leadership, mentoring or guiding engineering teams.
Desirable Experience
- Experience with data lineage and metadata platforms such as OpenLineage, DataHub or Data Catalog-style tooling.
- Experience building multi-tenant data platforms.
- Experience with tenant isolation and metadata-driven frameworks.
- Knowledge of data governance, data quality, auditability, retention and access controls.
- Experience integrating with external vendors and managing technical dependencies.
- Experience working within large-scale enterprise data environments.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location