TESTQ Technologies
Data Engineering Lead – GCP
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Lead the engineering and technical strategy for a scalable multitenant data integration platform leveraging Apache Airflow and GCP technologies
Job Description
Lead engineering delivery and technical strategy for a multitenant data integration platform
Build and maintain production-grade ETL/ELT pipelines orchestrated with Apache Airflow
Utilize container platforms such as GKE and Cloud Run
Apply strong software engineering skills in Python
APIs configuration management, testing, and performance tuning
Implement CI/CD pipelines, release controls, automated quality gates, and Infrastructure-as-Code (Terraform)
Design for reliability including retries, idempotency, deadletter patterns, and operational resilience
Translate complex technical concepts to diverse audiences and align cross-team efforts
Work with GCP BigQuery for large-scale data warehousing and analytics
Architect cloud solutions using GCP Cloud Architecture best practices
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Desirable Experience
Experience with data lineage/metadata management tools, multitenant architectures, data governance, and vendor system integrations
Roles and Responsibilities
- Own and drive the technical roadmap aligning engineering with business priorities and operational goals
- Lead solution design for new integrations and platform enhancements ensuring maintainability and scalability
- Establish and enforce engineering standards for coding, testing, documentation, observability, and security
- Contribute hands-on to critical ETLs framework components, design spikes, and performance optimizations
- Design and develop robust Airflow ETL pipelines with effective scheduling, dependency management, and error handling
- Develop onboarding patterns for new source and vendor systems ensuring reliability and traceability
- Drive data quality controls including validation, reconciliation, and exception handling
- Enhance the multitenant architecture focusing on tenant isolation, configuration, metadata-driven pipelines, and runtime scalability
- Optimize platform performance, cost, and reliability across environments with capacity planning and SLAs
- Improve automation for build, deployment, and release processes with CI/CD best practices and self-service tooling
- Implement comprehensive data lineage and observability with logging, metrics, and dashboards
- Partner with stakeholders including product owners, architects, vendors, and governance teams to manage requirements and delivery
- Lead agile ceremonies, remove blockers, and maintain delivery momentum
- Mentor engineers through reviews and coaching to foster continuous improvement and capability growth
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Skills
Mandatory Skills:
- Apache Airflow
- ETL Concepts
- GCP BigQuery
- GCP Cloud Architecture
- Python
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London