RedCat Digital
Data Engineer
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Data Engineer | Python & Databricks
London | Hybrid, 3 days per week in the office
6-month initial contract
£550–£850 per day, depending on experience
Inside IR35 | Paid via Umbrella
We are looking for an experienced Senior Data Engineer with strong hands-on expertise across Python, Databricks and Spark to join a large-scale data engineering environment in London.
This is a genuinely hands-on role, with around 30–50% of your time expected to be spent coding, solving technical problems and contributing directly to the development and optimisation of data platforms and pipelines.
Alongside the hands-on engineering work, you will take technical ownership across delivery, provide direction to other engineers and work closely with stakeholders to ensure data engineering initiatives are delivered effectively.
Key responsibilities
- Contribute directly to the design, development and optimisation of data pipelines using Python, PySpark and Databricks
- Build and enhance ETL/ELT processes and reusable data engineering frameworks
- Troubleshoot complex data and performance issues and provide hands-on support with production issues
- Contribute to code reviews, debugging, testing and performance optimisation
- Own delivery of data engineering initiatives, ensuring work is delivered to a high standard and aligned with business requirements
- Support sprint planning, backlog prioritisation and delivery tracking
- Identify and resolve technical risks, dependencies and blockers
- Provide technical direction and architectural oversight across data engineering initiatives
- Design and support modern Lakehouse architectures using Databricks and Delta Lake
- Lead and mentor other data engineers, providing technical coaching and hands-on guidance
- Work closely with product owners, architects and business stakeholders to translate requirements into effective technical solutions
- Promote best practice across engineering, CI/CD, DevOps, Agile delivery, data quality and governance
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What we're looking for
- 10+ years' experience across data engineering and/or software engineering
- Proven experience as a Senior Data Engineer, Technical Lead or similar
- Strong, recent and hands-on Python development experience is essential
- Extensive experience with Databricks, including notebooks, workflows and Delta Lake
- Strong Apache Spark / PySpark experience
- Proven experience building and optimising large-scale ETL/ELT data pipelines
- Strong understanding of data architecture and modern Lakehouse concepts
- Strong SQL skills and experience working with distributed data systems
- Experience with at least one major cloud platform, such as AWS, Azure or GCP
- Experience working in Agile delivery environments
- Ability to balance hands-on engineering with technical leadership and delivery responsibility
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Desirable experience
- Databricks certification
- Experience with Kafka or Spark Structured Streaming
- CI/CD and DevOps practices
- Delta Lake optimisation
- Airflow or Databricks Workflows
- Docker and/or Kubernetes
- Exposure to machine learning pipelines
The key requirement is that you remain hands-on. This is not a pure delivery or management position. You will be expected to write code, solve complex technical problems and contribute directly to the engineering work, while also providing technical leadership and supporting delivery.
Contract: 6 months initially
Rate: £550–£850 per day, depending on experience
Location: London, hybrid with 3 days per week in the office
IR35: Inside IR35
Payment: Via Umbrella
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location