How your CV stacks up
Upload your CV to see how well it fits this job role
?%
KEY INFORMATION
Position
Senior Data Engineer
Reporting to
Data Engineering Manager
Location
Manchester
Overview of the Role
As a Senior Data Engineer, you will be an integral part of a cross-functional feature development squad. You will play a key role in the evolution of our payments intelligence platform towards a Databricks-native and increasingly agentic operating model. You will design, build and operate secure data ingestion, orchestration and platform capabilities, using Databricks as the first-choice execution platform and Azure services where appropriate. The focus remains on production data engineering rather than AI model development: creating reliable, observable and well-governed foundations that can be operated by both engineers and automated/agentic workflows. You will work across client onboarding, data retrieval, ELT, CI/CD and platform reliability to deliver continuous business value.
Job Role
Key accountability of this role:
- Delivery, support and continuous improvement of secure, scalable, Azure/Databricks data capabilities and infrastructure on our payments intelligence platform.
Key responsibilities will include:
- Collaborate within a cross-functional squad to define and deliver scalable, secure data and platform capabilities aligned with business outcomes.
- Design Databricks-native solutions as the default, using Spark/SQL, Lakeflow Jobs, Unity Catalog, Volumes and Databricks Asset Bundles, with Azure services used where they add value.
- Build and operate data retrieval and ingestion across APIs, SFTP/FTPS and browser automation, migrating suitable orchestration from external services into Databricks-native workflows.
- Own client data onboarding flows, including configuration, historical backfills, validation, data quality, monitoring and operational handover.
- Build reusable platform capabilities and guardrails that enable agentic workflows to safely automate onboarding, operational diagnosis and routine engineering tasks, with human approval where required.
- Engineer for production reliability through observability, alerting, retries/idempotency, failure recovery, performance and cost optimisation; investigate complex failures and drive root-cause fixes.
- Deliver changes through version control, automated testing, CI/CD and Infrastructure as Code, maintaining secure coding and deployment standards through code review.
- Apply strong security and governance practices across Unity Catalog, Entra/RBAC, secrets, credentials and data access.
- Use advanced Python with strong SQL/Spark skills to develop maintainable, reusable libraries, frameworks and data pipelines.
- Mentor data engineers and drive continuous improvement, evaluating new Databricks and automation capabilities against clear technical and business value.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What We’re Looking For
You are a great match if:
- 3+ years professional experience as a Data Engineer, DevOps Engineer or Systems Engineer delivering production data platforms.
- Strong hands-on experience with Azure Databricks, including Spark/SQL, workflow orchestration and production lakehouse data engineering.
- Advanced proficiency in Python for data retrieval, processing, automation, integration and reusable engineering libraries.
- Experience designing and operating ETL/ELT pipelines with data quality controls, schema handling, monitoring and failure recovery.
- Expertise in CI/CD, version-controlled deployment and automated testing for data or platform workloads.
- Experience integrating data through APIs and secure file transfer (SFTP/FTPS), including secure handling of credentials and secrets.
- Strong Microsoft Azure knowledge, including ADLS, Azure DevOps and Entra/RBAC, with experience designing and deploying infrastructure.
- Strong production troubleshooting, problem-solving and operational support skills.
- Degree in Computer Science, Mathematics, Physics or other STEM related subject.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
What We Offer
- Excellent performance-based earning opportunity, including Objective-driven bonuses.
- Ability to advance career and expand professional experiences in a hyper-growth company.
- Future opportunity for equity, rewarded to high performers.
- Payments industry training and continuous training in respective role.
- Personalised individual development plan, aligned to professional goals.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills