Mastek
Sr. Data Engineer
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Data Engineer
Databricks | AWS | Python | PySpark | SQL
Location: Leeds, UK / Reading, UK
Work Pattern: Hybrid
Travel Requirements: Mostly Remote - Travel to customer and Mastek offices as required
Security Clearance: SC Eligible OR Active SC Candidates
Immediate Joiners
About Mastek
Mastek Limited is a global provider of enterprise AI, digital, and cloud services, enabling clients to achieve measurable and sustainable returns on their technology investments. With a presence in over 40 countries and a workforce of nearly 5,000 professionals, Mastek delivers innovative technology solutions that help organisations accelerate transformation and unlock business value.
Through our Lead with AI approach, we integrate intelligence across our solutions and operations, enabling ethical, scalable, and domain-driven AI adoption. We partner with industry leaders including Oracle, Salesforce, Microsoft, AWS, Snowflake, and Databricks, delivering outcomes across sectors such as Healthcare, Financial Services, Retail, Manufacturing, Higher Education, and the Public Sector.
Guided by our core values of Trust, Value, and Velocity, we foster a culture of collaboration, innovation, continuous learning, and career growth.
About the Role
We are looking for a hands-on Data Engineer to run and maintain production data pipelines on Databricks and AWS. You will keep scheduled processing reliable, resolve incidents, deliver permanent fixes and improve the performance of existing data services.
The role combines L2 and L3 production support with data engineering. You will work with service management, platform teams, data owners and downstream users to keep data accurate and available within agreed service levels and delivery deadlines.
Key Responsibilities
- Monitor scheduled jobs, ingestion processes and downstream loads. Identify failures, delays and unusual data volumes, assess the business impact and prioritise action against delivery deadlines.
- Own incidents from investigation through recovery and closure. Diagnose issues in Python, PySpark, SQL, Databricks jobs and AWS integrations, and provide clear progress updates and timely escalation.
- Recover failed or incomplete processing through controlled retries, reruns and backfills. Check dependencies and reconcile outputs to prevent duplicate records, missing data or inconsistent downstream results.
- Investigate data quality issues, schema changes and late or missing source files. Trace problems through pipeline stages and agree corrective action with source owners and data consumers.
- Maintain and improve existing Python, PySpark and SQL code. Deliver defect fixes and small enhancements, using code review and appropriate tests to protect existing processing rules.
- Tune Spark jobs and SQL queries, including joins, partitioning, data skew and memory usage. Review Databricks compute usage and S3 file layouts to reduce processing time and unnecessary cost.
- Troubleshoot S3 access, IAM permissions, encrypted data access and connectivity with platform and security teams. Support runtime and dependency upgrades, controlled releases, rollback plans and post-release validation.
- Automate health checks, reconciliation and routine support tasks. Improve alerts and track job failures, data freshness, processing duration and recurring incidents to guide preventive maintenance.
- Maintain runbooks, dependency maps, root cause analyses and support handovers. Check that new or changed pipelines have monitoring, support ownership and recovery procedures before accepting them into service.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Essential Skills and Experience
- Proven experience supporting and maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis and permanent remediation.
- Strong Python, PySpark and SQL skills, with the ability to understand unfamiliar code, diagnose defects and make maintainable changes.
- Practical Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI and performance troubleshooting.
- Experience with Spark SQL, Delta Lake, Parquet and Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files.
- Hands-on AWS experience, particularly S3, IAM and CloudWatch, with an understanding of role-based access, KMS encryption and diagnosing data access failures.
- Understanding of batch and incremental processing, job dependencies, restartability, late-arriving data and safe reprocessing without duplicate results.
- Experience implementing data quality checks and source-to-target reconciliation, and investigating missing, duplicate or incorrect records in large datasets.
- Working knowledge of Git, code reviews, CI/CD and controlled production deployments, including testing, rollback and release validation.
- Experience working within incident, problem and change management processes. Able to prioritise by business impact, maintain clear support records and communicate with technical and non-technical colleagues.
Desirable Experience
- AWS Glue, Lambda, Step Functions or other orchestration services; Linux and shell scripting; Terraform; and delivery tools such as GitLab CI or Jenkins.
- Monitoring and alerting automation, capacity planning, cloud cost optimisation and disaster recovery exercises for data services.
- Supporting healthcare, public sector or other regulated data environments, including access controls, audit trails and secure handling of sensitive information.
- Transitioning pipelines from project delivery into live support, with practical knowledge transfer and documented operational acceptance.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Working Expectations
- Take ownership of issues through to resolution, work calmly during service disruption and raise risks early. Follow agreed access and change controls, and coordinate maintenance and recovery with the teams affected. Any on-call or out-of-hours support arrangements will be agreed for the role.
Why Join Mastek?
- Work on large-scale digital transformation programmes across public and private sector clients.
- Collaborate with highly skilled and diverse teams across multiple disciplines.
- Gain exposure to modern technologies, frameworks, and delivery methodologies.
- Access ongoing learning, professional development, and certification opportunities.
- Contribute to meaningful projects that have real-world impact.
- Be part of an inclusive culture that values innovation, teamwork, and continuous improvement.
- Enjoy clear career progression opportunities within a global technology organisation.
Our UK Benefits Include
- Competitive salary and annual bonus opportunities
- Hybrid and flexible working options
- Generous annual leave entitlement plus bank holidays
- Company pension scheme with employer contributions
- Private medical insurance (for self and dependants)
- Life assurance cover
- Employee Assistance Programme (EAP)
- Wellbeing initiatives and gym support
- Learning and development programmes, certifications, and professional training
- Employee recognition and reward schemes
- Cycle to Work scheme
- Employee referral bonus programme
- Enhanced family-friendly policies
Diversity & Inclusion
At Mastek, we are committed to building a diverse workforce and creating an inclusive environment where everyone can succeed. We welcome applications from all backgrounds and are proud to be an equal opportunities employer.
We believe diversity drives innovation and are dedicated to fostering a workplace where every employee feels valued, respected, and empowered to make an impact.
Apply today and help shape the future with Mastek.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location