How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Lead Data Engineer
Lead Data Engineer – iGaming (Lakehouse & AWS)
About CreateFuture
CreateFuture is fast becoming the UK’s most recognisable digital consultancy, specialising in building digital products and services for major organisations—putting people first. Our offices are based in Edinburgh, Leeds, Manchester, London, and Bulgaria, with remote opportunities available nationwide.
We’re a team of creators—building code, strategy, culture, marketing, and AI solutions—collaborating with clients like PayPal, adidas, NatWest, FanDuel, and Money Saving Expert. Our reputation for delivering high-quality, scalable digital products has driven us to grow beyond 500 people, and our low employee turnover and recognition by Best Workplaces UK further attest to our supportive, friendly culture.
Join us in creating the future—one amazing project at a time.
About the Role & Team
We’re hiring an experienced Lead Data Engineer to shape our growing Data Practice, specialising in the iGaming sector. You’ll work with multidisciplinary teams to deliver data solutions for some of the world’s largest betting and gaming operators, handling millions of daily betting events across sportsbook, casino, and exchange platforms.
*Key Challenges
- Real-time odd feeds (sub-second latency demands)
- Regulatory compliance under the UK Gambling Commission and international standards
- Platform migrations from on-premise legacies to cloud-native AWS architectures
- High-throughput event processing with strict data integrity requirements
As a Lead Data Engineer, you’ll: ✅ Steer technical direction, mentor engineers, and align with client data teams ✅ Build low-latency streaming pipelines for betting, trading, and customer data ✅ Implement robust data integrity frameworks—validation, reconciliation, and compliance ✅ Integrate real-time data (e.g., in-play markets, odds, settlements) using AWS Kinesis, Kafka, or MSK ✅ Optimise cloud costs and infrastructure scaling via Spot instances, reserved capacity, or auto-scaling ✅ Lead architecture reviews and cloud-native migrations (e.g., Databricks/Snowflake implementations)
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What You’ll Bring
Deep iGaming & Data Expertise
- ➡ Proven experience with sportsbook, exchange, or casino platforms
- ➡ Understanding of high-speed data pipelines (odd feeds, in-play events, risk detection)
- ➡ Capability to handle reconciliation challenges in trading vs. settlement systems
- ➡ Hands-on with large-scale migrations from legacy systems to cloud architectures
AWS & Data Engineering Skills
- 🔹 AWS specialist (Glue, Lambda, S3, Redshift, Athena, Kinesis, MSK, Lake Formation)
- 🔹 Media Partner Monitoring via real-time pipelines (Spark/Flink/Storm)
- 🔹 Knowledge of cost-optimised architectures for high-volume streaming workloads
- 🔹 Optional: AWS Data Analytics or Solutions Architect certifications
Tooling & Programming
- 💻 Core: Python, SQL, Spark, plus build tools (dbt, Airflow)
- 🔍 Experience with data quality frameworks (e.g., Great Expectations)
- 🏗 Infrastructure as Code (Terraform, AWS CDK)
- 🌊 Bonus: Familiarity with Synapse, GCP Dataflow, or Airflow
Leadership & Consulting Mindset
- 🚀 Proven track record in mentoring data engineers and leading data teams through complex projects
- 💬 Ability to navigate stakeholder politics (multiple teams, legacy systems, competing priorities)
- 🤝 Adapts to ** clients’ constraints** for pragmatic yet high-quality solutions
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Tech Stack Overview
| Category | Critical Tech |
|---|---|
| Cloud | AWS (Glue, Lambda, S3, Redshift, Kinesis, MSK, Lake Formation, Step Functions) |
| Streaming | Kinesis, Apache Kafka/MSK, Flink, Spark Streaming |
| Languages | Python, SQL, Spark |
| Pipelines | dbt, Airflow (MWAA), AWS Glue |
| Data Quality | Great Expectations, dbt tests, custom reconciliation scripts |
| Infrastructure | Terraform, AWS CDK (preferred) |
What We Offer
Flexibility & Trust
- Hybrid/remote options
- Impact > hours—what matters is the outcome, not a rigid schedule
- Travel the world (clients/offices in UK/Europe)
Culture & Growth
✨ Flat hierarchies – creativity thrives ✨ Respect & collaboration – we listen to ideas ✨ Continuous learning – career progression backed by training
Our Inclusive Hiring Process
❌ No filler. Expect honesty, transparency, and fair feedback. ✅ Timeline:
- Call with our Talent Acquisition Team
- Technical/values assessment (short presentation, technical task, or case study in some roles)
🌍 We’re dedicated to inclusion. If you need adjustments or support during this process, let us know—we’ll make it work.
---fass fa-stars iGaming Data Engineer, CreateFuture Heroes, Future Builders
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location