Crystal Horizon IT Consulting
Data Engineering Lead (mAI – Data Foundation Core)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Data Engineering Lead (mAI – Data Foundation Core)
Data Foundation Lead (Market Access AI) – Remote Contract (UK/Europe)
Company Description
Crystal Horizon IT Consulting partners with organizations to transform their IT operations into agile, transparent ecosystems that deliver measurable business outcomes. By combining deep industry expertise, practical implementation strategies and disciplined execution, we help drive sophisticated technology initiatives.
Through our strategic partnership with Leontra Technologies, a global leader in IT services, we leverange an expanded talent pool, cutting-edge capabilities in digital transformation, and the deployment of modern, data-driven platforms. Our collaborative approach enables us to deliver high-impact, long-term value across diverse industries for clients worldwide.
Location
Anywhere in the UK or Europe | Remote Contract Role
Experience
- 12–15+ years across:
- Data Engineering
- Cloud Data Platforms
- Data Architecture
- Enterprise Data Solutions
Role Summary
You will lead the Data Foundation for Takeda’s Market Access AI platform, defining:
- Scalable AWS-based data architecture
- Enterprise-grade data pipelines
- Governance frameworks
- Reusable data services for core AI tools:
- Insights Generator
- Value Dossier
- Pricing
- OSS (Other Stakeholder Systems)
- mAI Assistant
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
This role significantly shapes Takeda’s ability to realize real-world AI outcomes with robust, well-governed data assets.
Key Responsibilities
-
Architecture & Technology Foundations
- Lead the design and implementation of a structured and unstructured AWS-based data platform supporting AI/GenAI solutions.
- Architect scalable data pipelines (ingestion, processing, storage).
- Define data models, metadata, lineage frameworks, and data quality measures.
-
Governance & Reusability
- Establish reusable data contracts and secure data access mechanisms across all enterprise AI products.
- Implement enterprise-wide governance models to ensure data consistency, traceability, and compliance.
-
Cross-Functional Leadership
- Collaborate with Takeda’s Enterprise Architecture, Data Engineering, AI, Security, and Product teams to align the Data Foundation roadmap with business needs.
- Drive engineering standards, technical reviews, and backlog prioritization to enhance system capabilities.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
- Team Mentorship & Delivery
- Mentor distributed engineering teams to refine technical practices and deliver efficiency gains.
- Use Terraform, GitHub, and CI/CD principles to ensure scalable, reproducible deployments.
Required Skills
-
AWS Proficiencies:
- S3, Glue, Lambda, Athena, Redshift, EMR
- Lake Formation, Step Functions, EventBridge
- API Gateway, IAM
-
Technical Stack:
- Python, PySpark, SQL
- Apache Spark (preferred)
- Kafka (highly preferred)
-
Data Lakehouse & Governance:
- Deep expertise in Data Lake/Lakehouse architecture.
- Data Governance, Metadata Management, Data Lineage.
-
AI/GenAI Expertise:
- Experience with GenAI frameworks, RAG (Retrieval Augmented Generation), Vector Databases, Knowledge Graphs.
-
DevOps & Infrastructure:
- Terraform for IaC, CI/CD pipelines.
-
Soft Skills:
- Exceptional stakeholder alignment and technical communication abilities.
Ready to Make an Impact?
If this fully aligns with your goals and expertise, take action now.
📩 Submit your CV at: info@crystalhorizon.uk
or send it directly via the chat channel.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location