Insight International (UK) Ltd
Lead Data Architect – AI & Cloud Infrastructure
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Role: Lead Data Architect – AI & Cloud Infrastructure
Location: Leeds, Manchester, Halifax - Hybrid model
Employment Type: FTC/Contract
Position Overview
We are seeking a visionary Lead Data Architect to spearhead the evolution of our enterprise data platform. In this role, you will bridge the gap between traditional data engineering and cutting-edge artificial intelligence. You will not build AI models from scratch; instead, you will architect the scalable frameworks, high-performance pipelines, and secure storage systems that power our Generative AI (GenAI) and Predictive Machine Learning (ML) initiatives. The ideal candidate will design enterprise-grade Blueprints for Vector databases, RAG (Retrieval-Augmented Generation) infrastructure, and unified data lakes that ensure our AI assets are secure, governed, and highly available.
Key Responsibilities
AI & Generative AI Infrastructure Design
- Architect RAG Pipelines: Design scalable end-to-end Retrieval-Augmented Generation infrastructure to inject real-time enterprise context into Large Language Models (LLMs).
- Vector Storage Management: Select, implement, and optimise enterprise vector databases (e.g., Pinecone, Milvus, pgvector) for high-performance embedding storage and semantic search.
- Agentic AI Enablement: Build high-throughput, low-latency data loops required to support autonomous AI agents in production.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Core Data Architecture & MLOps Integration
- Unified Data Foundations: Scale our modern data stack utilizing Lakehouse architectures (e.g., Delta Lake, Apache Iceberg) to handle both unstructured AI data and structured analytics.
- Feature Engineering Infrastructure: Design and maintain enterprise Feature Stores (e.g., Feast, Tecton) to serve unified data features consistently across offline training and online real-time inference.
- Streamline MLOps Pipelines: Partner with ML Engineers to integrate data pipelines seamlessly with lifecycle tracking frameworks like MLflow or Kubeflow.
AI Data Governance, Privacy & Quality
- Data Lineage Automation: Implement comprehensive data lineage tracking to audit the source datasets used for AI training, fine-tuning, and prompt context.
- Security & Compliance: Architect automated data masking, anonymisation, and PII-filtering pipelines to prevent sensitive data from leaking into foundational models.
- AI Data Cataloguing: Curate metadata structures within platform catalogs (e.g., Collibra, Atlan) to explicitly map physical data assets to corresponding AI applications.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Required Skills & Qualifications
- Experience: Minimum of 7+ years of experience in data architecture, data engineering, or enterprise infrastructure design.
- Cloud Mastery: Deep architectural expertise in at least one major cloud platform and its AI ecosystem:
- AWS: SageMaker, Bedrock, Glue, Redshift.
- Azure: Azure OpenAI Service, Azure Machine Learning, Synapse.
- GCP: Vertex AI, BigQuery ML, Dataflow.
- Advanced Data Modeling: Proven success modeling for both traditional relational/NoSQL analytical engines and high-dimensional vector spaces.
- Data Pipeline Frameworks: Hands-on experience with streaming and batch tooling including Apache Spark, Kafka, Flink, and orchestration tools like Apache Airflow or Prefect.
- Programming Literacy: Strong proficiency in Python, SQL, and database internals.
Preferred Qualifications
- Experience with unified data clouds such as Databricks or Snowflake.
- Relevant cloud certifications (e.g., AWS Certified Data Engineer, Azure Solutions Architect, Google Cloud Professional Data Engineer).
- Familiarity with Docker, Kubernetes, and infrastructure-as-code (Terraform).
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location