Insight International (UK) Ltd
Senior Data Engineer
London
Posted about 18 hours ago
Get notified of more jobs like this · No spam, ever
How your CV stacks up
1Upload CV
2Analyse CV
3Improve CV
Upload your CV to see how well it fits this job role
?%
Role Title: Data Engineer
Location: London, UK
Employment Type: Inside IR35 Contract
Technology Stack:
Snowflake, Snowflake Cortex, dbt, SQL, Python, AWS, CI/CD
Data Engineering & Pipeline Development
- Develop and maintain scalable data pipelines and ELT workflows.
- Build data ingestion and transformation pipelines across multiple source systems.
- Implement data cleansing, validation, reconciliation, error handling, and monitoring.
- Develop reusable and maintainable data engineering components.
- Ensure pipelines meet performance, reliability, scalability, and data-quality requirements.
Snowflake
- Develop data solutions using Snowflake.
- Create and optimize tables, views, stages, streams, tasks, and stored procedures.
- Develop scalable data models for analytics and reporting.
- Work with structured and semi-structured data, including JSON.
- Optimize Snowflake queries and workloads for performance and cost.
- Apply appropriate security, RBAC, and access-control practices.
Snowflake Cortex
- Hands-on experience with Snowflake Cortex is mandatory.
- Develop and integrate solutions using Cortex AI/LLM capabilities.
- Implement use cases such as text extraction, classification, summarization, enrichment, and other GenAI applications.
- Integrate Cortex capabilities into Snowflake data pipelines and analytics workflows.
- Apply best practices for security, governance, performance, and cost.
- Candidates without practical Snowflake Cortex project experience should not be considered.
dbt
- Develop and maintain dbt models, macros, tests, snapshots, and documentation.
- Build modular and reusable transformation workflows.
- Implement data-quality tests and manage model dependencies.
- Apply dbt best practices for Snowflake-based ELT pipelines.
- Participate in code reviews and troubleshooting.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
SQL
- Develop complex and optimized SQL for data transformation and analytics.
- Use joins, CTEs, window functions, aggregations, and analytical logic.
- Perform query tuning and troubleshoot data issues.
- Develop SQL-based data validation and reconciliation checks.
Python
- Develop Python solutions for data processing, automation, validation, and integration.
- Integrate Python with Snowflake and AWS services.
- Build reusable utilities and automation scripts.
- Apply best practices for testing, logging, exception handling, and maintainability.
AWS Cloud
- Develop data solutions using AWS services, including S3, Glue, Lambda, IAM, CloudWatch, and RDS.
- Build secure and scalable data movement between AWS and Snowflake.
- Support cloud-based data ingestion and processing.
- Monitor and optimize cloud resources for performance and cost.
Document Intelligence & Intelligent Data Extraction
- Develop Document Intelligence solutions to extract, classify, process, and enrich structured and unstructured documents.
- Build pipelines using Snowflake Cortex, GenAI, AWS, and dbt to convert document content into analytics-ready data.
- Work with OCR, document parsing, information extraction, summarization, entity extraction, and metadata generation.
- Implement data-quality, validation, security, and governance controls for healthcare documents.
- Optimize document-processing solutions for accuracy, performance, scalability, and cost.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
CI/CD & DevOps
- Implement and maintain CI/CD pipelines for data engineering solutions.
- Use Git-based source control and branching practices.
- Automate testing, validation, deployment, and environment promotion.
- Support CI/CD for dbt, SQL, Python, and Snowflake.
- Support deployments across development, QA/UAT, and production environments.
- Collaborate with DevOps teams on deployment automation and Infrastructure as Code.
Data Quality, Governance & Security
- Implement data-quality and validation checks.
- Maintain data lineage, metadata, documentation, and auditability.
- Follow security and access-control standards.
- Handle sensitive healthcare/life-sciences data in accordance with security and compliance requirements.
- Support data governance throughout the data lifecycle.
Snowflake Cortex – Desired Experience
- Snowflake Cortex AI Functions
- Cortex LLM/GenAI capabilities
- Text extraction and enrichment
- Text summarization and classification
- Sentiment analysis
- Embeddings and semantic search
- AI-enabled data transformation
- LLM integration within Snowflake
- Cortex-based enterprise AI use cases
- Integration of Cortex into production data pipelines
Required Skills
- Snowflake / Snowflake Cortex - GenAI
- dbt
Trusted by 25,000+ job seekers
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location