Infoplus Technologies UK Limited
Senior Data Engineer
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Data Engineer
Senior Data Engineer – Cloud & Data Platform Re-architecture (ICDP)
Location Warwick
Role Type Contract (InsideIR35)
Mode of Working Hybrid (3 days per week)
Duration 6 months
About the Role
The Interconnectors Data Platform (ICDP) is a cloud-based data warehouse underpinning critical commercial, financial modelling, and operational decision-making. As the platform evolves toward a modernized Medallion Architecture and Azure-native ingestion patterns, we are looking for a Senior Data Engineer to provide:
- Strategic architectural direction
- Hands-on technical leadership
- End-to-end pipeline modernization
This is a perfect opportunity to shape best-in-class data infrastructure while collaborating across Product, IT&D, and external partners.
Responsibilities
Data Architecture & Platform Engineering
- Lead the design and implementation of scalable data architectures (Bronze/Silver/Gold layer integration).
- Steward the platform’s technical roadmap, ensuring alignment with modern engineering best practices.
- Develop highly observable, secure, and performant ingestion and transformation pipelines.
Pipeline Development & Operations
- Drive the migration from legacy ETL tools (e.g., Matillion) to Azure-native pipelines, including:
- Azure Data Factory (ADF)
- Azure Functions
- Event-driven frameworks
- Build and scale optimized SQL transformations, curated data layers, and cost-effective data models.
- Embed CI/CD, scalable testing, version control, and monitoring into engineering workflows.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Data Quality & Governance
- Implement enterprise-grade data governance, including:
- Validation and reconciliation
- Profiling & lineage tracking
- Auditability mechanisms
- Work closely with business teams to ensure data output meets analytical and operational requirements.
Leadership & Collaboration
- Mentor junior engineers and oversee the technical upskilling of the ICDP team.
- Collaborate with Product, IT&D, and external partners to deliver high-impact data solutions.
- Establish engineering standards, reusable patterns, and best practices for the platform.
Your Profile
Essential Skills & Experience
Technical Proficiencies
(Production-grade experience assumed)
- Python: Advanced pipeline development, orchestration (Airflow, Wrangler), and static analysis testing.
- SQL: Expert-level optimization, modelling, and establishing parameters for performance tuning.
- Azure Cloud: Hands-on experience with:
- Azure Functions / ADF (orchestration)
- Azure Synapse/Databricks (transformations)
- Azure Storage (Blob/Datalake), Key Vault, Synapse Spark
- Security and compliance best practices
- Data Architecture:
- Medallion/Lakehouse and enterprise warehouse design.
- Best-of-breed analysis for target state architecture with cost/performance balance.
- DevOps:
- Git-based feature/feature flag branching workflows
- Automated testing (unit, CI/CD pipeline testing, coverage)
- Monitoring (Azure Monitor, Powershell/Dax queries for performance).
Get help with your application
Your very own career expert that helps elevate your application to the next level.
NATS/Inbound Data Flow
(Critical for>80%+ engineering load)
- API-driven (REST/Webhook-integrated/)
- Graceful fallbacks (Deadletter queues, membership change handling)
- SFT / custom container orchestration (Schema evolution tolerance & containerized SFTP services)
Soft Skills
- Problem-solving: Ability to tackle complex engineering dilemmas with principled solutions.
- Communication: Clear stakeholder briefings for non-technical decision-makers.
- Leadership: Inspirational technical quattro guide for teams through legacy-to-cloud modernisation.
Desirable Skills
- Medallion/Lakehouse patterns: Current engagement/worked with Snowflake or Databricks.
- Regulated/data-assurance pipelines: Experience ensuring compliance with ISO27001, GDPR, or MiFID requirements.
- Modelling teams: Fluency in working closely with BI teams and data science/Python engineers.
Iconic quote from team: "You’ll be a driver of this move toward cloud-native + pipeline-as-code excellence—critical for a 5-10 year path forward! Sheer opportunity to break tidal waves of legacy work + set new standards."
Note: This role is structured to allow for tailored hybrid growth within some OR NoCover if performance goals are met.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills