Cognitive Group | Part of the Focus Cloud Group
Data and Ingestion Engineer - Robotics Sector
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Data Ingestion Engineer
We are looking for an experienced Data Ingestion Engineer to join a cutting-edge AI and robotics organisation, working on the large-scale data infrastructure that underpins the training and continuous improvement of advanced AI models.
This is a highly technical, hands-on engineering role focused on ingesting and processing huge volumes of real-world driving, vehicle and sensor data from automotive manufacturers, fleet operators, dashcam providers and other external data sources.
Scale is critical. We are looking for engineers who have personally built or operated pipelines processing terabyte-scale data volumes per day and can clearly quantify the largest ingestion and data-processing environments they have worked with.
The ability to ingest new datasets quickly and reliably has a direct impact on how rapidly AI models can be retrained, evaluated and improved.
Key Responsibilities
- Design, build and operate high-throughput data ingestion pipelines processing terabyte-scale datasets
- Ingest real-world driving, vehicle, camera, sensor and robotics data from multiple external providers
- Build scalable ETL/ELT processes for collecting, aggregating, transforming and moving data across complex environments
- Develop production data-processing solutions using Python, Spark/PySpark and SQL
- Debug and resolve failing or blocked ingestion pipelines
- Investigate corrupt, malformed, incomplete and inconsistent datasets without allowing individual failures to stall wider processing workflows
- Improve pipeline resilience through validation, retries, queue management, observability and automated recovery
- Optimise distributed processing workloads for throughput, compute efficiency and reliability
- Orchestrate complex multi-stage ingestion workflows and downstream dependencies
- Integrate new data providers rapidly, enabling datasets to become available for AI/ML model training within demanding timescales
- Explore AI-assisted and self-healing pipeline capabilities to automate the diagnosis and remediation of common ingestion failures
- Build monitoring and observability using technologies such as Prometheus and Grafana
- Deploy data-processing services using modern containerisation, cloud and Infrastructure-as-Code practices
- Work closely with Data, AI, ML and Robotics Engineers to increase the speed at which real-world data can be used to improve AI models
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Essential Skills & Experience
- Strong experience building and operating large-scale production data ingestion, ETL or data-processing platforms
- Demonstrable experience processing terabyte-scale data volumes per day
- Strong hands-on programming experience with Spark/PySpark, Python and SQL
- Experience with additional languages such as Scala, Java and TypeScript/JavaScript would be beneficial
- Strong experience with distributed data processing and technologies such as Spark and Databricks
- Experience with data engineering technologies such as DLT, dbt, Dagster, Airflow, Kafka and AWS Glue
- Experience working with large-scale cloud data platforms and storage technologies including S3 and Apache Iceberg
- Experience with databases and data warehouses such as Snowflake, Redshift, PostgreSQL and Oracle
- Experience working across AWS and/or Azure, ideally alongside on-premises environments
- Strong understanding of workflow orchestration, dependencies, retries, queue management and failure recovery
- Proven experience debugging production pipeline failures and dealing with messy, corrupt or unexpected source data
- Experience deploying production workloads using Docker and Kubernetes, including platforms such as AWS EKS/ECS
- Infrastructure-as-Code experience using Terraform and/or CloudFormation
- Experience developing and operating CI/CD pipelines
- Experience with production monitoring and observability, ideally using Prometheus and Grafana
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Highly Desirable
- Previous experience within robotics, autonomous vehicles, ADAS, automotive AI or another real-world AI/ML environment
- Experience processing vehicle, camera, sensor, telemetry or robotics datasets at significant scale
- Experience working with data supplied by automotive OEMs, fleet operators or other large external data providers
- Understanding of how ingestion infrastructure feeds AI/ML training, retraining and evaluation pipelines
- Experience with Flyte or comparable data/ML workflow platforms
- Experience applying AI or agentic tooling to pipeline monitoring, diagnosis and automated remediation
- Experience with technologies such as Kyubi/Kyuubi and modern lakehouse architectures
- Data analysis experience using libraries such as Pandas, NumPy, Matplotlib or Plotly
This is an inside IR35 contract role working via umbrella company so please only apply for this role if you are willing to work on this basis and are resident in the UK.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location