bigspark
Data Engineer
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Data Engineer – Glasgow/Edinburgh Hybrid
About bigspark
We are creating a world of opportunity for businesses by responsibly harnessing data and AI to enable positive change. We adapt to our clients' needs and then bring our engineering, development, and consultancy expertise. Our people and our solutions ensure they head into the future equipped to succeed.
Our clients include Tier 1 Banking and Insurance clients, and we have also been listed in the Sunday Times Top 100 Fastest Growing Private Companies.
The Role
We're looking for a Data Engineer to develop enterprise-scale data platforms and pipelines that power analytics, AI, and business decision-making. You'll work in a hybrid capacity, which may require up to 2 days per week on a client's premises.
What You'll Do
- Develop highly available, scalable batch and streaming pipelines (ETL/ELT) using modern orchestration frameworks.
- Integrate and process large, diverse datasets across hybrid and multi-cloud environments.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
What You'll Bring
- 3+ years of commercial data engineering experience
- Strong programming skills in Python, Scala, or Java, with clean coding and testing practices.
- Big Data & Analytics Platforms: Hands-on experience with Apache Spark (core, SQL, streaming), Databricks, Snowflake, Flink, Beam.
- Data Lakehouse & Storage Formats: Expert knowledge of Delta Lake, Apache Iceberg, Hudi, and file formats like Parquet, ORC, Avro.
- Streaming & Messaging: Experience with Kafka (including Schema Registry & Kafka Streams), Pulsar, AWS Kinesis, or Azure Event Hubs.
- Data Modelling & Virtualisation: Knowledge of dimensional, Data Vault, and semantic modelling; tools like Denodo or Starburst/Trino.
- Cloud Platforms: Strong AWS experience (Glue, EMR, Athena, S3, Lambda, Step Functions), plus awareness of Azure Synapse, GCP BigQuery.
- Databases: Proficient with SQL and NoSQL stores (PostgreSQL, MySQL, DynamoDB, MongoDB, Cassandra).
- Orchestration & Workflow: Experience with Autosys/CA7/Control-M, Airflow, Dagster, Prefect, or managed equivalents.
- Observability & Lineage: Familiarity with OpenLineage, Marquez, Great Expectations, Monte Carlo, or Soda for data quality.
- DevOps & CI/CD: Proficient in Git (GitHub/GitLab), Jenkins, Terraform, Docker, Kubernetes (EKS/AKS/GKE, OpenShift).
- Security & Governance: Experience with encryption, tokenisation (e.g., Protegrity), IAM policies, and GDPR compliance.
- Linux administration skills and strong infrastructure-as-code experience.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
In return, you will receive:
- Competitive salary
- Generous Annual Leave
- Discretionary Annual Bonus
- Pension Scheme
- Life Assurance
- Private Medical Cover (inc family)
- Permanent Health Insurance Cover / Income Protection
- Employee Assistance Programme
- A Perkbox account
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location