Sundayy
Data Engineer
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
About The Company
Black Duck Software, Inc. is a leading provider in application security and software quality management. The company specializes in helping organizations build secure, high-quality software by minimizing risks and maximizing productivity. Recognized as a pioneer in the application security domain, Black Duck offers comprehensive solutions including Software Composition Analysis (SCA), Static Application Security Testing (SAST), and Dynamic Application Security Testing (DAST). These tools enable development teams to swiftly identify and remediate vulnerabilities and defects within proprietary code, open source components, and overall application behavior. Leveraging industry-leading tools, expert services, and deep domain knowledge, Black Duck supports organizations in enhancing security and quality throughout the entire DevSecOps lifecycle and software development process.
About The Role
We are seeking a highly skilled Data Platform Engineer to lead the design and development of cross-product data services across multiple product lines. This role involves establishing a governed data plane that ensures consistency, reliability, and security of data assets used internally and by customers. The successful candidate will define the customer data model, including canonical identifiers, shared dimensions, and consistent facts, to facilitate seamless data integration across various platforms. You will be responsible for building scalable ingestion pipelines for batch, streaming, and event-driven data, ensuring repeatable onboarding processes for new data sources. Additionally, you will own the operational aspects of data reliability, including data contracts, quality checks, lineage, monitoring, and incident response. The role also involves implementing accessible data interfaces such as curated datasets, secure query endpoints, and APIs, along with productizing customer-facing data assets with clear documentation, versioning, and ownership. You will work closely with cross-functional teams to ensure data models are optimized for both operational systems and analytics environments, supporting machine learning workflows and AI automation initiatives. The role requires a collaborative approach, partnering with product, engineering, and security stakeholders to align data solutions with strategic roadmaps and customer outcomes. You will also contribute to raising the technical standards through architecture reviews, adherence to best practices, and mentorship, while maintaining hands-on involvement in core systems.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Qualifications
- Extensive experience in building and operating large-scale production data platforms, including on-call and operational responsibilities
- Proficiency in SQL and Python for developing pipelines, services, and automation processes
- Hands-on experience managing cloud infrastructure on AWS and Google Cloud, including compute, storage, networking, and IAM
- Practical knowledge of operational databases (RDS-style) and analytics stores (columnar/OLAP), with performance tuning expertise
- Strong data modeling skills, including schema evolution, conformed dimensions, and establishing a single source of truth for metrics
- Proven track record delivering reliable data products with clear contracts and service-level agreements
- Ability to balance trade-offs between latency, accuracy, cost, and security in engineering decisions
- Experience with lakehouse architectures and open table formats, including governance and maintenance
- Familiarity with orchestration and streaming systems used in production environments, managing batch and real-time data flows
- Understanding of ML data requirements, including training/serving data splits, feature engineering, and evaluation datasets
Responsibilities
- Lead the design and build-out of cross-product data services from a centralized, governed data plane
- Define and implement the customer data plane model, ensuring consistency across products
- Develop scalable ingestion pipelines for batch, streaming, and event-driven data sources, with repeatable onboarding processes
- Maintain the operational health of data systems through contracts, quality checks, lineage tracking, and incident management
- Implement secure and accessible data interfaces, including curated datasets, query interfaces, and APIs for internal and external use
- Productize data assets with comprehensive documentation, version control, and clear ownership assignments
- Design data models optimized for operational and analytical systems, focusing on performance and cost efficiency
- Support machine learning workflows by providing high-quality, trusted datasets and feature outputs
- Enable AI automation by delivering low-latency, governed data products suitable for automated decision-making
- Collaborate with product, engineering, and security teams to align data solutions with strategic goals and customer needs
- Conduct architecture reviews, enforce standards, and mentor team members to elevate technical excellence
- Stay hands-on with core systems, ensuring robustness and continuous improvement of data platforms
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Benefits
- Competitive salary and comprehensive benefits package
- Flexible hybrid work model with collaboration hours accommodating UK and US time zones
- Opportunities for professional development and career growth
- Engagement in innovative projects at the forefront of application security and data engineering
- Supportive and inclusive work environment committed to diversity and equal opportunity
Equal Opportunity
Black Duck Software, Inc. is an equal opportunity employer. We consider all applicants for employment without regard to race, color, national origin, religion, sex, gender identity or expression, age, disability, sexual orientation, veteran or military service status, or any other characteristic protected by applicable law. We are committed to creating a diverse and inclusive workplace and provide reasonable accommodations to individuals with disabilities in accordance with legal requirements.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location