Bloomberg
Senior Software Engineer - ETL Pipeline Orchestration Platform
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Location
London
Business Area
Engineering and CTO
Ref #
10046054
Description & Requirements
Bloomberg is a global leader in business and financial information, delivering trusted data, news, and insights that bring transparency, efficiency, and fairness to capital markets. Bloomberg's Data Technology Infrastructure Department designs and builds large-scale platforms to acquire financial reference and pricing data globally from various sources, delivering billions of data points daily to clients for investment analysis.
The ETL Pipeline Orchestration Platform team owns a low code platform that helps engineers and analysts author and manage data transformation, derivation, validation, and database loading themselves. As adoption accelerates, we are redesigning key components to support further throughput and resiliency while maintaining the availability of the system.
What’s in it for you?
We are the connecting tissue across the various core capabilities offered by our sibling teams. If you join our team, you will be working on solving the fundamental problems in event-driven orchestration systems using both open source technologies as well as in-house ones. We are a multi-lingual team taking care of Python applications, performant queue broker in Go to C-based Python extensions and everything in between. If you have a knack for distributed systems or a love for orchestration platforms, join us!
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Future Initiatives
We would like to work on the following in this year and next:
- Develop, deploy, and integrate our internal priority queue system to mitigate head-of-line blocking issues and optimize the time to market for the most important tasks
- Deprecate the on-cloud orchestration system by unifying with on-prem system
- Component registration / management faculty to make the onboarding processing easier for the service owners
- Introduce multi-tenancy with granular privileging for both pipeline creation and edit time as well as runtime
- Better support and capability for creation of DAG in orchestration
- Programmatic recipe creation, modification, and interaction to be API first
- LLM integration to assist in authoring ETL tasks
We’ll Trust You To
- Work with us to design the user-friendly, fault tolerant, and scalable orchestration platform
- Participate in the full SDLC of various components and systems that make up the orchestration platform
- Familiarize yourself with the Data Processing / Analysis tasks taken on by the Data Department and systems developed by our sister teams
- Have your antenna up and bring the latest development in OSS and capabilities into our stack
- Mentor less experienced members of the team
Get help with your application
Your very own career expert that helps elevate your application to the next level.
You’ll Need To Have
- Demonstrable experience building fault tolerant and horizontally scalable distributed applications / systems dealing with distributed system specific concerns
- Demonstrable experience writing production code in either Python, Go, Rust, JavaScript/TypeScript, or Java
- Strong problem solving skills
We’d Love To See
- Knowledge of Database Systems and trade-offs in distributed systems
- Familiarity with API Designs
- Familiarity with Orchestration Frameworks such as Apache Airflow, Argo Workflows, Dagster, etc.
- Experience working with and designing systems utilizing AWS
If indicated, please note that years of experience are a guide; we will consider applications from all candidates who can demonstrate the skills necessary for the role.
Discover what makes Bloomberg unique - watch out for an inside look at our culture, values, and the people behind our success.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location