CommuniTech Recruitment Group
Lead Data Engineer - Python/ AWS. Asset Management. £120,000 -£140,000+ Discretionary Bonus + Benefits. Hybrid 2 Days a week in Central London office.
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Lead Data Engineer - Python/ AWS. Asset Management. £120,000 -£140,000+ Discretionary Bonus + Benefits. Hybrid 2 Days a week in Central London office.
Lead Data Engineer - Python/AWS
Asset Management £120,000 - £140,000+ (discretionary bonus + benefits) Hybrid (2 days in Central London, rest remote)
About the Company
My client is a $5bn+ alternative asset manager with offices in New York, BVI, London, Switzerland, Dubai, Singapore, and Hong Kong. They serve institutional investors and high-net-worth clients through a range of alternative investment strategies, grounded in a proprietary, cloud-native platform.
The technology team is lean and high-output, consisting of three engineers led by the COO. This structure ensures:
- Significant ownership of systems
- Direct impact on the platform’s evolution
- Close visibility to senior leadership and business stakeholders
The Role
They are seeking a Data Engineer to join their small engineering team, collaborating alongside two other engineers to evolve their data platform.
This is not a passive "build what you're told" role. Instead, they need someone who can:
- Identify business problems and propose end-to-end data solutions
- Work directly with operations, research, and investment teams to diagnose bottlenecks (e.g., delayed or missing data) and optimise workflows
The role is data-engineering focused, requiring expertise in:
- Production system ownership (reliability, performance)
- Cross-team collaboration when necessary
- Balancing pipeline maintenance with new capability development (shifted based on business needs)
Key attributes for success: ✔ Comfortable in a small, autonomous team ✔ Ability to context-switch between deep technical work and stakeholder engagement ✔ Pragmatic problem-solving over overly abstract solutions
Key Responsibilities
- Solve business problems with data:
- Partner with ops, research, and investment teams to identify pain points and deliver tangible solutions (beyond ticketing)
- Improve the data platform end-to-end:
- Ingestion → Transformation → Storage → Serving → Observability → Reliability
- Optimise data freshness and reliability:
- Reduce latency in pipelines
- Eliminate stale data
- Strengthen alerting mechanisms to prevent silent failures
- Map and optimise data architecture:
- Identify inefficiencies and unnecessary handoffs
- Streamline data flows from vendors to consumers
- Operate critical production pipelines:
- Maintain systems ingesting financial data from external vendors
- Expand vendor integrations and capabilities:
- Build and enhance new data products, pipeline features, and vendor hooks
- Manage AWS infrastructure:
- Deploy solutions using Terraform
- Incorporate AI/ML into data workflows:
- Develop RAG pipelines and OCR-based document extraction to unlock unstructured data sources
- Collaborate cross-functionally:
- Work with the engineering team, including systems interacting with the NextJS client-facing app
- Plan and prioritise work:
- Align technical and non-technical stakeholders on roadmaps
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Core Technologies Used
| Category | Tools/Technologies |
|---|---|
| Primary Language | Python (ETL, API clients, data validation, pipeline development) |
| Data Querying | SQL (analytical & transactional queries, transformations) |
| Cloud Infrastructure | AWS (S3, Lambda, ECS, Kinesis, RDS, etc.) |
| Infrastructure-as-Code | Terraform |
| Data Storage | MongoDB (document storage) |
| Observability | Monitoring/alerting tools (e.g., Prometheus, CloudWatch, custom dashboards) |
| CI/CD | GitHub Actions |
| Project Management | Linear (task tracking) |
| AI Engineering | Claude Code, Cursor, Codex (integrated into daily workflows) |
| Frontend (exposure) | NextJS / TypeScript (collaboration with UI teams; not a full-stack requirement) |
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Requirements
Required
- Proven Data Engineer experience (or equivalent role)
- Ability to identify business problems and design end-to-end data solutions (not just execute specs)
- Strong Python and SQL proficiency
- Hands-on AWS experience (e.g., EC2, S3, Lambda, RDS, DynamoDB)
- Infrastructure-as-Code (e.g., Terraform, CloudFormation)
- Comfort with production operations (monitoring, debugging, incident response)
- Exceptional stakeholder communication (bridging technical and non-technical teams)
- Familiarity with modern AI engineering tools (e.g., fine-tuning LLM-based workflows)
Preferred
- Background in financial services, asset management, or hedge funds
- MongoDB experience
- Exposure to modern data tools (e.g., Apache Airflow, dbt, Presto)
- Familiarity with vendor API integrations and handling messy real-world data
- Experience with data lake patterns (e.g., Partitioning, delta lake, iceberg)
- Curiosity with NextJS / TypeScript (though not mandatory)
- Understanding of RAG architectures, OCR, or LLM-based extraction
- Comfort with agile workflows and lightweight project delivery
Who Thrives Here?
You’ll excel if you: 🔹 Proactively solve problems (don’t wait for instructions) 🔹 Own systems end-to-end with full accountability 🔹 Balance feature development with production stability 🔹 Prefer pragmatic, simple solutions over overly theoretical abstractions 🔹 Wear multiple hats (engineering, operations, stakeholder management) 🔹 Communicate clearly with non-technical teams 🔹 Enjoy autonomy and trust in your technical judgments
Interested? Send your CV for immediate consideration.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location