N Consulting Global
Kafka Architect
London
Posted about 12 hours ago
Get notified of more jobs like this · No spam, ever
How your CV stacks up
1Upload CV
2Analyse CV
3Improve CV
Upload your CV to see how well it fits this job role
?%
Role: Kafka Architect
Location: Malmo, Sweden (Hybrid) (Any where is Europe is fine)
Job Description
We are seeking an Expert Kafka professional with overall experience of 8 to 12+ Years.
5+ years of deep expertise in designing and implementing enterprise-scale event streaming solutions using Confluent Kafka. The ideal candidate will possess extensive experience in large-scale Confluent Kafka infrastructure (3+ years) and designing complex Kafka integration solutions (5+ years) that power real-time data pipelines, event-driven architectures, and mission-critical business applications. This senior role requires a technical professional who can design highly available, scalable, and secure Kafka ecosystems, establish streaming best practices, and drive the adoption of event-driven patterns across the organization. The Expert Kafka professional will serve as the subject matter expert for all Kafka-related initiatives, mentoring engineering teams, optimizing platform performance, and ensuring the reliability of streaming data infrastructure that supports business-critical operations.
Roles & Responsibilities
Confluent Kafka Infrastructure Architecture (3+ Years)
- Architect and design enterprise-scale Confluent Kafka infrastructure supporting high-throughput, low-latency event streaming requirements
- Design Kafka cluster topologies including multi-region, multi-datacenter, and hybrid cloud deployments
- Plan and implement Kafka broker sizing, partition strategies, replication factors, and cluster scaling patterns
- Architect Confluent Platform components: Confluent Server, ZooKeeper/KRaft, Schema Registry, Kafka Connect, ksqlDB, Confluent Control Center
- Design high-availability and disaster recovery strategies for Kafka clusters including backup, replication, and failover mechanisms
- Implement Kafka security architectures including SSL/TLS encryption, SASL authentication, and ACL-based authorization
- Optimize Kafka infrastructure for performance, throughput, and cost-efficiency across cloud and on-premises environments
- Evaluate and select deployment options: Confluent Cloud, Confluent Platform on-premises, or self-managed Kafka on cloud infrastructure
Kafka Integration Solutions Design (5+ Years)
- Design end-to-end event streaming solutions using Kafka for real-time data integration and event-driven architectures
- Architect Kafka Connect solutions for integrating with databases, cloud storage, SaaS applications, and legacy systems
- Design stream processing applications using Kafka Streams, ksqlDB, or Apache Flink for real-time data transformation
- Implement event sourcing and CQRS (Command Query Responsibility Segregation) patterns using Kafka as the event store
- Design microservices integration patterns using Kafka for asynchronous communication and event-driven choreography
- Architect data pipeline solutions for ETL/ELT, data synchronization, and real-time analytics use cases
- Design schema management strategies using Confluent Schema Registry with Avro, Protobuf, and JSON Schema
- Implement exactly-once processing semantics and transactional messaging patterns for critical business applications
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Performance Optimization & Tuning
- Optimize Kafka cluster performance: throughput tuning, latency reduction, and resource utilization
- Tune Kafka producer and consumer configurations for optimal performance and reliability
- Design and implement partitioning strategies for balanced load distribution and parallel processing
- Monitor and optimize Kafka consumer lag, rebalance operations, and consumer group management
- Implement Kafka compaction strategies and log retention policies for storage optimization
- Tune JVM settings, garbage collection, and OS-level configurations for Kafka brokers
- Conduct performance benchmarking and capacity planning for future growth
Security & Governance
- Architect comprehensive Kafka security models including authentication, authorization, and encryption
- Implement SSL/TLS encryption for data in transit and encryption at rest for sensitive data
- Design SASL-based authentication mechanisms: SASL/PLAIN, SASL/SCRAM, SASL/GSSAPI (Kerberos), SASL/OAUTHBEARER
- Implement fine-grained ACLs and role-based access control for Kafka resources
- Design data governance frameworks for event streaming including data lineage, quality, and compliance
- Ensure compliance with regulatory requirements: GDPR, PCI-DSS, HIPAA, SOX for streaming data
- Implement audit logging and monitoring for security events and access patterns
Platform Operations & Reliability
- Design monitoring and observability solutions for Kafka infrastructure using Confluent Control Center, Prometheus, Grafana
- Implement alerting mechanisms for cluster health, consumer lag, and critical metrics
- Create runbooks and operational procedures for Kafka cluster maintenance and incident response
- Design automated failover and self-healing mechanisms for Kafka infrastructure
- Plan and execute Kafka upgrades, migrations, and platform maintenance with minimal downtime
- Implement disaster recovery strategies including MirrorMaker 2.0 for cross-cluster replication
- Conduct chaos engineering and resilience testing for Kafka infrastructure
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Integration Architecture
- Design integration patterns between Kafka and enterprise systems: databases, data lakes, data warehouses
- Architect Kafka-to-Kafka replication strategies for multi-region and multi-cloud scenarios
- Design integration with cloud-native services: AWS Kinesis, Azure Event Hubs, Google Pub/Sub
- Implement CDC (Change Data Capture) patterns using Debezium or similar tools with Kafka
- Design API integration patterns using Kafka as the messaging backbone
- Architect hybrid cloud and multi-cloud Kafka deployments with data sovereignty considerations
Development & Best Practices
- Establish Kafka development standards, coding guidelines, and best practices for engineering teams
- Design and review Kafka producer and consumer applications for performance and reliability
- Implement error handling, dead letter queues, and retry mechanisms for robust event processing
- Create reusable Kafka components, libraries, and templates for common patterns
- Establish testing strategies for Kafka applications including unit, integration, and chaos testing
- Mentor development teams on Kafka patterns, anti-patterns, and optimization techniques
- Conduct code reviews and architecture reviews for Kafka-based solutions
Stakeholder Collaboration
- Collaborate with data engineers, software engineers, and architects to design streaming solutions
- Work with security and compliance teams to ensure Kafka implementations meet regulatory requirements
- Partner with infrastructure and DevOps teams for Kafka deployment automation and CI/CD integration
- Engage with business stakeholders to understand streaming requirements and translate to technical solutions
- Present Kafka architecture designs, roadmaps, and recommendations to technical and executive audiences
- Contribute to internal communities of practice and knowledge sharing sessions
Innovation & Strategy
- Research and evaluate emerging technologies in the event streaming and real-time analytics space
- Develop Kafka platform roadmaps and migration strategies for legacy integration systems
- Explore new Confluent Platform features and capabilities for potential adoption
- Drive adoption of event-driven architecture patterns and streaming best practices
- Identify opportunities for automation, optimization, and cost reduction in Kafka infrastructure
- Contribute to open-source Kafka community and stay current with Kafka ecosystem developments
Trusted by 25,000+ job seekers
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Location