Required Technical Skills & Experience

Experience Baseline

• 10+ years in enterprise architecture, infrastructure engineering, or platform engineering roles.
• 5+ years focused specifically on AI/ML systems design and delivery, including at least 2 years working with generative AI/LLM workloads.
• Demonstrated track record leading technical delivery (not just advisory) on enterprise-scale AI or HPC infrastructure programmes.

AI Hardware & Data Center Infrastructure

• GPU/accelerator architectures: NVIDIA / AMD, including multi-node scale-out design.
• Accelerator interconnects: NVLink, NVSwitch.
• High-performance networking: InfiniBand and RoCEv2 fabric design, 400G/800G Ethernet, rail-optimized topologies for AI clusters.
• Data center facilities: power density, liquid cooling, and rack-level design considerations specific to AI compute.
• Storage: parallel and high-throughput file systems (e.g. Everpure, WEKA, VAST, NetApp) sized for training and checkpointing workloads.

AI Software, MLOps & Generative AI

• Orchestration & containers: Kubernetes, Docker, Slurm, Run:ai or equivalent GPU scheduling platforms.
• ML frameworks: PyTorch and TensorFlow at a working, hands-on level.
• Distributed training: Horovod, DeepSpeed, Megatron-LM, or equivalent multi-node training frameworks.
• Inference & serving: NVIDIA Triton, vLLM, TensorRT-LLM, or equivalent high-throughput serving platforms.
• MLOps/LLMOps: Kubeflow, MLflow, and at least one hyperscaler ML platform (SageMaker, Azure ML, or Vertex AI).
• Generative AI: LLM fine-tuning (LoRA/QLoRA), RAG architecture design, vector databases (Pinecone, Milvus, Weaviate), and agentic frameworks (LangChain, LangGraph, Semantic Kernel).
• Data pipelines: data lake/lakehouse architectures, ETL/ELT, and data quality/lineage tooling that feed AI systems.

Automation, Orchestration & Observability

• Infrastructure-as-code: Terraform and Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery.
• Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines.
• Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager for GPU job scheduling; Kubernetes-native GPU scheduling including device plugins and MIG partitioning for multi-tenant clusters.
• CI/CD/CT for ML: automated model testing, validation gates, and promotion pipelines (continuous training/continuous delivery) that move models safely from experimentation to production.
• Infrastructure & GPU observability: NVIDIA DCGM, Prometheus/Grafana, and related telemetry stacks for GPU utilization, thermal, and cluster health monitoring.
• Model & LLM observability: production model performance monitoring, data/concept drift detection, and LLM-specific observability (token usage, latency, cost, hallucination/quality metrics) using tools such as Arize, WhyLabs, or Langfuse.
• Logging & tracing: centralized logging (ELK/OpenSearch) and distributed tracing (OpenTelemetry) across data, training, and inference pipelines for end-to-end root-cause analysis.

Integration — AI Stack, Enterprise Networks & Service Provider Environments

• Platform integration: API-based and event-driven integration of AI platforms with enterprise systems, using REST/gRPC APIs and message/event streaming platforms (e.g. Kafka).
• Enterprise network integration: designing connectivity between AI/GPU infrastructure and existing campus, data center, and WAN environments, including capacity and latency planning for east-west training traffic and north-south inference traffic.
• Hybrid & multi-cloud connectivity: integrating on-premises AI platforms with cloud AI services via dedicated interconnects (Direct Connect, ExpressRoute) and multi-cloud/hybrid connectivity patterns for distributed training or burst inference.
• Service provider & third-party integration: experience architecting connections into external or service-provider-hosted environments — colocation interconnects, managed GPU-as-a-service offerings, and third-party/external inference endpoints — including the commercial and technical boundary considerations involved.
• Secure exposure of AI services: working knowledge of API gateways, service mesh, and mutual TLS as applied to exposing or consuming AI services safely across organizational and network boundaries.
• Cross-functional design: proven ability to partner directly with network and security architects to define end-to-end integration architecture spanning AI platforms, enterprise networks, and external/customer environments.

Governance, Risk & Compliance

• Working knowledge of model risk management frameworks and responsible AI principles (fairness, explainability, human oversight).
• Familiarity with data privacy regulation (GDPR, CCPA) as applied to AI training and inference data.
• Working knowledge of emerging AI-specific regulation and standards: EU AI Act, NIST AI Risk Management Framework, ISO/IEC 42001.
• Experience establishing model documentation, audit trail, and approval-gate processes for production AI systems.

Security & Cloud

• AI-specific security fundamentals: model security, prompt-injection defenses, supply chain security for open-source/open-weight models.
• Solutions-architect level expertise in at least one hyperscaler (AWS, Azure, or GCP), including their native AI/ML services.
• Ability to design for hybrid on-premises/cloud AI deployments, including data residency and sovereignty constraints.

Architecture Governance & Design Authority

• Proven experience hosting and chairing formal Architecture Review Board (ARB) and Technical Design Authority (TDA) forums, including agenda ownership, decision logging, and stakeholder facilitation.
• Ability to define and operate governance gates across the engagement lifecycle: design authority sign-off, change control, and exception/waiver management for AI platform decisions.
• Experience producing and maintaining architecture decision records (ADRs), design standards, and reference architectures that are actively enforced through ARB/TDA governance.
• Comfortable presenting and defending architecture decisions to senior client governance bodies, including constructively challenging non-compliant or high-risk designs.

Communication & Delivery

• Excellent executive communication and presentation skills; able to flex between deep technical detail and C-suite framing.
• Proven ability to author low-level design (LLD) and high-level design (HLD) documentation to a professional services standard.