Model selection: open-weight vs. closed, and why

• Hold a documented, evidence-based framework for choosing between closed frontier models accessed through a provider's API (Anthropic, OpenAI, Google) and open-weight models run on Citation's own infrastructure (Llama, Mistral, Qwen and similar) — and apply it per workload, not as a single blanket choice.
• The framework should weigh:
  • Task complexity — complex reasoning and long-horizon agentic work generally favour frontier closed models; simpler classification, extraction, and templated generation are often better served by smaller open-weight models at a fraction of the cost
  • Volume and cost at scale — at high token volumes, self-hosting open-weight models can cross into materially cheaper territory; this role owns the analysis of where that break-even sits for Citation's actual workloads, not a generic industry number
  • Data sensitivity and residency — for HR, employment, and compliance data, self-hosted open-weight models remove a class of third-party data-sharing risk that closed providers (even with strong contractual terms) don't fully eliminate
  • Fine-tuning and customisation — where an open-weight model fine-tuned on Citation's own data would outperform a general-purpose model at lower running cost, that's a build case this role should be able to make with evidence, not intuition
  • Latency, licensing, and total cost of ownership — including the practical overhead of running and maintaining your own models, not just the sticker price of tokens

Tokenomics — cost as an architectural input, not an afterthought

• Understand and actively manage the unit economics of every AI solution recommended, including:
  • Modelling likely token cost at design time, before a system is built, so the business knows what it's signing up for
  • Specifying model routing (cheaper models for simpler steps, frontier models reserved for what actually needs them), prompt and context engineering for efficiency, and caching where appropriate
  • Recognising that agentic and multi-agent patterns can multiply token consumption several-fold over a single well-scoped call, and that orchestration-pattern choice is itself a cost decision
  • Setting up the observability to track cost-per-outcome and cache performance over time, not just the total spend line

Data architecture and readiness

• Assess whether the data behind any AI initiative is structured, accessible, and reliable enough to support the intended behaviour — across Citation's Salesforce, Atlas, Snowflake, and integration layers — and flag readiness issues before they become delivery blockers.

Design authority and governance

• Hold the approval path for new AI architectural patterns and material departures from existing standards.
• Attend initiative design conversations early enough to shape them, not just review them, and sign off on the AI design elements of partner Statements of Work.

Proof of concept and technical validation

• Define scope and success criteria for AI spikes and proofs of concept before they start, validate vendor and partner capability claims before they're embedded in a committed design, and make sure proof-of-concept outputs are evaluated against real delivery constraints — not vendor demo conditions.

Third-party delivery oversight

• Act as the internal architectural counterpart to Citation's delivery partners, the way any enterprise architecture function holds its critical vendors to a defined standard: review proposals and Statements of Work before commitments are made, run design reviews during delivery, and give partners a well-defined target with technical challenge where their decisions need scrutiny.

Production quality, observability, and cost

• Set the standard for how AI systems are monitored and evaluated once live — offline and online evaluation, drift detection, guardrails — and own the response framework for AI incidents in production, alongside the cost governance described above.

Security and compliance

• Bring Security in as a standing stakeholder on every significant AI initiative, addressing data boundaries, personal data handling, prompt injection risk, and third-party model provider assessments early rather than at the end.
• Maintain alignment with the ISO 27001 information security standard and data protection law, with particular care where personal employment or HR data is used as model input — this is where the open-weight-vs-closed decision above earns its keep.

Responsible AI

• Define and maintain guardrails on what AI systems should and shouldn't decide autonomously, ensure outputs are explainable where they affect clients or employees, assess new initiatives for bias risk, and track emerging responsible-AI regulation for its implications on Citation's systems.

Internal enablement

• Run informal sessions or working groups to build AI literacy across Engineering, Product, and the Business.
• Be a first point of contact for AI questions and support onboarding of technical staff into Citation's AI standards.

Horizon scanning and strategic input

• Monitor the AI landscape — including the open-weight ecosystem specifically, given how fast it's moving — and evaluate new tools, models, and platforms against Citation's actual needs before they gain internal momentum on hype alone.
• Give the Chief AI Officer a clear, current view of AI architectural risk and direction across the portfolio.

What this role requires

Must have

• 6+ years in software, data, or platform engineering or architecture roles, including meaningful hands-on time on production AI systems — pilots and proofs of concept alone won't have exposed you to the failure modes this role exists to prevent
• Practical experience choosing between closed frontier models and open-weight/self-hosted models for real workloads, not just familiarity with the debate
• Working knowledge of AI unit economics — token pricing, routing, caching, context engineering — sufficient to model cost credibly at design time
• Grounding in Citation's actual stack: Amazon Web Services' AI services, agentic and multi-model patterns
• Comfortable working with regulated, personal-data-heavy employment and compliance data — of the kind behind Citation's 120,000+ SME clients — and treating that as a design constraint from day one, not a compliance afterthought
• Working knowledge of information security standards and data protection law as they apply specifically to AI systems (prompt injection, model provider risk, data boundaries), not just general IT security