AI Research Engineer, Model Optimization and Inference

The Mission

At Iconic, our virtual actors don't just generate “text” or “actions"—they perform. They need to speak, move, and perceive in milliseconds, often running locally on a player's machine alongside a rendering engine. You will bridge the gap between massive research models and the constraints of real-time interactive entertainment.

The Role

You will architect and build the inference engine that powers our digital entities. Your main task will be tearing apart the model architecture to make it run as fast as possible on consumer hardware while keeping their abilities intact for the intended usage.

As part of a small, focused team, you'll have significant autonomy and end-to-end ownership. You will work at the intersection of System ML and Game Tech. You might spend one day implementing a custom pruning algorithm for our TTS model, and the next day writing a C++ wrapper to expose that model to our game engine. You will work closely with our Character Research team to ensure that optimization never comes at the cost of the character's soul.

Key Responsibilities

• Architect Low-Latency Runtimes: Build and maintain high-performance inference pipelines for Multimodal LLMs, TTS, and Vision models, targeting both server-side (H100/A100) and consumer edge (RTX 5090, Apple Silicon) environments.
• State-of-the-Art Optimization: Implement advanced techniques like Speculative Decoding, KV-Cache quantization, PagedAttention, and Layer Pruning to minimize Time-To-First-Token (TTFT) and Time-Per-Output-Token (TPOT), maximizing throughput.
• Model Compression: Lead our efforts in post-training quantization (AWQ, GPTQ, GGUF) and distillation to fit massive models into consumer VRAM budgets.
• Engine Integration: Collaborate with the game engineering team to ensure thread-safe, non-blocking asynchronous inference within the game loop.
• Custom Kernel Development: Write custom ops in CUDA, Triton, or Metal when off-the-shelf kernels aren't fast enough.