Help a top AI lab evaluate and improve large language models through security-focused coding tasks

Bring your software-engineering judgment and hands-on security experience to work involving vulnerabilities, exploit verification and security patches.

This is a contracting engagement, with potential for a longer-term engagement. Remote candidates in the selected countries are eligible.

What you will do

• Evaluate coding tasks involving software vulnerabilities, exploit verification and security patches.
• Create high-quality coding prompts and reference answers for benchmark-style problems.
• Evaluate model outputs for code generation, refactoring, debugging and implementation.
• Identify and document model failures, edge cases and reasoning gaps.
• Compare private language models with leading external models.
• Build or configure coding environments for evaluation and reinforcement learning.
• Follow detailed annotation and evaluation guidelines consistently.