Physician - Clinical AI Expert (Remote)

Job Description

We are seeking physicians to review, validate, and design clinical tasks to help advance frontier AI models across a range of clinical scenarios and formats.

The project is designed to surface where models perform poorly in clinical scenarios such as missed findings, incorrect clinical information, overconfidence, or failure to seek context before answering.

You will work asynchronously on a dedicated platform, within your own specialty and day-to-day scope of practice. This is a remote, flexible, and short-term project.

Key Responsibilities

• Scenario Design: Design realistic clinical scenarios and instructions that mirror the work you carry out day-to-day in your specialty.
• Content Review: Assess clinical outputs for accuracy, flagging anything incorrect, outdated, or can cause harm.
• Reference Answer Validation: Judge whether reference answers reflect what you would actually write in practice, including caveats, uncertainty, and requests for further information.
• Grading Criteria Assessment: Design and evaluate whether scoring criteria capture what matters clinically and whether each is weighted appropriately, particularly safety-critical items.
• Written Rationale: Provide short, specific reasoning where you disagree, and note the weakness each new item is designed to expose.
• Feedback & Continuous Improvement: Highlight edge cases and ambiguity to refine guidelines and improve consistency across reviewers.