Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Remote Physician for Medical AI Evaluation

A practicing US physician evaluates clinical vignettes, reasoning, and treatment responses. The work supports medical AI through blind reviews, scoring, and physician adjudication.

Description

  • Review EHR-based clinical vignettes, questions, proposed answers, and distractors for diagnosis and treatment tasks
  • Rate clinical reasoning, vignette accuracy and completeness, answer correctness and gradeability, and distractor quality
  • Verify that each reasoning chain reaches its answer using only the facts in the vignette
  • Edit reasoning traces and provide brief supporting rationales
  • Take part in three independent blind reviews followed by physician adjudication
  • Work on project tasks estimated at 10–20 hours per week during active phases
  • Help improve medical AI tools through clinical evaluation

Requirements

  • Be a US-based, actively practicing physician
  • Hold an MD or DO degree
  • Have an active, unrestricted US medical license, verified by the issuing medical board
  • Be board-certified or have completed residency training
  • Have recent direct patient-care experience; attending-level experience is preferred
  • Have broad diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine
  • Demonstrate strong clinical reasoning in differential diagnosis, next-step management, and applying evidence-based guidelines
  • Experience in medical AI evaluation, clinical content or exam-question review, or medical annotation or quality assurance is a plus
  • Have strong written English proficiency (C1 or higher)
  • This role is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or people in non-clinical healthcare roles
  • Submit a CV in English and state your English proficiency level
  • Submit tasks by their deadlines and meet the stated acceptance criteria

Benefits

  • Project-based participation
  • Part-time, non-permanent project work
  • Earn up to the equivalent of $130 per hour, paid per accepted task; this is not a base salary
  • Choose a flexible workload according to project needs during active phases

Related Jobs