Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Physician, Medical AI Evaluation

US physicians review clinical EHR vignettes and assess medical reasoning for Mindrift’s AI programs. This part-time, project-based work supports the evaluation of clinical AI systems.

Description

  • Review clinical EHR vignettes, questions, proposed answers, and distractor answers for diagnosis and treatment tasks.
  • Assess cases across cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.
  • Rate clinical reasoning, vignette accuracy and completeness, answer disclosure and correctness, gradeability, and distractor quality.
  • Check that each reasoning chain reaches its answer using only facts from the vignette.
  • Correct reasoning traces and provide brief rationales.
  • Complete three independent blind reviews followed by physician adjudication.
  • Submit work by deadlines and satisfy the stated acceptance criteria.

Requirements

  • Be a US-based, actively practicing MD or DO.
  • Hold a medical degree (MD or DO).
  • Have an active, unrestricted US medical license verified with the issuing medical board.
  • Be board-certified or have completed residency training.
  • Have recent direct patient-care experience; attending-level experience is preferred.
  • Have broad diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.
  • Demonstrate strong clinical reasoning in differential diagnosis, next-step management, and applying evidence-based guidelines.
  • Have strong written English skills at C1 level or higher.
  • Experience in medical AI evaluation, clinical content or exam-question review, or medical annotation or quality assurance is a plus.
  • Do not be a medical student, unlicensed medical graduate, non-physician clinician, or non-clinical healthcare professional.

Benefits

  • Payment is provided for each accepted task.
  • Compensation can reach the equivalent of $130 per hour, depending on qualification tier and task efficiency.
  • Project work is estimated at 10–20 hours per week during active phases.
  • Part-time, non-permanent project participation.

Related Jobs