Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Remote Physician for Medical AI Evaluation

US-based physicians assess clinical vignettes, reasoning, and answers to help improve Mindrift’s medical AI systems. The work includes independent blind reviews and physician adjudication.

Description

  • Assess clinical EHR vignettes, questions, proposed answers, and distractors for diagnosis and treatment tasks.
  • Review cases across cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.
  • Rate clinical reasoning, vignette accuracy and completeness, answer correctness and gradeability, and distractor quality.
  • Verify that each reasoning chain reaches its answer using only facts provided in the vignette.
  • Revise reasoning traces and provide concise rationales.
  • Complete three independent blind reviews followed by physician adjudication.
  • Meet task deadlines and the stated acceptance criteria.

Requirements

  • Be a US-based, actively practicing physician.
  • Hold an MD or DO degree.
  • Have an active, unrestricted US medical license verified by the issuing medical board.
  • Be board-certified or have completed residency training.
  • Have recent direct patient-care experience; attending-level experience is preferred.
  • Bring broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
  • Demonstrate strong clinical reasoning in differential diagnosis, next-step management, and evidence-based guideline application.
  • Write English at a strong C1 level or above.
  • Experience in medical AI evaluation, clinical content or exam-question review, or medical annotation or quality assurance is a plus.
  • This role is not suited to medical students, unlicensed medical graduates, non-physician clinicians, or people in non-clinical healthcare roles.

Benefits

  • Payment is provided for each accepted task.
  • Compensation can reach the equivalent of $35 per hour, depending on qualification tier and task efficiency.
  • Part-time, project-based work.
  • Flexible workload during active project phases, estimated at 10–20 hours per week.

Related Jobs