Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Remote Physician for Medical AI Evaluation

US-based physicians review clinical EHR vignettes and assess reasoning quality for Mindrift’s project-based medical AI programs. Evaluate diagnosis and treatment benchmarks.

Description

  • Review EHR vignettes, questions, proposed answers, and distractors for diagnosis and treatment tasks.
  • Assess cases across cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.
  • Rate clinical reasoning, vignette accuracy and completeness, answer disclosure and correctness, gradeability, and distractor quality.
  • Verify that each reasoning chain reaches its answer using only facts from the vignette.
  • Revise reasoning traces and provide brief rationales.
  • Take part in three independent blind reviews followed by physician adjudication.
  • Meet task deadlines and stated acceptance criteria.

Requirements

  • Provide a CV in English and state your English proficiency level.
  • Be a US-based, actively practicing physician with an MD or DO.
  • Hold an active, unrestricted US medical license verified by the issuing medical board.
  • Have board certification or completed residency training.
  • Have recent direct patient care experience; attending-level experience is preferred.
  • Bring broad diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.
  • Demonstrate strong clinical reasoning in differential diagnosis, next-step management, and evidence-based guidelines.
  • Experience with medical AI evaluation, clinical content or exam-question review, or medical annotation and quality assurance is a plus.
  • Have strong written English skills (C1 or higher).
  • This role is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare professionals.

Benefits

  • Part-time, temporary work on a project basis.
  • Payment is issued for each accepted task.
  • Compensation can reach the equivalent of $130 per hour, depending on qualification tier and task efficiency.

Related Jobs

Kreato Global | BPO and Language Solutions

Remote English-Spanish OPI/VRI Interpreter

Kreato Global | BPO and Language Solutions
201 – 500 Employees
HealthcareHospitalityLogistics

Interpret remotely between English- and Spanish-speaking people in medical, financial, social service, and customer care settings. Provide language support for Kreato Global across Latin America.

Open