Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Remote Physician (MD/DO), Medical AI Evaluation — Florida

US physicians review clinical EHR vignettes and assess medical reasoning. Mindrift engages contributors on a project basis to help improve medical AI systems.

Description

  • Review clinical EHR vignettes with questions, proposed answers, and distractor answers for diagnosis and treatment tasks.
  • Assess cases across cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.
  • Rate clinical reasoning, vignette accuracy and completeness, answer correctness and gradeability, answer disclosure, and distractor quality.
  • Check that the reasoning derives the answer from the vignette facts alone.
  • Correct reasoning traces and provide brief rationales.
  • Complete three independent blind reviews followed by physician adjudication.

Requirements

  • Be a US-based physician currently providing patient care.
  • Hold an MD or DO degree.
  • Have an active, unrestricted US medical license, verified with the issuing medical board.
  • Have board certification or completed residency training.
  • Have recent direct patient-care experience; attending-level experience is preferred.
  • Bring broad diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.
  • Demonstrate strong clinical reasoning, including differential diagnosis, next-step management, and use of evidence-based guidelines.
  • Experience in medical AI evaluation, clinical content or exam-question review, or medical annotation or quality assurance is a plus.
  • Have strong written English skills (C1 or higher).
  • This role is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or people in non-clinical healthcare roles.
  • Submit tasks by their deadlines and meet the stated acceptance criteria.

Benefits

  • Take part in project-based, part-time work on non-permanent projects.
  • Receive payment for each accepted task.
  • Earn an effective rate of up to the equivalent of $130 per hour, depending on qualification tier and task efficiency.

Related Jobs

CHAP

National DMEPOS Compliance Surveyor

CHAP
51 – 200 Employees
ConsultingHealthcareNon-profit

Inspect medical equipment suppliers across the United States for CHAP, assessing compliance with CMS standards. The role includes site visits, investigations, regulatory reviews, and nationwide travel.

Open