Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Data Scraping Engineer – Freelance Remote

Mindrift is seeking a senior Python engineer to build dependable web-scraping and data-processing workflows for AI projects. The role focuses on extracting, validating, and scaling structured datasets from complex websites.

Description

  • Lead end-to-end data extraction workflows across complex websites
  • Deliver complete, accurate, and dependable structured datasets
  • Apply available tools and custom workflows to speed up data collection, validation, and task execution
  • Collect data from dynamic and interactive sources while adapting to JavaScript-rendered content and changing website behavior
  • Maintain data quality through validation checks, cross-source consistency reviews, formatting requirements, and systematic verification
  • Scale large-dataset scraping through efficient batching or parallel processing
  • Monitor failures and preserve scraper stability through minor website structure changes
  • Work with Apify, OpenRouter, and other relevant technologies

Requirements

  • 5+ years of relevant experience in data engineering, web scraping, automation, or software development
  • Upper-intermediate English proficiency at B2 level or higher
  • Strong Python scraping experience with BeautifulSoup, Selenium or comparable tools, dynamic content such as JavaScript, AJAX, and infinite scroll, and proxy-based APIs
  • Ability to extract information from complex structures, including hierarchies, archived pages, and inconsistent HTML
  • Experience cleaning, normalizing, and validating data for delivery in CSV, JSON, or Google Sheets
  • Experience managing anti-bot mechanisms and changing website structures at scale
  • Practical experience with AWS or equivalent cloud infrastructure and Docker containerization
  • Hands-on experience using LangChain, OpenRouter, or similar LLM frameworks for automation
  • Strong attention to detail and dedication to data accuracy
  • Ability to work independently and troubleshoot effectively
  • A Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related technical discipline is a plus
  • A GitHub profile link is a plus

Benefits

  • Freelance engagement
  • Part-time remote work
  • Approximately 10–20 hours per week during active project phases
  • Up to $40 per hour equivalent, depending on experience level and contribution pace

Related Jobs

Mindrift

Senior Python Data Scraping Engineer (Freelance, Remote in Sweden)

Mindrift

Mindrift is seeking a senior Python engineer to build dependable scraping workflows for AI projects. The role focuses on extracting, validating, and scaling structured data from complex, dynamic websites.

Open
Mindrift

Senior Python Data Scraping Engineer (Freelance, Remote in Saudi Arabia)

Mindrift

Senior Python engineer developing scalable scraping workflows for Mindrift’s Tendem AI project. The role focuses on extracting, validating, and structuring complex web data with Python and automation tools.

Open
Mindrift

Senior Python Data Scraping Engineer — Freelance Remote in Romania

Mindrift

Mindrift is seeking a senior Python engineer to build dependable scraping and data-extraction workflows for AI projects. The role focuses on converting dynamic websites into accurate, validated, structured datasets.

Open
Mindrift

Senior Python Data Scraping Engineer — Remote in the Philippines

Mindrift

Mindrift is seeking a senior Python engineer to build dependable scraping workflows for AI projects. The role focuses on extracting, validating, and scaling structured data from complex, dynamic websites.

Open