Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Data Scraping Engineer (Freelance, Remote in Egypt)

Build dependable Python scraping systems for Mindrift’s AI projects. Extract, validate, and scale structured data from complex, dynamic websites.

Description

  • Manage complete data extraction workflows across complex websites
  • Deliver comprehensive, accurate, and dependable structured datasets
  • Use tools and tailored workflows to speed up data collection, validation, and task completion
  • Extract data reliably from dynamic, interactive, and JavaScript-rendered sources
  • Adjust scraping methods as website behavior and structures evolve
  • Protect data quality through validation, cross-source checks, formatting standards, and systematic review
  • Scale large-dataset scraping with efficient batching or parallel processing
  • Track extraction failures and preserve stability through minor website changes
  • Apply web scraping, data extraction, and data processing expertise to the Tendem project

Requirements

  • 5+ years of relevant experience in data engineering, web scraping, automation, or software development
  • Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related technical discipline is preferred
  • Advanced Python scraping experience with BeautifulSoup, Selenium or comparable tools, JavaScript/AJAX content, infinite scrolling, APIs, and proxies
  • Ability to extract information from complex structures such as hierarchies, archived pages, and inconsistent HTML
  • Strong data cleaning, normalization, and validation skills, with experience producing CSV, JSON, or Google Sheets datasets
  • Practical experience managing anti-bot protections and changing website structures at scale
  • Experience with AWS or comparable cloud infrastructure and Docker-based containerization
  • Hands-on use of LLM frameworks such as LangChain, OpenRouter, or similar tools for automation
  • Exceptional attention to detail and a consistent focus on data accuracy
  • Ability to work independently, troubleshoot issues, and manage tasks with minimal supervision
  • Upper-intermediate English proficiency (B2) or higher
  • GitHub profile or portfolio link is an advantage

Benefits

  • Freelance project engagement
  • Part-time workload of approximately 10–20 hours per week during active project phases
  • Work remotely
  • Flexible workload aligned with project needs

Related Jobs

Mindrift

Senior Python Web Scraping Engineer - Brazil Remote

Mindrift

Senior Python engineer responsible for dependable web scraping, data validation, and structured dataset delivery for Mindrift’s AI technology projects. The role covers dynamic websites, anti-bot systems, cloud infrastructure, and LLM-based automation tools.

Open
Mindrift

Senior Python Data Scraping Engineer — Remote in Romania

Mindrift

Senior Python engineer responsible for building dependable web-scraping and data-processing workflows for Mindrift’s Tendem AI project. The role focuses on extracting, validating, and scaling structured datasets from complex websites.

Open