Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Data Scraping Engineer (Freelance, Remote in Saudi Arabia)

Senior Python engineer developing scalable scraping workflows for Mindrift’s Tendem AI project. The role focuses on extracting, validating, and structuring complex web data with Python and automation tools.

Description

  • Lead specialized data-scraping workflows for Mindrift’s Tendem project and its real-world applications
  • Manage end-to-end extraction pipelines across complex websites
  • Produce complete, accurate, dependable, and well-structured datasets
  • Apply Apify, OpenRouter, and custom workflows to streamline collection, validation, and task execution
  • Capture data from dynamic, interactive sources while adapting to JavaScript-rendered content and changing site behavior
  • Protect data quality with validation checks, cross-source comparisons, formatting controls, and systematic verification
  • Scale large-volume scraping through efficient batching and parallel processing
  • Investigate failures and preserve pipeline stability through minor website structure changes
  • Independently solve complex scripting, automation, extraction, data-structuring, and validation challenges

Requirements

  • 5+ years of relevant experience in data engineering, web scraping, automation, or software development
  • Upper-intermediate English proficiency (B2) or higher
  • Advanced Python scraping experience with BeautifulSoup, Selenium or comparable tools, dynamic JavaScript/AJAX content, infinite scrolling, and proxy-based APIs
  • Ability to extract information from complex hierarchies, archived pages, and inconsistent HTML
  • Strong knowledge of data cleaning, normalization, and validation
  • Practical experience managing anti-bot protections and changing site structures at scale
  • Experience with AWS or comparable cloud infrastructure
  • Experience packaging applications with Docker
  • Hands-on use of LangChain, OpenRouter, or similar LLM frameworks for automation
  • High attention to detail with a strong focus on data accuracy
  • Self-motivated approach and confidence troubleshooting independently
  • Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related technical discipline is advantageous
  • GitHub profile link is advantageous

Benefits

  • Flexible part-time freelance engagement
  • Expected workload of 10–20 hours weekly during active project periods
  • Compensation of up to $37 per hour equivalent, based on experience level and contribution pace
  • Chance to contribute to innovative AI and technology development projects
  • Fully remote work

Related Jobs

Innosphere

Senior Full-Stack Developer at Innosphere — Canada Remote

Innosphere
51 – 200 Employees
ConsultingLogisticsMarketing

Join Innosphere as a senior full-stack developer building and modernizing cloud applications across frontend and backend systems. Work with AWS infrastructure, APIs, scalable services, and automated testing for client projects.

Open
Swoop

Senior Full-Stack Engineer (Hybrid, Canada)

Swoop

Senior Full-Stack Engineer developing scalable APIs and web products for NimbleRx’s patient and pharmacy platform. The role covers end-to-end feature ownership, system performance, testing, and technical delivery.

Open
CyberCube

Lead Actuary, Cyber Risk Analytics - CyberCube (Hybrid, New York)

CyberCube

Lead actuarial analysis and develop statistical models and tools for CyberCube’s cyber-risk analytics platform serving global insurers. Stress-test and validate models and data, then communicate findings to internal teams and customers.

Open