Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Data Scraping Engineer (Freelance, Remote in Sweden)

Mindrift is seeking a senior Python engineer to build dependable scraping workflows for AI projects. The role focuses on extracting, validating, and scaling structured data from complex, dynamic websites.

Description

  • Lead complete data extraction workflows across complex websites
  • Deliver comprehensive, accurate, and dependable structured datasets
  • Apply available tools and tailored workflows to speed up data collection, validation, and task completion
  • Collect data from dynamic, interactive websites while adapting to JavaScript-rendered content and changing site behavior
  • Protect data quality with validation checks, cross-source comparisons, formatting standards, and systematic verification
  • Scale large-volume scraping through efficient batching and parallel processing
  • Track extraction failures and preserve system stability through minor website structure changes
  • Use Apify, OpenRouter, and other relevant technologies in combination with strong technical judgment

Requirements

  • At least five years of relevant experience in data engineering, web scraping, automation, or software development
  • Advanced Python scraping experience with BeautifulSoup, Selenium or comparable tools, dynamic content such as JavaScript, AJAX, and infinite scroll, and proxy-based APIs
  • Ability to extract information from complex structures, including hierarchies, archived pages, and inconsistent HTML
  • Experience cleaning, normalizing, and validating data for structured CSV, JSON, or Google Sheets outputs
  • Practical experience managing anti-bot protections and changing website structures at scale
  • Experience using AWS or comparable cloud infrastructure and Docker containers in production workflows
  • Hands-on experience applying LLM frameworks such as LangChain, OpenRouter, or similar tools to automation
  • Strong attention to detail with a consistent focus on data accuracy
  • Independent, self-directed approach to troubleshooting and problem-solving
  • Upper-intermediate English proficiency (B2) or higher
  • Bachelor’s or master’s degree in engineering, applied mathematics, computer science, or a related technical discipline is a plus
  • A GitHub profile is a plus

Benefits

  • Freelance engagement
  • Part-time remote work
  • Estimated 10–20 hours per week during active project phases

Related Jobs

Malbek

Senior Full-Stack Developer — Malbek, Remote United States

Malbek

Build and scale Malbek’s SaaS platform across Angular, Spring, APIs, and databases. Deliver reliable, high-performance features end to end for customers in the United States.

Open
Talkdesk

Senior Solution Consultant, CXA and CCaaS - United States

Talkdesk

Lead strategic implementations of Talkdesk CXA and CCaaS solutions, helping customers integrate the technology and achieve lasting adoption. Partner with customers, executives, and internal teams to guide implementation strategy and long-term success.

Open