Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Web Scraping Engineer - Brazil Remote

Senior Python engineer responsible for dependable web scraping, data validation, and structured dataset delivery for Mindrift’s AI technology projects. The role covers dynamic websites, anti-bot systems, cloud infrastructure, and LLM-based automation tools.

Description

  • Lead complete data extraction workflows across complex websites
  • Deliver structured datasets with comprehensive coverage, high accuracy, and dependable turnaround
  • Use available tools and tailored workflows to speed up data collection, validation, and task execution
  • Extract information from dynamic and interactive sources, including JavaScript-rendered pages
  • Adjust scraping methods as website behavior changes
  • Maintain data quality through validation, cross-source checks, formatting requirements, and systematic verification
  • Scale large-volume scraping through efficient batching or parallel processing
  • Track failures and preserve stability through minor changes in website structure
  • Integrate tools including Apify, OpenRouter, and related technologies into scraping and processing workflows

Requirements

  • At least five years of relevant experience in data engineering, web scraping, automation, or software development
  • A bachelor’s or master’s degree in engineering, applied mathematics, computer science, or a related technical discipline is a plus
  • Strong technical fundamentals with practical experience in scripting, automation, and data extraction workflows
  • Extensive Python scraping experience with BeautifulSoup, Selenium or comparable tools, dynamic content, and proxy-based APIs
  • Ability to extract information from complex structures, including hierarchies, archived pages, and inconsistent HTML
  • Solid experience with data cleaning, normalization, and validation
  • Proven ability to manage anti-bot systems and dynamic website structures at scale
  • Experience with AWS or comparable cloud infrastructure and Docker containerization
  • Practical experience applying LangChain, OpenRouter, or similar LLM frameworks to automation tasks
  • Strong attention to detail and dedication to data accuracy
  • Self-directed approach with the ability to troubleshoot independently
  • Upper-intermediate English proficiency (B2) or higher
  • A GitHub profile link is a plus

Benefits

  • Freelance engagement
  • Part-time remote work
  • Approximately 10–20 hours per week during active project phases
  • Compensation of up to $25 per hour equivalent, depending on experience level and contribution pace
  • Opportunity to support innovative technology and AI development projects

Related Jobs

Malbek

Senior Full-Stack Developer — Malbek, Remote United States

Malbek

Build and scale Malbek’s SaaS platform across Angular, Spring, APIs, and databases. Deliver reliable, high-performance features end to end for customers in the United States.

Open
Talkdesk

Senior Solution Consultant, CXA and CCaaS - United States

Talkdesk

Lead strategic implementations of Talkdesk CXA and CCaaS solutions, helping customers integrate the technology and achieve lasting adoption. Partner with customers, executives, and internal teams to guide implementation strategy and long-term success.

Open