Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Web Scraping Engineer (Freelance, Portugal Remote)

Build dependable web scraping and data-processing pipelines for Mindrift’s Tendem AI project. Extract, validate, and scale structured datasets from complex and dynamic websites.

Description

  • Manage end-to-end extraction from complex websites, delivering complete, accurate, and dependable structured datasets
  • Use tools and tailored workflows to speed up data collection, validation, and task completion
  • Collect data from dynamic and interactive sources, adapting to JavaScript-rendered content and changing website behavior
  • Maintain data quality through validation checks, cross-source comparisons, formatting requirements, and systematic review
  • Expand scraping operations for large datasets through efficient batching or parallel processing
  • Track extraction failures and preserve stability when sites undergo minor structural changes
  • Use web scraping, data extraction, and data-processing expertise across specialized Tendem project workflows

Requirements

  • Bring at least five years of relevant experience in data engineering, web scraping, automation, or software development
  • Demonstrate strong Python scraping skills with BeautifulSoup, Selenium or comparable tools, JavaScript and AJAX content, infinite scroll, APIs, and proxies
  • Show the ability to extract information from complex hierarchies, archived pages, and inconsistent HTML
  • Have a solid foundation in data cleaning, normalization, and validation, with experience producing structured CSV, JSON, and Google Sheets datasets
  • Demonstrate experience managing anti-bot protections and changing site structures at scale
  • Have experience with AWS or comparable cloud infrastructure and Docker-based containerization
  • Have practical experience applying LLM frameworks such as LangChain, OpenRouter, or similar tools to automation tasks
  • Work with strong attention to detail and a consistent focus on data accuracy
  • Work independently, take ownership of troubleshooting, and maintain a self-directed approach
  • Have English proficiency at Upper-intermediate (B2) level or higher
  • A Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related technical discipline is a plus
  • A GitHub profile link is a plus

Benefits

  • Freelance engagement
  • Part-time workload estimated at 10–20 hours weekly during active project phases
  • Remote work arrangement
  • Opportunity to work on innovative technology projects through the Mindrift platform

Related Jobs

Innosphere

Senior Full-Stack Developer at Innosphere — Canada Remote

Innosphere
51 – 200 Employees
ConsultingLogisticsMarketing

Join Innosphere as a senior full-stack developer building and modernizing cloud applications across frontend and backend systems. Work with AWS infrastructure, APIs, scalable services, and automated testing for client projects.

Open
Swoop

Senior Full-Stack Engineer (Hybrid, Canada)

Swoop

Senior Full-Stack Engineer developing scalable APIs and web products for NimbleRx’s patient and pharmacy platform. The role covers end-to-end feature ownership, system performance, testing, and technical delivery.

Open
CyberCube

Lead Actuary, Cyber Risk Analytics - CyberCube (Hybrid, New York)

CyberCube

Lead actuarial analysis and develop statistical models and tools for CyberCube’s cyber-risk analytics platform serving global insurers. Stress-test and validate models and data, then communicate findings to internal teams and customers.

Open