Mindrift
Mindrift
Mindrift is an artificial intelligence talent marketplace and SaaS platform that connects professional contributors and domain experts with AI training and evaluation projects from technology companies. As part of Toloka, it works with writers, editors, QA specialists, scientists, and other subject-matter experts who help generate training data, assess model safety and reasoning, evaluate media quality, and create real-world scenarios for improving AI performance and reliability. Mindrift recruits and vets contributors through assessments and identity verification, then offers flexible paid project opportunities across AI training and evaluation work.

Senior Python Data Scraping Engineer — Freelance, India Remote

Mindrift is seeking a senior Python engineer to extract, validate, and scale structured web data workflows for the Tendem AI project. The role involves Python, cloud infrastructure, Docker, and LLM-based automation frameworks.

Description

  • Manage complete data extraction workflows across complex websites
  • Deliver structured datasets with comprehensive coverage, high accuracy, and dependable turnaround
  • Use specialized tools and tailored workflows to speed up collection, validation, and task execution
  • Retrieve information from dynamic and interactive sources, including JavaScript-rendered pages
  • Adjust scraping methods as website behavior and page structures change
  • Maintain data quality through validation, cross-source checks, formatting standards, and systematic review
  • Scale large-volume scraping through efficient batching and parallel processing
  • Track failures and support reliable, stable scraping operations
  • Apply expertise in Python, web scraping, data extraction, and data processing to the Tendem project

Requirements

  • 5+ years of relevant experience in data engineering, web scraping, automation, or software development
  • A bachelor’s or master’s degree in engineering, applied mathematics, computer science, or a related technical field is a plus
  • Strong practical foundation in scripting, automation, and data extraction workflows
  • Advanced Python scraping experience with BeautifulSoup, Selenium or comparable tools, dynamic content, and proxy-based APIs
  • Ability to extract information from complex structures, including hierarchies, archived pages, and inconsistent HTML
  • Strong experience with data cleaning, normalization, and validation
  • Practical experience managing anti-bot protections and dynamic website structures at scale
  • Experience with AWS or comparable cloud infrastructure and Docker containerization
  • Hands-on use of LangChain, OpenRouter, or similar LLM frameworks for automation
  • Meticulous approach to data accuracy and quality
  • Able to work independently, troubleshoot issues, and manage tasks proactively
  • Upper-intermediate English proficiency (B2) or higher
  • A GitHub link is a plus

Benefits

  • Freelance engagement
  • Part-time remote work from India
  • Approximately 10–20 hours per week during active project phases
  • Opportunities to work on technology projects through the Mindrift platform
  • Compensation of up to $30 per hour equivalent, depending on experience level and contribution pace

Related Jobs

Talkdesk

Senior Solution Consultant, CXA and CCaaS - United States

Talkdesk

Lead strategic implementations of Talkdesk CXA and CCaaS solutions, helping customers integrate the technology and achieve lasting adoption. Partner with customers, executives, and internal teams to guide implementation strategy and long-term success.

Open
Naveera Technology LLC

Senior .NET Core Engineer – React and Claude AI

Naveera Technology LLC
201 – 500 Employees
ConsultingHealthcareLogistics

Lead full-stack engineering work across insurance and payments platforms using .NET, React, AWS, and Claude AI. Drive architecture, batch processing, and distributed-systems initiatives.

Open