Freelance Data Scraping Engineer (Python)
Mindrift is looking for highly skilled <strong>Web Scraping</strong> specialists to join the Tendem project (https://tendem.ai/) and drive specialized data scraping workflows for real-world use cases.<br><br>Mindrift is looking for highly skilled <strong>Python Data Scraping Engineers</strong> to join the Tendem project and drive specialized data scraping workflows for real-world applications.<br><br>In this role, you'll apply your expertise in web scraping, data extraction, and data processing to deliver accurate, reliable, and high-quality results. This part-time remote opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction, and processing.<br><br><strong>What We Do<br><br></strong>The Mindrift platform connects specialists with innovative technology projects. Our mission is to help develop high-quality AI technologies by combining real-world expertise from professionals across the globe with advanced AI development efforts.<br><br><strong>About The Role<br><br></strong>This is a freelance role for a Tendem project. <strong>As a Python Data Scraping Engineer,</strong> you'll handle data scraping tasks requiring technical precision for web extraction and processing, utilizing tools such as Apify, OpenRouter, and other technologies, alongside your own technical expertise and approaches.<br><br><strong>Key Responsibilities<br><br></strong><ul><li>Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets</li><li>Leverage available tools and custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements</li><li>Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior</li><li>Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery</li><li>Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes<br><br></li></ul><strong>Educational Qualifications<br><br></strong><ul><li>At least 3+ years of relevant experience in data engineering, web scraping, automation, or software development (required)</li><li>Bachelor's or Master's Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus<br><br></li></ul><em><strong>Academic and/or Professional Experience<br><br></strong></em>Candidates should have a strong technical foundation and practical experience with scripting, automation, and data extraction workflows. We are looking for specialists who can solve non-trivial problems, work confidently with modern web technologies and data processing tools, and systematically collect, structure, and validate data from diverse sources. A methodical, detail-oriented approach and the ability to work independently are essential.<br><br><strong>Technical Skills (Essential)<br><br></strong><ul><li>Strong experience in Python web scraping (BeautifulSoup, Selenium or similar), including dynamic content (JS, AJAX, infinite scroll) and APIs via proxies</li><li>Proven ability to extract data from complex structures (hierarchies, archived pages, inconsistent HTML)</li><li>Solid background in data cleaning, normalization, and validation, delivering structured datasets (CSV, JSON, Google Sheets)<br><br></li></ul><strong>Additional Requirements<br><br></strong><ul><li>Hands-on experience with LLMs and AI frameworks to enhance automation and problem-solving</li><li>Strong attention to detail and commitment to data accuracy</li><li>Self-directed work ethic with ability to troubleshoot independently</li><li>A link to GitHub is a plus</li><li>English proficiency: Upper-intermediate (B2) or above (required)<br><br></li></ul><strong>Project time expectations <br><br></strong>For this project, tasks are estimated to require around 10-20 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active.<br><br><strong>Compensation<br><br></strong>On this project, contributors can earn up to <strong>$24 per hour equivalent</strong>, depending on their level and pace of contribution.<br><br>Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements.