Mindrift
Senior Python Data Scraping Engineer (Freelance)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Python Data Scraping Engineer (Freelance)
Senior Python Data Scraping Engineer (Freelance)
Mindrift is seeking highly skilled Senior Python Data Scraping Engineers to join the Tendem project and lead specialized data scraping workflows within our hybrid AI + human system.
In this role, you’ll operate as an AI Pilot, collaborating with Tendem Agents that handle repetitive tasks while you provide critical thinking, domain expertise, and quality assurance to deliver accurate, actionable results. This part-time remote opportunity is perfect for professionals with hands-on experience in web scraping, data extraction, and processing.
About the Role
The Mindrift platform connects specialists with AI projects from leading tech innovators. Our mission is to unlock the potential of Generative AI by integrating real-world expertise from across the globe.
As a Senior Python Data Scraping Engineer, you’ll work as part of a freelance Tendem project, focusing on data scraping tasks requiring technical precision for web extraction and processing. Tools like Apify and OpenRouter will be provided, along with flexibility to develop your own tailored approaches.
Key Responsibilities
- Own end-to-end data extraction workflows for complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets.
- Leverage internal tools (Apify, OpenRouter) and custom workflows to streamline data collection, validation, and execution while meeting project requirements.
- Extract data reliably from dynamic and interactive web sources, adapting techniques to handle JavaScript-rendered content and evolving site structures.
- Enforce high-quality data standards through validation checks, cross-source consistency assessments, formatting compliance, and supervised verification before delivery.
- Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and ensure stability against minor HTML/CSS structure changes.
- Deliver datasets in formats like CSV, JSON, or Google Sheets.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements
- 5+ years of experience in data engineering, web scraping, automation, or software development (mandatory).
- Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related technical field (preferred).
- Proficiency in Python web scraping, including:
- Libraries such as BeautifulSoup, Selenium, or alternatives.
- Handling dynamic content (JavaScript, AJAX, infinite scroll).
- Experience with APIs, proxies, and adherence to anti-bot mechanisms.
- Strong ability to extract structured data from complex systems (hierarchies, archived pages, inconsistent HTML).
- Hands-on experience with data cleaning, normalization, and validation.
- Familiarity with cloud infrastructure (AWS or equivalent) and containerization (Docker) in real-world workflows.
- Experience integrating LLM frameworks (e.g., LangChain or OpenRouter) into automation tasks.
- Problem-solving mindset with a detail-oriented approach.
- Ability to work self-directed, troubleshoot independently, and meet tight deadlines.
- Linked GitHub account (advantageous).
- English proficiency: Upper-intermediate (B2) or higher (mandatory).
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Project Commitment & Compensation
- Time expectation: ~10–20 hours per week during active project phases (estimates vary per workload).
- Hourly rate: Up to $45/hr, subject to individual performance and project scope.
- Note: Compensation may differ across projects. Rewards are determined by project complexity, requirements, and deliverables.
Experience our fast-paced, AI-driven workflows as an AI Pilot while gaining exposure to emerging tools and data strategies. Apply today to shape the future of AI-assisted data extraction!
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills