Mindrift
Senior Python Data Scraping Engineer (Freelance)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Python Data Scraping Engineer (Freelance)
Senior Python Data Scraping Engineer (AI Pilot) – Mindrift & Tendem Project
About the Role
Mindrift is seeking highly skilled Senior Python Data Scraping Engineers to join the Tendem project and lead specialized data scraping workflows within our hybrid AI + human system.
As an AI Pilot at Mindrift, you’ll collaborate with Tendem Agents on repetitive tasks while applying critical thinking, domain expertise, and quality control to ensure accurate and actionable results.
This is a part-time, remote freelance opportunity ideal for professionals with expertise in web scraping, data extraction, and processing.
About Us
The Mindrift platform connects technical specialists with AI-driven projects from major tech innovators. Our mission is to harness the power of Generative AI by tapping into global expertise.
For this Tendem project, you’ll handle intricate data scraping tasks—leveraging tools like Apify, OpenRouter, and your own technical approaches—while delivering high-quality results.
Key Responsibilities
- Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and structured dataset delivery.
- Use Apify, OpenRouter, and custom workflows to streamline data collection, validation, and execution while meeting project requirements.
- Extract data from dynamic, interactive, and JavaScript-rendered content, adapting techniques to withstand changing site behavior.
- Enforce data quality standards through:
- Validation checks
- Cross-source consistency controls
- Formatting adherence
- Systematic verification prior to delivery
- Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor structural changes.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements
Core Experience & Skills
✔ 5+ years in data engineering, web scraping, automation, or software development – required. ✔ Bachelor’s or Master’s Degree in Computer Science, Engineering, or Applied Mathematics – preferred. ✔ Strong technical foundation in:
- Python web scraping (BeautifulSoup, Selenium, or similar)
- Dynamic content handling (JavaScript, AJAX, infinite scroll)
- APIs via proxies ✔ Proven ability to extract from complex structures like:
- HTML hierarchies
- Archived/static pages
- Inconsistent HTML layouts ✔ Expertise in data cleaning, normalization, and validation for structured outputs (CSV, JSON, Google Sheets). ✔ Experience handling anti-bot mechanisms and large-scale scraping challenges. ✔ Knowledge of cloud infrastructure (AWS or equivalent) and containerization (Docker). ✔ LLM framework experience (LangChain, OpenRouter) for automation tasks. ✔ Automatic problem-solving, detail orientation, and strong self-sufficiency.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Soft Skills & Attributes
- Upper-intermediate (B2) English fluency – required.
- Link to GitHub portfolio – a plus.
- Interest in scalable automation, ML-assisted workflows, and innovative data extraction.
Time Commitment & Compensation
This freelance role will require 10–20 hours per week (recommended estimate for active phases). Actual workload varies by project phase.
Compensation:
- $45/hour equivalent (based on experience and project requirements).
- Rates may vary across projects depending on scope and complexity.
Need more details? Apply via Mindrift for full project considerations.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills