Mindrift
Senior Python Data Scraping Engineer (Freelance)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Python Data Scraping Engineer (Freelance)
Senior Python Data Scraping Engineer (AI Pilot) – Freelance Role
About the Role
Mindrift is seeking highly skilled Senior Python Data Scraping Engineers to join the Tendem project, collaborating within a hybrid AI + human system framework. In this part-time remote freelance position, you'll function as an AI Pilot, blending critical thinking, domain expertise, and quality control to deliver accurate, structured, and actionable data—while partnering with AI agents to streamline repetitive tasks.
The Mindrift Mission
Our platform connects technical specialists with Generative AI projects from leading tech innovators. Our goal? Unlock the potential of AI through real-world expertise sourced globally.
Project Focus
This role supports a Tendem project requiring precise web scraping, data extraction, and processing using Apify, OpenRouter, and custom tools. You’ll design and execute end-to-end extraction workflows, ensuring robust automation of dynamic, javascript-heavy, andvariable-content websites.
Key Responsibilities
- End-to-End Data Extraction: Lead full-cycle scraping workflows, guaranteeing coverage, accuracy, and reliability of structured datasets.
- Tool Integration: Combine Apify/OpenRouter with custom scripts to accelerate extraction, validation, and task execution while adhering to project requirements.
- Dynamic Content Mastery: Extract data reliably from JavaScript-rendered, AJAX, or infinite-scroll interfaces, adapting strategies as site behaviors evolve.
- Data Quality Assurance: Enforce rigorous validation—cross-source reviews, format compliance, and systematic checks—before delivery.
- Scalability & Resilience: Optimize scraping pipelines for large datasets (parallel/batch processing), monitor failures, and handle minor structural shifts in targets.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements
- Experience: 5+ years in data engineering, web scraping, automation, or software development (mandatory).
- Education: Preference for Bachelor’s/Master’s degree in Engineering, Applied Mathematics, Computer Science, or related fields.
- Python Proficiency: Hands-on expertise with scrapy, BeautifulSoup, Selenium, or similar for static/dynamic content, including APIs, proxies, and complex HTML hierarchies.
- Data Sanitization: Skilled in data cleaning, normalization, and validation, delivering outputs in CSV, JSON, or Google Sheets format.
- Anti-Bot & Scalable Workflows: Proven ability to navigate anti-bot defenses, scale operations, and tolerate minor structural changes.
- Cloud/DevOps: Experience with AWS/containerization (Docker) in production workflows.
- LLM Integration: Familiarity with LangChain, OpenRouter, or similar frameworks for AI-assisted automation.
- Shrink-Sleuth Mindset: Methodical, detail-oriented, and self-motivated, with robust problem-solving and independent troubleshooting skills.
- English: Fluent at Upper-Intermediate (B2) or higher (required).
- Bonus: GitHub portfolio/access for showcasing projects.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Project Commitment
Flexible/part-time role with estimated workloads of 10-20 hours/week (varying by project phase). Active weeks only.
Compensation
- $45/hour equivalent (varies by expertise/complexity).
- Earnings adjust per project scope, tooling, and deliverable scope—offerings can range across platform opportunities.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills