Mindrift
Senior Python Data Scraping Engineer (Freelance)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Python Data Scraping Engineer (Freelance)
Senior Python Data Scraping Engineer (AI Pilot) – Part-Time Remote Opportunity
Position: Senior Python Data Scraping Engineer / AI Pilot (Part-Time, Remote – Freelance Tendem Project)
At Mindrift, we're seeking highly skilled Senior Python Data Scraping Engineers to join our Tendem project, collaborating with Tendem Agents on specialized data scraping workflows within a hybrid AI + human system.
About the Role
This is a part-time, fully remote role where you’ll serve as an AI Pilot—providing critical thinking, domain expertise, and quality control to guide Tendem Agents handling repetitive tasks. You’ll extract, process, and validate data from complex websites to deliver accurate, actionable results.
The project utilizes Apify, OpenRouter, and custom workflows, requiring a mix of technical precision, adaptability, and independent problem-solving.
Key Responsibilities
- End-to-End Data Extraction: Own workflows for structured data extraction across complex websites, ensuring full coverage, accuracy, and reliability.
- Scraping & Validation: Leverage Apify/OpenRouter + custom tools to automate extraction, validation, and execution while meeting precision requirements.
- Dynamic Content Handling: Extract data from JavaScript-rendered pages, AJAX-loaded content, and constantly evolving site structures.
- Data Quality Assurance: Implement validation checks, consistency controls, and formatting standards; cross-verify datasets before delivery.
- Scalable Scraping: Optimize workflows for large datasets via batching, parallelization, or edge-case handling; monitor failures for resilience.
Requirements
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Experience & Technical Skills
✔ 5+ years in data engineering, web scraping, automation, or software development (essential) ✔ Bachelor’s/Master’s degree in Engineering, Applied Mathematics, Computer Science, or related fields (preferred but not mandatory)
Core Technical Proficiencies
✔ Deep Python expertise (BeautifulSoup, Selenium, and dynamic content handling with JavaScript/AJAX) ✔ Experience scraping complex structures (enterprise sites, hierarchical data, inconsistent HTML archives) ✔ Proficiency in data cleaning/normalization to produce structured datasets (CSV, JSON, Google Sheets) ✔ Handling anti-bot mechanisms and scalable scraping operations facing changing website structures ✔ Practical knowledge of cloud infrastructure (AWS, cloud providers) and Docker/containerization ✔ Hands-on experience with LLM frameworks (LangChain, OpenRouter) for automation tasks ✔ GitHub portfolio (a plus)
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Soft Skills & Work Ethic
✔ Detail-oriented with a focus on data accuracy ✔ Self-directed and able to troubleshoot without constant supervision ✔ English proficiency at Upper-intermediate (B2) or above
Project Scope & Workload
- Time Commitment: 10–20 hours/week during active project phases (not guaranteed, tied to project milestones).
- Compensation: Up to $45/hour equivalent, based on expertise and deliverables. Note: Rates vary across projects.
About Mindrift
Mindrift connects AI innovators with real-world specialists to unlock the potential of Generative AI. Our platform accelerates projects from the world’s most groundbreaking organizations by integrating technical precision with human expertise.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills