hackajob
Senior Python Data Scraping Engineer (Freelance)
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Senior Python Data Scraping Engineer (Freelance)
Senior Python Data Scraping Engineer (AI Pilot) – Tendem Project
About the Role
We are collaborating with Mindrift to connect with highly skilled Senior Python Data Scraping Engineers to join the Tendem project, driving specialized web scraping workflows within their hybrid AI + human system.
In this part-time remote freelance role (as an "AI Pilot" at Mindrift), you’ll collaborate with Tendem Agents—AI tools handling repetitive tasks—while bringing critical thinking, domain expertise, and quality control to deliver accurate and actionable results.
This opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction, and processing.
What We Do
The Mindrift platform connects specialists with AI projects from leading tech innovators. Our mission is to unlock the potential of Generative AI by leveraging real-world expertise globally.
As a Senior Python Data Scraping Engineer, you’ll use tools like Apify and OpenRouter alongside your own solutions to handle data scraping tasks requiring precision.
Key Responsibilities
- Own end-to-end data extraction workflows across complex websites, ensuring:
- Full coverage, accuracy, and reliable delivery
- Delivery of structured datasets in usable formats
- Leverage internal tools (Apify, OpenRouter) and custom workflows to:
- Speed up data collection, validation, and task execution
- Meet defined quality and delivery standards
- Extract data from dynamic and interactive web sources, including:
- JavaScript-rendered content
- AJAX and changing site structures
- Enforce data quality standards through:
- Validation checks
- Source consistency controls
- Formatting adherence
- Systematic verification before delivery
- Scale scraped operations via:
- Batch processing and parallelization
- Failure monitoring and stability measures
- Adaptation to slight structural changes
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements
Experience & Technical Skills
- Minimum 5+ years in data engineering, web scraping, automation, or software development (required).
- Degree preferences:
- Bachelor’s or Master’s in Engineering, Applied Mathematics, Computer Science, or related fields (a plus).
- Must demonstrate:
- Strong technical foundation in scripting, automation, and AI-assisted workflows.
- Problem-solving skills for non-trivial challenges.
- Ability to ** confidently use Large Language Models (LLMs)** in automation.
- Methodical, detail-oriented approach for data collection and validation.
- Python Web Scraping Proficiency:
- Strong knowledge of BeautifulSoup, Selenium, or similar tools.
- Experience with dynamic content (Javascript, AJAX, infinite scroll).
- APIs with proxy support.
- Data Extraction & Processing:
- Experience extracting data from complex structures (hierarchies, archives, messy HTML).
- Expertise in data cleaning, normalisation, and validation.
- Consistently delivers datasets in formats like CSV, JSON, or Google Sheets.
- Advanced Scraping Challenges:
- Experience avoiding anti-bot mechanisms and scaling for large datasets.
- Cloud & Containerization:
- Hands-on experience in AWS or equivalent.
- Practical use of Docker in workflows.
- LLM Integration:
- Key fine-tuning knowledge of frameworks like LangChain or OpenRouter.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Soft Skills
- High attention to detail and commitment to data accuracy.
- Self-directed work ethic with strong independent troubleshooting abilities.
- GitHub link as a bonus for collation of relevant work.
Language
- English proficiency at Upper-Intermediate (B2) or above (required).
Project Expectations & Compensation
Time Commitment
- 10–20 hours/week during active phases (flexible timeline). Estimate applies only while the project is live.
Compensation
- Competitive rate of up to $45/hour, based on skill level and contribution pace.
- Rates may vary by project scope, complexity, and expertise requirements.
Horizon Mindrift scale rates across partnerships and project priority.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location