Jobgether
SWE-Bench AI Task Auditor - Freelance AI Trainer Project
Get notified of more jobs like this · No spam, ever
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
SWE-Bench AI Task Auditor - Freelance AI Trainer Project
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a SWE-Bench AI Task Auditor - Freelance AI Trainer Project based in United Kingdom.
This freelance opportunity is designed for experienced software engineering professionals who want to contribute their expertise to the development and evaluation of advanced AI systems.
Role Overview
You will audit software engineering tasks used to train and assess AI models, ensuring they reflect realistic and technically rigorous development scenarios.
- Evaluate software engineering tasks for technical accuracy, realism, solvability, reproducibility, and alignment with established SWE-Bench standards.
- Review complex codebases and real-world application scenarios to determine whether tasks accurately reflect practical software engineering challenges.
- Assess the reliability and completeness of tests, evaluation criteria, and expected outcomes associated with assigned tasks.
- Identify codebase integration issues, logic errors, test failures, and other technical inconsistencies that could affect task quality or evaluation accuracy.
- Troubleshoot complex technical scenarios and provide clear, actionable feedback to improve task implementation and evaluation workflows.
- Apply specialized software engineering knowledge to ensure AI training and evaluation tasks meet high standards of technical rigor and practical relevance.
- Document findings clearly and consistently so that identified issues can be addressed efficiently.
Accountabilities
- Evaluate software engineering tasks for technical accuracy, realism, solvability, reproducibility, and alignment with established SWE-Bench standards.
- Review complex codebases and real-world application scenarios to determine whether tasks accurately reflect practical software engineering challenges.
- Assess the reliability and completeness of tests, evaluation criteria, and expected outcomes associated with assigned tasks.
- Identify codebase integration issues, logic errors, test failures, and other technical inconsistencies that could affect task quality or evaluation accuracy.
- Troubleshoot complex technical scenarios and provide clear, actionable feedback to improve task implementation and evaluation workflows.
- Apply specialized software engineering knowledge to ensure AI training and evaluation tasks meet high standards of technical rigor and practical relevance.
- Document findings clearly and consistently so that identified issues can be addressed efficiently.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Requirements
- Demonstrable professional experience and deep technical expertise in software engineering, including complex codebase navigation and real-world application development.
- Strong familiarity with software engineering concepts and the ability to evaluate tasks against practical development standards, including SWE-Bench-related requirements.
- Strong analytical and problem-solving skills, with the ability to rigorously test, troubleshoot, and evaluate complex technical scenarios.
- Strong attention to detail and the ability to identify subtle logic, integration, testing, and reproducibility issues.
- Excellent written communication skills for providing precise, constructive, and actionable technical feedback.
- Ability to work independently and apply specialized expertise to project-based freelance assignments.
- Candidates will be evaluated for a specific area of specialization based on their professional background; expertise across multiple domains is not required.
- Ability to provide a secure computer and reliable high-speed internet connection for freelance project work.
Benefits
- Compensation of $70–$100 per hour, with the exact rate determined based on experience, expertise, and geographic location.
- Final compensation may vary from the published range following evaluation of your qualifications and location.
- Freelance contract structure offering flexibility for project-based work.
- Remote work environment.
- Opportunity to contribute directly to the training and evaluation of advanced AI systems.
- Opportunity to apply specialized software engineering expertise to technically challenging AI evaluation projects.
- Health insurance, paid time off, and other company-sponsored employee benefits are not provided under the freelance contract.
- Contractors are responsible for supplying their own secure computer and high-speed internet connection.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
How Jobgether Works
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills
Location