Software Engineering Expert, AI Trainer

Fully remote, project-based work checking how well AI coding agents handle real software engineering. You write tasks based on your own experience and grade where the agent gets it wrong. $40 to $125 USD an hour, at least 10 hours a week, as an independent contractor. For working software engineers in the US, UK, Ireland, Canada, or Australia.

AI coding agents produce working code quickly, and it is not always code a good engineer would ship. This role puts your experience to work finding out where. You turn engineering problems you know into realistic tasks, run them through AI agents, and judge the answers the way you would review a colleague's pull request. You set your own schedule, and the work rewards careful writing as much as expertise.

What you will do

• Build realistic tasks out of problems from your own software engineering work, along with the material needed to solve them
• Give those tasks to frontier AI models and judge the results against the standard of your profession
• Weigh paired model answers against each other and explain which one holds up
• Write grading rubrics for your tasks and set out your reasoning in writing
• Review and improve tasks written by other engineers

Requirements

• Based in the United States, United Kingdom, Ireland, Canada, or Australia
• Bachelor's degree or higher, completed or in progress
• Available at least 10 hours per week
• PayPal account for payment
• Willing to work without LLM assistance where the project does not permit it
• Use one email address to sign up, take the assessment, and get paid

Ideal qualifications

• Comfortable with git, testing, and debugging in your main language
• Full professional or native-level written English

Nice to have

• Professional experience in the field, 2+ years especially (a plus, not required)
• Experience across more than one language or stack
• Have used Claude or ChatGPT in your professional work

Pay and contract

• $40 to $125 USD an hour. The rate varies with your expertise, education, and experience. Minimum 10 hours a week with no weekly maximum; full-time hours are available.
• Independent contractor, fully remote. Payment runs through PayPal, so you need a PayPal account. As an independent contractor, you handle your own taxes.
• Getting on: a skills assessment that takes most people four to six hours. It is required to join the project, and you get one attempt, so take your time and do your best work. Work produced with AI tools is rejected.
• After you pass: onboarding and a short training project, then you propose a task from your own practice, build it, and run it against frontier models. Each stage is reviewed before you move on, and you are paid for the work you complete.

Where you can work from

This job is 100% remote, for applicants who live in the United States.