AI Summary
We are seeking a skilled Software Engineer to craft realistic developer prompts, evaluate AI outputs, and contribute to structured evaluation datasets. The ideal candidate will have strong experience in software engineering, technical research, or educational content development. Proficiency in one or more languages such as Python, JavaScript, Java, or C++ is required.
Key Highlights
Craft realistic developer prompts
Evaluate AI outputs
Contribute to structured evaluation datasets
Key Responsibilities
Craft realistic, chat-style developer prompts covering code review, debugging, error diagnosis, configuration, and related scenarios.
Source and adapt real pull requests and engineering discussions to create authentic evaluation tasks.
Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality.
Evaluate AI outputs based on logic, clarity, and technical judgment rather than executable correctness.
Apply software engineering expertise to ensure prompts and responses reflect real-world developer workflows.
Contribute to structured evaluation datasets used to assess AI reasoning and communication quality.
Collaborate asynchronously with research teams to refine task quality and consistency.
Technical Skills Required
Benefits & Perks
$70-$120/hour
Remote work
10-40 hours/week
Job Description
Position: Software Engineer – Code QA
Type: Hourly contract
Compensation: $70–$120/hour
Location: Remote
Commitment: 10–40 hours/week
Role Responsibilities
- Craft realistic, chat-style developer prompts covering code review, debugging, error diagnosis, configuration, and related scenarios.
- Source and adapt real pull requests and engineering discussions to create authentic evaluation tasks.
- Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality.
- Evaluate AI outputs based on logic, clarity, and technical judgment rather than executable correctness.
- Apply software engineering expertise to ensure prompts and responses reflect real-world developer workflows.
- Contribute to structured evaluation datasets used to assess AI reasoning and communication quality.
- Collaborate asynchronously with research teams to refine task quality and consistency.
Interested in remote work opportunities in QA & Testing? Discover QA & Testing Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
- Strong experience in software engineering, technical research, or educational content development.
- Bachelor’s degree in Software Engineering, Computer Science, or a related field.
- Strong proficiency in one or more languages such as Python, JavaScript, Java, or C++.
- Hands-on experience with debugging, testing, and validating code.
- Strong technical writing skills with high attention to detail.
- Ability to work independently in a fully remote, project-based environment.
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
- Upload resume
- Interview (15 min)
- Submit form
Similar Jobs
Explore other opportunities that match your interests
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Mid-Senior level
Jobgether
Canada
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Mid-Senior level
Jobgether
Canada
Senior QA Engineer - Trading Backend Services
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
ink fndn
Canada