Machine Learning Researcher - AI Research Plan Evaluation

Machine Learning Researchers

Hourly Contract | Remote | $140 per hour

1. About the Role

Mercor is partnering with a leading AI research lab on Project Vesuvius — an initiative aimed at evaluating and enhancing the ability of large language models (LLMs) to generate structured, high-quality research plans for open-ended machine learning problems.

We are seeking experienced Machine Learning Researchers and PhDs to assess AI-generated research plans, provide structured feedback, and help improve how advanced models function as brainstorming and planning partners for real-world ML research.

This is a remote, high-impact opportunity for researchers passionate about advancing AI reasoning, experimentation, and planning capabilities.

2. Key Responsibilities

• Evaluate and compare AI-generated ML research plans for clarity, feasibility, and technical validity.
• Design and compile machine learning tasks inspired by real-world research problems and competitions.
• Draft detailed, executable natural-language research workflows for model training, experimentation, and validation.
• Implement and test selected plans using Python within a Docker-based environment.
• Assess performance using structured rubrics and provide quantitative and qualitative feedback.

3. Ideal Qualifications

• 5+ years of experience in applied machine learning, or a PhD in ML, AI, or a closely related field.
• Deep understanding of machine learning research methods, experimental design, and evaluation metrics.
• Strong technical writing and analytical reasoning skills.
• Experience with benchmarking, reproducibility, or research validation workflows preferred.
• Ability to deliver high-quality structured assessments independently and consistently.

4. Engagement Details

• Type: Independent Contractor
• Location: Fully Remote and Asynchronous
• Commitment: Flexible (up to 80 hours per week)
• Project Name: Project Vesuvius
• Start Date: Immediate onboarding for selected candidates

This role is ideal for researchers and engineers who value autonomy, rigor, and contribution to cutting-edge AI research.

5. Compensation & Contract Terms

• Rate: Up to $140/hour
• Classification: Independent Contractor
• Payments: Weekly via Stripe Connect
• Structure: Remote, milestone-based evaluation with flexible scheduling

6. Application Process

1. Submit your resume or CV highlighting relevant ML research or engineering experience.
2. Complete a short AI-led interview and brief questionnaire about your experience with reproducibility and benchmarking.
3. Selected candidates will receive onboarding materials and project access within days.

⚡ PS: Mercor reviews applications daily. Please complete your interview and onboarding steps to be considered for this opportunity. ⚡