AI Summary
Design and improve LLM training pipelines, build and operate scalable data labeling and evaluation workflows, and partner with distributed teams to drive measurable model performance improvement.
Key Highlights
Design and improve LLM training pipelines
Build and operate scalable data labeling and evaluation workflows
Partner with distributed teams to drive measurable model performance improvement
Key Responsibilities
Design and improve LLM training pipelines for supervised fine-tuning and RLHF
Build and maintain annotation tooling, task routing, and QA evaluation checks
Define labeling taxonomies for NLP tasks (e.g., named entity recognition, classification)
Support computer vision annotation workflows (bounding boxes, polygons, segmentation QA)
Run prompt evaluation and rubric-based model evaluation to identify failure modes
Implement content safety labeling policies, audits, and escalation paths
Create annotation guidelines, sampling plans, and ensure annotation guidelines compliance
Analyze training data quality metrics, disagreement patterns, and error distributions
Document datasets and evaluation artifacts for traceability and reproducibility
Technical Skills Required
Job Description
About Rex.zone
Rex.zone connects India-aligned STEM talent with Remote, Full-Time engineering programs supporting AI/ML training workflows. You will help improve training data quality and evaluation rigor across LLM training pipelines, RLHF, and model evaluation.
About The Role
You will build and operate scalable data labeling and evaluation workflows, implement annotation tooling, run prompt evaluation, and partner with distributed teams to drive measurable model performance improvement.
Key Responsibilities
- Design and improve LLM training pipelines for supervised fine-tuning and RLHF
- Build and maintain annotation tooling, task routing, and QA evaluation checks
- Define labeling taxonomies for NLP tasks (e.g., named entity recognition, classification)
- Support computer vision annotation workflows (bounding boxes, polygons, segmentation QA)
- Run prompt evaluation and rubric-based model evaluation to identify failure modes
- Implement content safety labeling policies, audits, and escalation paths
- Create annotation guidelines, sampling plans, and ensure annotation guidelines compliance
- Analyze training data quality metrics, disagreement patterns, and error distributions
- Document datasets and evaluation artifacts for traceability and reproducibility
Interested in remote work opportunities in Development & Programming? Discover Development & Programming Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
- Mid-Senior experience in engineering, data operations engineering, or applied ML workflows
- Hands-on experience with data labeling systems, QA evaluation, or model evaluation pipelines
- Familiarity with RLHF concepts and large language model evaluation methods
- Working knowledge of NLP and/or computer vision annotation
- Strong written communication for remote collaboration
Apply via Rex.zone and highlight experience in training data quality, annotation guidelines compliance, QA evaluation, RLHF, prompt evaluation, NLP/NER, computer vision annotation, and content safety labeling.
Similar Jobs
Explore other opportunities that match your interests
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Mid-Senior level
Guidehouse
United State
AI Engineer for Autonomous Trading Research Systems
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
People In AI
United State
Solution Director, Analytics & Al/ML
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
Rackspace Technology
United State