Senior QA Automation Engineer responsible for validating LLMs, RAG pipelines, AI agents and predictive ML systems. Build automated frameworks ensuring AI products are reliable, scalable and production-ready. Requires 5+ years QA automation experience with strong Python skills and AI/ML testing expertise.
Key Highlights
Key Responsibilities
Technical Skills Required
Benefits & Perks
Nice to Have
Job Description
Senior QA Automation Engineer (AI & LLM Systems)
📍 Fully Remote – Work from Anywhere
We're partnering with an innovative AI company and are looking for a Senior QA Automation Engineer to help shape the quality of production AI systems.
This is not traditional software testing. You'll be validating LLMs, RAG pipelines, AI agents and predictive ML systems, building automated frameworks that ensure AI products are reliable, scalable and production-ready.
What you'll be working on
- Build automation frameworks for AI, ML and LLM applications
- Validate RAG pipelines and AI-generated outputs
- Develop UI, API and backend automation using Python
- Integrate automated testing into CI/CD pipelines
- Design regression testing for AI models and prompt changes
- Test enterprise integrations with large-scale platforms
- Work closely with AI Engineers, Data Scientists and Product teams to improve product quality throughout the SDLC
Interested in remote work opportunities in QA & Testing? Discover QA & Testing Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
What we're looking for
- 5+ years in QA Automation
- Strong Python programming skills
- Experience with PyTest, Playwright and/or Selenium
- API testing using Requests or similar frameworks
- Experience testing AI, Machine Learning or data-driven applications
- Knowledge of LLM evaluation, RAG or Generative AI is highly desirable
- Experience with SQL, CI/CD and GitLab pipelines
- Comfortable working in fast-paced product engineering environments
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
Why apply?
✅ Fully remote – work from anywhere in the world
✅ Build cutting-edge AI products using the latest LLM technologies
✅ Work alongside world-class AI engineers and researchers
✅ Long-term contract with exciting greenfield projects
✅ High-impact role working on enterprise-scale AI platforms
If you've been looking for a chance to work on the future of AI rather than traditional QA, we'd love to hear from you.
Similar Jobs
Explore other opportunities that match your interests