Senior QA Engineer - AI

QAT Global, a 30+ years IT consulting firm based in the US, is seeking a highly skilled Senior QA Engineer – AI with strong English communication skills. This position will work directly with US-based teams and collaborate with software developers, QA engineers, and AI-focused stakeholders across our Brazil, Costa Rica, and US operations.

This opportunity is CLT-only in Brazil and can be fully remote.

Key Responsibilities

• Design, execute, and maintain functional, regression, integration, and API test strategies
• Develop and maintain automated test suites for web applications, APIs, and backend services
• Participate in test planning, test case design, defect tracking, and release validation activities
• Support CI/CD quality processes and automated test execution within Azure DevOps pipelines
• Perform API testing using tools such as Postman and validate backend integrations and data flows
• Execute SQL-based validations and support data integrity testing activities
• Collaborate closely with software engineers, product teams, and stakeholders to improve software quality and delivery reliability
• Analyze complex defects, identify root causes, and support resolution efforts across distributed teams
• Contribute to the continuous improvement of QA processes, automation coverage, and testing standards

AI / LLM Quality Responsibilities

• Support evaluation and validation activities for AI-generated outputs and LLM-based workflows
• Participate in prompt regression testing and help identify quality degradation scenarios
• Assist with evaluation datasets, scoring criteria, and regression validation for AI systems
• Analyze AI-generated responses, edge cases, hallucinations, and inconsistent outputs

Interested in remote work opportunities in Development & Programming? Discover Development & Programming Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

• Review agent traces, tool-call sequences, and generated outputs to support troubleshooting and quality analysis
• Collaborate with prompt engineers and developers to improve AI reliability and output consistency
• Validate AI-generated C# / .NET code and identify logic or architectural inconsistencies

Required Qualifications

• Strong experience in Software Quality Assurance and test automation
• Experience with automated testing frameworks such as Playwright, Selenium, Cypress, or similar
• Experience with API testing and REST services validation
• Experience with SQL and backend data validation
• Experience integrating automated tests into CI/CD pipelines
• Experience with Azure DevOps or similar DevOps platforms
• Strong analytical, troubleshooting, and debugging skills
• Ability to work independently in distributed international teams
• Strong English communication skills

Technical Qualifications

• Ability to read and validate C# / .NET Core code
• Familiarity with Entity Framework and backend application validation
• Understanding of regression testing methodologies and quality assurance best practices

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

• Familiarity with non-deterministic system behavior and AI-generated output validation is a plus
• Exposure to Generative AI, LLMs, AI agents, or prompt engineering concepts is a plus

Preferred Qualifications

• Experience with AI/LLM evaluation workflows or AI-assisted software development environments
• Familiarity with tools such as LangSmith, Langfuse, Promptfoo, OpenAI Evals, or similar platforms
• Experience with AI agents, prompt engineering, or generative AI workflows
• Experience in consulting environments supporting enterprise international clients
• Experience working directly with US-based stakeholders or distributed global teams

What Makes This Role Different

• This role combines traditional Senior QA Engineering responsibilities with emerging AI quality and evaluation practices
• The position includes both structured software testing and AI-generated output validation
• The engineer will collaborate not only with developers and QA teams, but also with AI-focused engineering and prompt engineering initiatives
• The environment emphasizes automation, continuous quality validation, and modern software delivery practices

About QAT Global

QAT Global is an international IT consulting firm with more than 30 years of experience delivering technology solutions for enterprise clients. Headquartered in the US, with operations in Brazil and Costa Rica, we specialize in building high-performance distributed teams for global software initiatives.