Seargin, a dynamic multinational tech company, seeks an experienced AI QA Automation Engineer to drive innovation in various industries. The role involves designing and testing RAG architectures, deploying LLM models in Azure, and building advanced automated testing frameworks. The ideal candidate has extensive experience in QA automation, Python development, and Azure Cloud.
Key Highlights
Technical Skills Required
Benefits & Perks
Job Description
Seargin is a dynamic multinational tech company operating in 50 countries.
At Seargin, we drive innovation and create projects that shape the future and greatly enhance the quality of life. You will find our solutions in the space industry, supporting scientists in the development of cancer drugs, and implementing innovative technological solutions for industrial clients worldwide. These are just some of the areas in which we operate.
Currently, for the new Seargin project, we are looking foran experienced AI QA Automation Engineer.
ABOUT THE OFFER
Model: Fully remote from the EU region
Start: ASAP is preferred
Language: English (C1)
Recruitment process: 2 interviews (technical interview + soft interview)
Contract: B2B
FTE: 1 (full-time)
ABOUT THE POSITION
We are seeking a senior-level engineer with deep expertise in Quality Assurance, Python development, Azure Cloud, and Generative AI systems. This includes hands-on experience designing and testing RAG architectures, deploying and evaluating LLM models in Azure AI Studio or Azure OpenAI, and building advanced automated testing frameworks in Python.
The ideal candidate brings extensive experience in QA automation, strong knowledge of LLM evaluation methods, and solid proficiency in Azure DevOps, CI/CD, and DataOps workflows.
We value candidates who have worked with agentic AI frameworks, advanced Azure AI Search features, and modern orchestration tools such as LangChain or Semantic Kernel for building testing pipelines. Experience creating evaluation workflows with Prompt Flow, developing ground-truth datasets, or holding an advanced academic background in AI or Data Science will significantly strengthen your profile.
Additional strengths include hands-on experience fine-tuning Small Language Models (SLMs), familiarity with Azure ML environments, or supplementary coding skills in C# / .NET. Knowledge of red-teaming techniques, experiment tracking tools like MLflow, or certifications such as Azure AI Engineer Associate (AI-102) will be considered valuable bonuses.
QUALIFICATIONS
The most important skills
Generative AI
• 1–2 years of Generative AI experience.
• Practical hands-on experience with RAG architecture (Vector Search + LLM).
Azure Cloud
• 5+ years of Azure experience.
• Experience with Azure AI Studio (or Azure OpenAI Service) — must have deployed/tested models in Azure, not just locally.
Coding (Python)
• 7+ years of Python experience.
• Python Specialist: Able to write complex scripts (Pandas, PyTest), not basic Selenium-type scripting.
QA & Automation
• 8+ years of QA experience.
• Deep understanding of LLM evaluation metrics, including “LLM-as-a-Judge” vs deterministic assertions.
DataOps / CI/CD
• 5+ years of DataOps/CI/CD.
• Azure DevOps pipelines that automatically trigger AI evaluations on pull requests.
Preferred skills
Generative AI
• Experience with Agentic AI (LangGraph, AutoGen) or multi-agent systems.
Azure Cloud
• Experience with Azure AI Search (Cognitive Search), including deep knowledge of indexing, vector profiles, and hybrid search.
Coding (Python)
• Experience with LangChain or Semantic Kernel for building test harnesses.
QA & Automation
• Experience using Prompt Flow to build evaluation pipelines.
DataOps / CI/CD
• Experience creating Golden Datasets (ground truth) for regression testing.
Education
• Master’s degree in Data Science or an AI-related field.
Nice to have:
Generative AI
• Experience fine-tuning Small Language Models (SLMs) like Phi-3 or Llama.
Azure Cloud
• Experience with Azure ML (AML), Workspaces, and Compute Instance management.
Coding (Python)
• Background in C# / .NET to integrate with the client’s backend.
QA & Automation
• Experience with Red Teaming (security, jailbreak, adversarial testing).
DataOps / CI/CD
• Experience with MLflow for experiment tracking.
Education
• Certification: Azure AI Engineer Associate (AI-102).
Discover the Power of IT Excellence. Apply!
To learn more about Seargin, please visit our web page: www.seargin.com