Gen AI Inferencing Software Engineer

📍 Location: Addison, TX / Charlotte, NC (Onsite)

💼 Employment: W2 Only

🛂 Visa: USC / H4EAD (please do not apply except these)

📌 Relocation: Same state only

We are looking for a highly skilled Software Engineer – Gen AI Inferencing to design, build, and operate reusable toolkits supporting GenAI RAG capabilities. This role focuses on developing scalable inferencing frameworks, AI platforms, and automation systems that power enterprise-grade AI/ML solutions.

If you have strong experience in Python-based large-scale systems and hands-on expertise in GenAI lifecycle, RAG pipelines, and inference frameworks — we want to hear from you!

Key Responsibilities:

• Design, develop, and maintain reusable GenAI inferencing toolkits and RAG frameworks

Searching for Development & Programming roles that provide visa sponsorship? Connect with international employers through Development & Programming Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

• Build scalable AI/ML solutions meeting functional, non-functional, and compliance standards
• Deploy and optimize models using vLLM or Triton Inference Server in containerized environments
• Automate CI/CD pipelines and release workflows
• Develop automated testing frameworks (integration, regression, performance)
• Perform proof-of-concepts (POCs) and risk mitigation spikes
• Collaborate with product teams, data scientists, and stakeholders
• Mentor engineers and promote DevOps and automation best practices

Required Qualifications:

• 5+ years of OOP development experience (Python / Scala / Java)
• Strong hands-on experience with GenAI / AI-ML lifecycle management
• Experience building RAG pipelines (chunking, embeddings, retrieval, reranking, summarization)
• Model deployment experience with vLLM / Triton Inference Server
• Experience with containerization and CI/CD automation
• Experience building API-based applications (FastAPI, JWT, API Gateway)
• Hands-on DevOps experience (Git, Jenkins, SonarQube, pytest, Artifactory, Ansible)
• Experience working in large collaborative multi-repo environments

Desired Qualifications:

• Experience building GenAI inferencing platforms using open-source toolsets
• AI Gateway, observability, and policy store implementation
• Strong research mindset with ability to prototype innovative solutions
• Experience driving quality, automation, and experimentation culture

Key Skills:

Application Development | GenAI | RAG | Python | MLOps | DevOps | Architecture | Automation | CI/CD | Containerization | MongoDB | Redis | React/Angular | API Development | Test Engineering | Collaboration