Software Engineer

AI Platform Engineer - LLM Evaluation & Serving (NYC, Onsite)

Applied AI meets world-class infra | Profitable | In-office NYC

Realm are supporting a stealthy, profitable AI company in NYC hire AI Platform Engineers to build the backbone for how LLM agents are evaluated, served, and improved in production. It’s a brilliant mix of ML systems, distributed infra, and metrics-driven engineering.

The Role

You’ll be designing the infrastructure that helps AI agents learn and iterate safely. Expect to work closely with the founding team and engineers from top-tier AI labs and infra startups.

What You’ll Work On

• Build evaluation pipelines (offline + online) and telemetry systems
• Scale and operate serving infrastructure (Ray, vLLM, Triton, KServe)
• Help design frameworks to evaluate and monitor agent performance in real time

Ideal Profile

• 2–6+ years in ML platform, infra, or backend roles
• Strong Python skills; familiarity with Ray, vLLM, or KServe a plus
• Experience in metrics, observability, or distributed systems
• Excited to work onsite in NYC (5d/w) - relocation supported

What’s on Offer

• $150k–$300k base + strong equity
• Chance to shape the AI evaluation layer from the ground up
• Tight-knit, no-bureaucracy culture focused on learning and delivery

If you’re an engineer who enjoys the intersection of AI systems and infra, this is a rare opportunity. Realm are managing this search confidentially and can share more on a quick call.