Job Description
AI Platform Engineer - LLM Evaluation & Serving (NYC, Onsite)
Applied AI meets world-class infra | Profitable | In-office NYC
Realm are supporting a stealthy, profitable AI company in NYC hire AI Platform Engineers to build the backbone for how LLM agents are evaluated, served, and improved in production. It’s a brilliant mix of ML systems, distributed infra, and metrics-driven engineering.
The Role
You’ll be designing the infrastructure that helps AI agents learn and iterate safely. Expect to work closely with the founding team and engineers from top-tier AI labs and infra startups.
What You’ll Work On
- Build evaluation pipelines (offline + online) and telemetry systems
- Scale and operate serving infrastructure (Ray, vLLM, Triton, KServe)
- Help design frameworks to evaluate and monitor agent performance in real time
Ideal Profile
- 2–6+ years in ML platform, infra, or backend roles
- Strong Python skills; familiarity with Ray, vLLM, or KServe a plus
- Experience in metrics, observability, or distributed systems
- Excited to work onsite in NYC (5d/w) - relocation supported
What’s on Offer
- $150k–$300k base + strong equity
- Chance to shape the AI evaluation layer from the ground up
- Tight-knit, no-bureaucracy culture focused on learning and delivery
If you’re an engineer who enjoys the intersection of AI systems and infra, this is a rare opportunity. Realm are managing this search confidentially and can share more on a quick call.