ML Infrastructure Engineer

ML Infrastructure Engineer — Training, Serving, and Multi-Cloud Inference

San Francisco, CA · On-site · Full-time (Relocation supported)

$200K–$300K base + competitive equity

The company

This is an AI document processing platform that turns PDFs, spreadsheets, and images into LLM-ready structured inputs.

The product sits in the ingestion layer between messy enterprise files and downstream AI systems. That means accuracy and reliability are not aspirational; they are the product.

Since launch, hundreds of companies have signed up. The platform now processes tens of millions of pages every month for teams ranging from startups to Fortune 10 enterprises, including Harvey and Scale.

The company is backed by $108M from a16z, Benchmark, and First Round Capital. It has also posted 8x YoY revenue growth, and the team is 75 people in San Francisco, founded in 2023.

The role

You will own the systems that let models move from experiment to production without infra friction.

This is an infrastructure role, not a research role. You will work directly with ML researchers to keep serving fast, training reliable, and deployment controlled.

If you compare this to senior infra work at big tech or frontier AI labs, the scope is similar: end-to-end ownership, clear tradeoffs, and accountability for production behavior.

The technical problem

Document inputs vary by format, layout, resolution, and quality. Financial statements, medical records, and other enterprise documents do not arrive in clean batches.

The infrastructure has to support training models from 300M to 30B parameters across 1–3 node environments, serve models efficiently on single-to-double node setups, and keep the whole stack observable and debuggable in production.

The hard part is not any single service. The hard part is making training, serving, observability, and cost controls work together without slowing model iteration or hurting accuracy.

Why now

The company already has real usage and real customers. The next bottleneck is not demand; it is whether the ML system can scale without getting slower, more expensive, or harder to operate.

Every improvement in serving, training, or routing shows up in customer latency, page throughput, and unit economics. The infra layer now determines how quickly new models can ship and how well they perform in production.

What you'll own

• Model serving infrastructure: keep inference fast, reliable, and measurable; improve deployment safety, health checks, rollbacks, and monitoring so serving never becomes the bottleneck.

• Training infrastructure: improve training for models ranging from 300M to 30B parameters across 1–3 node environments, with tooling for repeatable experiments, checkpointing, recovery, and efficient compute usage.

• Observability across the ML stack: build logging, metrics, and alerting that make model, pipeline, and infrastructure regressions visible early.

• Research-to-production tooling: build internal data pipelines and workflows that help ML researchers move from experiment to production with less manual glue.

• Multi-cloud inference arbitration: architect the logic that decides where inference runs, balancing accuracy, latency, and cost across multiple cloud providers.

• Operational reliability: reduce blast radius, shorten incident response, and keep customer-facing systems stable as usage grows.

Who this is for

You are likely a strong fit if you have:

• 3+ years of experience in ML infrastructure, platform engineering, or closely related systems work.

• Already owned production ML infrastructure end to end, not just contributed to it.

• Shipped serving or training systems where latency, throughput, reliability, and cost were all real constraints.

• Built observability, pipelines, or internal tooling that changed how researchers or engineers operate ML systems.

• Experience with 1–3 node training environments or similarly constrained serving setups.

• Comfort debugging production issues from logs, metrics, and model behavior.

• The ability to work directly with researchers and turn ambiguous requirements into systems decisions.

• Comfort operating in a codebase where model quality, infra quality, and product quality are tightly coupled.

This role is not for you if

• You want a narrow support queue with well-defined tickets.

• You want to stay purely in research.

• You are not comfortable owning latency, reliability, and spend.

• You need a large platform org around you to make progress.

• You do not want to work on-site in San Francisco.

Compensation and logistics

• Base salary: $200K–$300K

• Equity: competitive equity

• Location: San Francisco, CA

• Workplace: on-site, 5 days per week in the Financial District

• Employment: full-time

• Visa support: H-1B transfers and OPT/F-1 sponsorship available; new H-1B applications are not supported due to filing-window constraints

About Aurora

Aurora helps exceptional engineers find the right role at some of the most ambitious startups worldwide.

We work with teams that value high ownership and strong technical standards.