LLMOps engineer
The one who keeps in production what never answers the same way twice
An LLMOps engineer keeps in production systems whose output changes on every call, which makes half the usual instruments useless, like a mechanic whose engine never twice makes the same noise.
- Floor 2The Harness
- P3Writes the system
- Emerging titleIt appeared recently. What it covers is not yet stable from one employer to the next.
What they answer for
The deliverableA platform where other people deploy without breaking things: instruction versions, quotas, traces, attributed cost. If they leave, nobody knows what is running, or what it costs.
The profile
Five activities, scored 0 to 3- ResearchProducing knowledge that does not yet exist.
- BuildShipping a system that runs, deploys and breaks.
- OperateKeeping it in production: cost, incidents, drift, on-call.
- VerifyMeasuring, testing, attacking. Producing a verdict that holds.
- LeadDeciding, persuading, driving adoption, answering to others.
What the work is
What the output changes
All classical operations rest on one assumption: equal input, equal output. It falls here. You can no longer replay an incident identically, nor compare two versions by equality, nor test by assertion. What replaces them is a family of new instruments, and that is what split this trade off from MLOps.
Cost becomes a metric
Cost stops being an infrastructure line and becomes a product metric. Every call is paid for, the price depends on what went into the window, and a badly tuned feature can cost more than the team that wrote it. Attributing cost per feature is a job in its own right, and it did not exist before.
The trap
The trap is that the title is not settled: depending on the organisation, the same work is called MLOps, AI platform engineer, or simply SRE. What identifies the post is not the name but two phrases in the advert: instruction versions, and cost per call. Where those are missing, non-deterministic output has not been met yet.
A week in the role
- Tooling: instruction versions, switches, rollbacks without redeployment.
- Reading traces, when an answer has drifted with no alert going off.
- Cost tracking, per feature and not per server.
- Quotas and retries, because a provider slowing down is an incident like any other.
Ways in
- From operations or site reliability, accepting that determinism goes away.
- From classical MLOps, which is the shortest route: half the instruments carry over.
- What the role does not require: training a model, or even knowing how one is trained.
Reading an advert
2 signs- The advert describes classical MLOps: model registry, drift, retraining.
- That is an MLOps post. Useful, but the problems of non-deterministic output are not in it.
- No mention of cost per call.
- That is what takes up most of the time in this trade. An advert ignoring it describes a service that has not met its bill yet.