What AI model drift is, and why it is invisible without measurement

AI model drift is the quiet change in how a deployed model behaves over time, even when the name on the endpoint never changes. It is not the model getting smarter or dumber; it is the model answering the same question differently than it did last month.

When people say a model “feels different” than it did a few weeks ago, they are usually describing drift. The endpoint still answers to the same name, the documentation still lists the same version, but the disposition underneath has shifted: it now refuses something it used to answer, hedges where it used to commit, or takes a firmer line on a contested question. None of that shows up in a capability score, and none of it is announced.

Two things people call “drift”

The first is serving change: the provider routes the alias you call to a different underlying build, for cost, capacity, or a safety patch. The name is stable; the weights or the system scaffolding are not. The second is ordinary variation: the same build, sampled at a non-zero temperature, returns a distribution of answers rather than one fixed answer. Both are real, and telling them apart is the whole problem. A single prompt run once cannot distinguish “the model changed” from “I rolled the dice once.”

Why drift is invisible by default

Software has changelogs. Model behavior does not. There is no version pin for disposition, no diff you can read, no notification when a stance moves. If you were not measuring the old behavior on fixed inputs, the old behavior is simply gone; a deprecated build can never be re-measured. That is what makes drift dangerous for anything reproducible: a prompt you validated, a compliance argument you filed, an evaluation you trust. It can all quietly stop describing the system you are actually calling.

How you detect it

Drift is a claim about change, and change needs at least two measurements of the same thing. The method is unglamorous: fix the inputs, fix the protocol, repeat on a schedule, and compare. A greedy, un-cacheable probe run every day catches a serving swap the moment the fixed answer moves. Repeated sampling of contested questions, coded into a distribution with its uncertainty, catches a shift in disposition once it clears the noise. The unit of a finding is a delta with an error bar, not an adjective.

This is exactly the record Modelometer keeps: daily serving-integrity canaries and weekly stance measurements on frontier models, every run hash-chained and externally timestamped, so “the model changed on this date” is a provable statement rather than a vibe.