Monitoring in MLOps, explained in 5 lines

Module 14 · MLOps / LLMOps · 22 concepts

In depth / 15 lines

Monitoring, in more detail

Monitoring watches a running system continuously rather than after a complaint.

The core signals are latency, error rate, throughput and cost.

Cost belongs there for model systems, since a bad change can multiply the bill overnight.

Machine learning adds a signal ordinary software does not have.

The code can be correct while the model quietly becomes wrong.

Data drift means live inputs no longer resemble the training data.

Prediction drift means the distribution of outputs has shifted.

Either can appear with no error and no alert firing.

Compare live input distributions against the training set on a schedule.

Watch the rate of each predicted class, since a sudden shift is usually upstream.

Where labels arrive later, track accuracy over time as the ground truth catches up.

For language systems, monitor refusal rates, output length and cost per request.

Alert on user-visible symptoms rather than on every metric wobble.

An alert nobody acts on trains everyone to ignore alerts.

Sample real traffic into an evaluation set regularly, since production is the honest test.

Want the full technical article? Wikipedia ↗

Deep dive

What is underneath Monitoring

The model does not break, the world moves

A deployed model's weights are fixed. What changes is the data arriving at it, and the failure is silent because nothing throws an exception.

\text{covariate shift} : p_{\mathrm{new}}(\mathbf{x}) \neq p_{\mathrm{train}}(\mathbf{x}), \quad p(y \mid \mathbf{x}) \text{ unchanged}

\text{concept drift} : p(y \mid \mathbf{x}) \text{ changed}

The first means the inputs look different. The second means the right answer for the same input is now different, and only the second is unfixable by retraining on old labels.

Measuring input drift without labels

\mathrm{PSI} = \sum_{i=1}^{B} (a_i - e_i) \ln\frac{a_i}{e_i}

$a_i$ : the share of current traffic in bin $i$ .
$e_i$ : the share of the training data in that bin.

Population stability index compares two histograms of the same feature, and it needs no ground truth, which is why it runs in production.

\mathrm{PSI} < 0.1 : \text{stable}, \quad 0.1 \text{ to } 0.25 : \text{watch}, \quad > 0.25 : \text{act}

Those thresholds are conventions from credit scoring, and they are a starting point rather than a law.

D_{\mathrm{KL}}(P \Vert Q) = \sum_i p_i \log \frac{p_i}{q_i}

KL divergence measures the same idea and is unbounded when a bin empties, which is why PSI's symmetric form is preferred for alerting.

What to watch, in order of usefulness

Prediction distribution, since a shift there is the cheapest signal that something changed.
Input feature distributions, one PSI per feature, which localises the change.
Latency and error rates, which catch the infrastructure half.
Ground truth metrics, whenever labels arrive, however late.
Business outcomes, which are the only measure that decides whether any of it matters.

The gap between a prediction and its outcome is the central difficulty. Fraud is labelled in days, churn in months, and a model can be wrong for a whole quarter before the metric says so.

The model-specific signals

\bar{c} = \frac{1}{n}\sum_{i=1}^{n} \max_j \hat{p}_{ij}

Average confidence falling is an early warning that the inputs have moved away from what the model knows.

For text, the share of unknown tokens and the distance of the input embedding from the training distribution both work as drift detectors with no labels.

For a generation system, the measurable proxies are refusal rate, output length, tool call failure rate and the share of responses a judge marks unsupported.

Alerting that survives contact with a rota

Alert on the derivative, not the level.

A metric that is slightly bad every day trains people to ignore it. A metric that moved sharply today is worth waking someone for.

Every alert should say what action it implies. Drift detected with no attached decision is a notification, and a rota full of notifications is an unmonitored system.

A good retrain trigger combines a PSI threshold, a metric drop and the time since the last retrain.

Writing that rule down in advance is what turns monitoring into a loop rather than a dashboard nobody opens.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1

Live input data no longer resembles your training data. What is this called?

MLOps / LLMOps

22 concepts in this module, then a graded test with every answer explained.

Study this module →