Model drift and monitoring: interactive demo

Drift is a change in the data a model meets after deployment: a new patient mix, a change in practice, a new scanner or laboratory method, a new coding rule. The model does not change, and its performance does. In the demo the true AUC is flat at 0.85 for six months and then declines at the rate you set, which is the white dashed line. Nobody sees that line in practice.

What is seen is the blue line: the AUC measured each month on the cases that have a confirmed outcome. One in five of those cases is positive, so 100 cases a month means about 20 positives, and the measured value swings from month to month by several points even when nothing has changed. The alarm line sits below the baseline by the margin you choose, and the rule requires two months in a row below it.

The margin sets the trade-off. A narrow margin reacts sooner and also fires on noise. A wide margin is quiet and late. More labelled cases reduce the noise, which is what allows a narrower margin. A monitoring plan should therefore state the metric, the source and number of labelled cases, the rule and the expected delay, and should also track inputs and case mix, which shift before outcomes are known. See clinical model validation.

Picture a hospital that deploys a risk model after a careful validation, with an AUC of 0.85. The team agrees to keep an eye on it: each month the cases with a confirmed outcome are collected, the AUC is computed, and a review is triggered if it drops clearly below the validated figure.

The demo’s default settings are that plan: 100 labelled cases a month, an alarm margin of 0.05 below baseline, and a model that starts to lose 0.010 of AUC a month after month 6. The white dashed line falls steadily to a true AUC of 0.67 at month 24. The blue points, which are all the team can see, scatter around it.

Alarm fires reads month 12. Drift undetected for reads 6 months, and True AUC at the alarm reads 0.79. For half a year the model was worse than validated and the monitoring said nothing, because with about 20 positive cases a month the measured AUC moves by several points on its own and the rule has to wait for a clear, repeated drop.

Reduce the labelled cases to 20 a month, about 4 positives. The alarm now fires in month 4, before any drift, and the readout says false alarm. Return to 100 cases and narrow the margin to 0.03: the alarm again fires in month 4 by chance. Widen it to 0.10 and it fires in month 17, 11 months after the drift began, at a true AUC of 0.74. With 1,000 cases a month and the default margin it still fires in month 12, because the true AUC has to cross the alarm line first.

The correct practice is to design monitoring as carefully as validation. Decide the metric and where the labels come from. Work out how many cases arrive each month and how noisy the monthly figure will be, then choose a margin and rule with a known false-alarm rate and delay. Pool months if the numbers are small. Track input distributions and case mix too, since they change before outcomes are available, and define in advance what happens when the alarm fires.

• Which performance metric is tracked after deployment, and how often?
• Where do the confirmed outcomes come from, and how many arrive each month?
• How much does the monthly figure vary by chance at that sample size?
• What rule triggers a review, and what are its false-alarm rate and expected delay?
• Are input distributions, case mix and alert rates tracked as earlier signals?
• Who is responsible when the alarm fires, and what are they expected to do?

Related: our clinical model validation service, the wiki page on medical AI validation, and the demos on sample size for validation and calibration and decision thresholds.

How do I monitor a clinical AI model after deployment?

Track a performance metric on cases with confirmed outcomes at a regular interval, compare it with the validated baseline, and define a rule that triggers a review. Also track inputs, case mix and alert rates, which need no outcome labels. In this demo a monthly AUC on 100 cases with a 0.05 margin raises the alarm 6 months after drift begins.

How long does it take to detect model drift?

It depends on how fast the model degrades, how many labelled cases arrive and how wide the alarm margin is. In the demo, with the AUC falling by 0.01 a month, the alarm fires after 6 months at a margin of 0.05 and after 11 months at a margin of 0.10. A slower drift takes longer to detect.

Why does my model monitoring raise false alarms?

Because a metric measured on a small monthly sample varies by chance. In the demo, with 20 labelled cases a month, about 4 of them positive, the alarm fires in month 4 although the model has not changed. More cases per period, pooling periods or a wider margin reduce false alarms, at the cost of a longer delay.

What causes a medical AI model to drift?

Changes in the patients or the data: a different case mix, new clinical practice, a new scanner or laboratory method, a change in coding or documentation, or a change in how the model’s own output is used. The model stays the same and the world it was fitted to does not.

How many labelled cases a month does monitoring need?

Enough for the monthly figure to be stable relative to the drop you want to detect. In the demo 100 cases a month, about 20 of them positive, support a margin of 0.05 without a false alarm, while 20 cases a month do not. If volumes are low, pool several months or monitor quantities that need no labels.