Causal Inference in Healthcare and Medical AI

Causal inference in healthcare is the set of methods used to estimate the effect of a treatment, exposure or care decision on patient outcomes, particularly from observational sources such as electronic health records, registries and claims data where treatment was not assigned at random. It matters for medical AI because a model that predicts an outcome accurately does not, for that reason, tell a clinician what will happen if they intervene. The main threats are confounding by indication and design errors such as immortal time bias; the main tools are randomized trials, target trial emulation and propensity-based adjustment.

This page is an educational overview of methods. It is not medical advice and does not replace study-specific statistical or clinical guidance.

Why is causal inference harder in healthcare than in an A/B test?

In an online experiment, treatment is assigned by a random number generator. In routine care, treatment is assigned by clinicians and patients for reasons: severity, comorbidity, frailty, preference, access. Those reasons usually also affect the outcome. Health data adds further difficulties: treatments start, stop and change over time; follow-up is incomplete; outcomes are recorded only when a patient is seen; and many relevant factors, such as a clinician's overall impression of a patient, are not recorded in structured form. Randomized controlled trials remain the reference standard, but they are not available for every question, population or comparison, which is why careful observational analysis is needed. For the general framework, see causal inference in business and data science and the ladder of causality.

What is confounding by indication?

Confounding by indication occurs when the reason a treatment is given (its indication) is also a cause of the outcome. Patients who receive a treatment tend to be those who need it most, so in raw data treated patients often have worse outcomes than untreated ones even when the treatment is beneficial.

Approaches that reduce it include comparing the treatment with an active comparator (another treatment used for the same indication) instead of with no treatment, restricting to new users so that covariates are measured before treatment starts, and adjusting for measured markers of severity. None of these addresses severity that was not recorded, which is why residual confounding is the standard caveat on observational treatment comparisons.

What is immortal time bias?

Immortal time is a span of follow-up during which, because of how the exposure group is defined, the outcome cannot occur. A typical case: patients are classified as "treated" if they fill a prescription at any time after hospital discharge, and follow-up is counted from discharge. Every treated patient must have survived from discharge to the prescription date, or they would not be in the treated group. If that interval is counted as treated time, or dropped, the treated group is credited with survival that was guaranteed by the definition, and the treatment appears protective.

The remedy is a matter of design: start follow-up at the moment when eligibility is met and treatment strategy is assigned, classify exposure using only information available at that moment, or model treatment as a time-varying exposure so that time before treatment is counted as untreated.

What is target trial emulation?

Target trial emulation is a framework developed by Miguel Hernán and James Robins. The analyst first writes the protocol of the hypothetical randomized trial (the target trial) that would answer the question, and then emulates each component with observational data. The protocol components are:

• Eligibility criteria: who would be enrolled, defined using information available at baseline.
• Treatment strategies: the specific strategies being compared, including what happens after initiation.
• Assignment: random in the target trial; emulated by adjusting for baseline confounders.
• Time zero: the start of follow-up, when eligibility is met and a strategy is assigned.
• Outcomes and follow-up: what is measured and for how long.
• Causal contrast: the observational analogue of the intention-to-treat or per-protocol effect.
• Analysis plan: how confounding, censoring and deviation from the strategy are handled.

The discipline of the method is that eligibility, assignment and time zero must coincide, as they do in a real trial. That alignment removes immortal time bias and several related selection biases by construction. It does not remove unmeasured confounding: emulation fixes the design and leaves the adjustment problem to be solved separately.

How do propensity score methods work?

The propensity score, introduced by Rosenbaum and Rubin, is the probability that a patient receives the treatment given their measured baseline covariates. Among patients with the same propensity score, the distribution of those measured covariates is expected to be similar in treated and untreated groups. The score is used in four main ways: matching treated to untreated patients with similar scores, stratifying on the score, inverse probability of treatment weighting (weighting each patient by the inverse of the probability of the treatment they received), and including the score as a covariate.

The methods depend on three conditions: exchangeability (no unmeasured confounding after conditioning on the covariates), positivity (every type of patient has some chance of receiving each treatment) and consistency (the treatment is well defined). Good practice is to check covariate balance after matching or weighting, commonly with standardized mean differences, to inspect the overlap of the score distributions, and to report sensitivity analyses for unmeasured confounding. When treatment and confounders both change over time and earlier treatment affects later confounders, standard adjustment is biased, and the g-methods associated with Robins, such as marginal structural models fitted by inverse probability weighting and the parametric g-formula, are used.

Why is a predictive model's feature importance not a treatment effect?

A clinical prediction model is trained to predict an outcome from whatever inputs are informative. Its coefficients, SHAP values or other importance measures describe how the prediction depends on each input in the training data. They sit on the first rung of the ladder of causality. There are two main reasons they cannot be read as effects:

• The model was not adjusted for the right things. An effect estimate requires a deliberate choice of confounders for one exposure. A prediction model includes any useful covariate, including consequences of the exposure. Interpreting every coefficient in one multivariable model as an effect is the error that Westreich and Greenland named the Table 2 fallacy.
• The data encodes existing care. A widely cited example, discussed by Caruana and colleagues, is a pneumonia risk model that learned that patients with asthma had lower mortality risk. Those patients were routinely given more intensive care, which lowered their observed risk. Used naively for triage, the pattern would have directed less care to a higher-risk group.

Explanations remain useful for auditing what a model relies on. Estimating what a treatment does requires a causal design. Estimating which patients benefit most is a question of heterogeneous treatment effects, the clinical counterpart of uplift modeling, and is best supported by randomized data.

Where does causal thinking matter in clinical-model validation?

• What the model predicts. Outcomes in training data occurred under the treatment patterns of the time. A risk score therefore predicts risk under usual care, which may differ from risk if untreated. The intended use should say which one is meant.
• Shortcuts and dataset shift. Models can learn features of the acquisition process, such as the scanner, the ward or a marker in the image, instead of the disease. A causal view of how the data was generated helps predict which associations will hold at a new site, and motivates external validation.
• Effects of deployment. Once a model influences care, successful interventions prevent the outcomes it predicted, so its measured performance changes and data collected after deployment is affected by the model itself.
• Accuracy versus benefit. Discrimination and calibration show that a model predicts well. Whether using it improves patient outcomes is a rung-two question that needs an impact evaluation, ideally a randomized or stepped-wedge comparison of care with and without the model.

The wider validation process, including external validation, calibration and reporting standards, is covered in Medical AI: Validation and the Regulatory Landscape.

Frequently asked questions

What is confounding by indication?

Confounding by indication occurs when the clinical reason for giving a treatment is itself related to the outcome. Because sicker patients are more likely to be treated, a simple comparison of treated and untreated patients mixes the effect of the treatment with the effect of the underlying severity.

What is immortal time bias?

Immortal time bias arises when follow-up for the treated group includes a period during which, by the way the group is defined, the outcome could not have occurred, such as the time between cohort entry and the first prescription. Counting that period as treated time, or excluding it, makes the treatment look more protective than it is.

What is target trial emulation?

Target trial emulation, set out by Miguel Hernán and James Robins, is a framework for analysing observational data by first specifying the protocol of the randomized trial you would ideally run, and then emulating each element of that protocol with the available data. Aligning eligibility, treatment assignment and the start of follow-up removes several common design biases.

Can a prediction model's feature importance be read as a treatment effect?

No. Feature importance describes how much a model relies on a variable to predict the outcome in the data it was trained on. It reflects correlations, including those created by confounding and by existing treatment patterns, and does not estimate what would happen to a patient if that variable were changed.

Do propensity scores remove all confounding?

No. Propensity score methods balance the covariates that were measured and included in the model. They do not adjust for confounders that were not measured or were measured poorly, so results should be accompanied by balance checks and sensitivity analyses.

Related in this wiki: Causal Inference in Business and Data Science, The Ladder of Causality, Uplift Modeling and Medical AI: Validation and the Regulatory Landscape. For how MLAIA approaches this work, see clinical AI and prediction model validation.

MLAIA consults on medical AI and clinical data science — model development, validation design, and regulatory-grade evaluation. See our medical AI domain expertise or contact us. Related reading: Unleashing the Power of Graphs for Machine Learning.