Causal inference lab · Ladder of causality
What is the ladder of causality? One dataset, three answers
The ladder of causality, Judea Pearl’s ladder of causation, separates three kinds of question: association (seeing), intervention (doing) and counterfactual (imagining). They sound alike and have different answers. This demo asks all three of one synthetic dataset about a retention offer and gets three different numbers.
These demos are illustrative. Every customer, account, region and order is synthetic, generated and analysed in your browser, and every number shown is computed from that synthetic data. They show general principles of causal analysis, not the results of any real campaign, product or project.
Nothing is uploaded. No customer data is used.
Association / intervention / counterfactual
Three questions. Three different numbers.
What this shows: how often offered customers renew, what the offer does to renewal, and how many renewals the offer caused are three questions. One dataset gives each a different answer.
Choose the question
15% of the other customers get the offer at every setting. At 15% the offer is unrelated to risk.
One retention offer, asked three ways
10,000 synthetic customersSynthetic data · illustrative · not results from a real business
Forecasting is a seeing question, a rollout decision is a doing question, and attribution or credit for an outcome is an imagining question. Using the number from a lower rung for a decision on a higher one is a common and costly mix-up.
What this experiment models
10,000 synthetic customers, 40% at risk. Without the offer 35% of at-risk and 80% of other customers renew; the offer adds 25 and 5 percentage points. 15% of other customers get the offer; the slider sets the share of at-risk customers who do. Every customer has both outcomes fixed in advance, and the offer never turns a renewal into a loss. Seeing is the difference in observed renewal rates. Doing is the difference between everyone’s outcome with the offer and without it; the estimate from the data is the within-group difference averaged over the two groups by size. Imagining counts offered customers who renewed and would not have without the offer; the estimate from the data uses, within each group, the ratio of renewal rates without and with the offer, which is valid only because the offer never hurts. Fixed seed.
The principle
How it works
What the demo shows
The demo generates 10,000 synthetic customers, 40% of them at risk of leaving. A retention offer raises the chance of renewal by 25 percentage points for at-risk customers and by 5 points for the others. By default 70% of at-risk customers and 15% of the others get the offer, so 75% of offered customers are at risk, against 18% of the rest.
Seeing. Customers who got the offer renewed at 67.1%; those who did not, at 71.0%. The difference is −3.9 pts. This is a correct statement about the data, and the right basis for predicting whether a customer you observe will renew.
Doing. If everyone got the offer, 75.3% would renew; if no one did, 61.8%. The effect of the offer is +13.5 pts. The simulation knows this because it holds each customer’s outcome under both choices. Estimated from the observed data alone, by comparing within the at-risk and other groups and averaging by group size, it comes to +14.3 pts.
Imagining. Of the 2,478 customers who got the offer and renewed, 68.6% would have renewed without it, and 31.4% renewed because of it. Estimated from the data, under the assumption that the offer never makes anyone less likely to renew, the figure is 31.5%.
Move the slider to 15%, where at-risk customers are no more likely to get the offer than anyone else, and seeing and doing agree apart from noise.
Why it matters for a decision
Each rung matches a different decision. Forecasting next quarter’s renewals is a seeing task, and a predictive model is the right tool. Deciding whether to roll the offer out is a doing task; here the seeing number gets its sign wrong. Crediting renewals to the offer is an imagining task, and the answer is that most of those customers would have renewed anyway.
What to check in your own data
- Which rung is the question on? Who will renew, what happens if we send the offer, and did the offer cause this renewal each need different evidence.
- For a doing question, is there an experiment, or a recorded reason for who got the action that you can adjust for?
- For an imagining question, what assumption links a customer’s outcome with the action to their outcome without it? State it, because the data cannot test it. Without one, only a range can be given.
- Are model explanations being read as effects? Feature importance describes what a predictive model uses, which is a seeing answer.
The ladder of causation on the practice page walks through the three rungs with business examples. For how the questions are handled in a project, see causal inference consulting; for background, the causal inference guide.
FAQ
Common questions
What are the three rungs of the ladder of causation?
Association: what does observing one thing tell me about another? Intervention: what happens if I act? Counterfactual: given what happened, what would have happened had I acted differently? The framing is Judea Pearl’s.
Why can’t a predictive model answer what-if questions?
A predictive model learns associations in data produced under past decisions. Changing the decision changes how the data is produced, and the model holds no information about that unless it was trained on experiments or built on a causal design.
What is a counterfactual?
A statement about the same unit under a different history, for example whether a customer who received an offer and renewed would have renewed without it. Only one history is ever observed, so counterfactual answers need a causal model with assumptions beyond those needed for average effects.
Why are the three numbers in the demo different?
They answer different questions. −3.9 pts compares two groups made up of different customers. +13.5 pts is the average change from giving the offer. 31.4% is a share of one specific group, the offered customers who renewed. None of them is an estimate of another.
Causal inference lab
More demos
All five demos on one page: the causal inference labCausal inference consulting →Causal inference guide →