Causal inference

Everything this module built so far estimates conditional distributions: $P(Y \mid X)$ , learned from data the world happened to generate. But almost every decision that data is supposed to inform is a question about an intervention: what happens to revenue if we lower the price, to churn if we send the email, to recovery if we prescribe the drug. Those are questions about $P(Y \mid do(T))$ — the distribution of $Y$ in a world where we set $T$ ourselves — and no amount of predictive accuracy on $P(Y \mid T)$ answers them, because the world that generated your data chose $T$ for reasons of its own. This lesson is about when and how the gap can be crossed. It is the difference between data science that describes and data science that decides.

The fork: one diagram, most of the problem

Draw variables as nodes and direct causes as arrows. The diagram that poisons most observational comparisons is the fork:

T \longleftarrow Z \longrightarrow Y, \qquad T \longrightarrow Y .

$Z$ — the confounder — causes both the treatment and the outcome. Sicker patients get the aggressive drug and die more; motivated students take the prep course and score higher anyway. The observed contrast decomposes as

\underbrace{\mathbb{E}[Y \mid T{=}1] - \mathbb{E}[Y \mid T{=}0]}_{\text{what you measure}} \;=\; \underbrace{\tau}_{\text{causal effect}} \;+\; \underbrace{b \,\big( \mathbb{E}[Z \mid T{=}1] - \mathbb{E}[Z \mid T{=}0] \big)}_{\text{confounding bias}}

in the linear world ( $Y = \tau T + bZ + \varepsilon$ ), and the second term is zero only when treatment assignment carries no information about $Z$ . A randomized controlled trial achieves exactly that by force: a coin flip severs the $Z \to T$ arrow, every backdoor closes, and the naive difference of means is the causal effect — which is why the A/B test of the previous lesson needed none of this machinery. Causal inference is what remains when you cannot randomize.

Closing the backdoor: adjustment

If the confounder is measured, condition on it. Pearl’s backdoor criterion says: find a set $S$ of variables that blocks every path from $T$ to $Y$ that enters $T$ through an arrow into it (the “backdoors”), without containing any descendant of $T$ . Then

P(Y \mid do(T{=}t)) \;=\; \sum_{s} P(Y \mid T{=}t, S{=}s)\, P(s)

— estimate the effect within each stratum of $S$ , then average over how common each stratum is (not over how common it is among the treated: that re-introduces the bias). In the linear model the stratified average collapses to something you already own from the MLE lesson: regress $Y$ on $T$ and $Z$ together, and the coefficient on $T$ is the adjusted estimate.

Each dot is a unit, plotted by its confounder Z and outcome Y; color is treatment. With Z → T strength up, treated units sit visibly to the RIGHT — they were different before treatment, and the naive contrast bills that difference to τ. The two fitted lines are the adjusted model: their vertical gap is τ̂, and it tracks the truth whatever you do to the confounding sliders — including τ = 0 with strong confounding, the classic mirage where naive says “the treatment works” and adjustment says “the treatment does nothing; selection did everything”. Now the honest caveat: this works because Z is on the plot. Delete the Z axis in your mind and nothing on the remaining 1-D view distinguishes causation from selection. Adjustment is only as good as your census of confounders, and no statistic computed from (T, Y) alone can tell you the census is complete.

Two conditions do the real work, and both fail silently. No unmeasured confounding: every backdoor variable is in your data — untestable from the joint distribution of $(T, Y, S)$ alone; it is a claim about the world, defended with subject knowledge, not p-values. Overlap: every stratum needs both treated and control units — if all the sick took the drug, “the effect among the sick” is an extrapolation wearing an estimate’s clothes. (Propensity-score methods — matching or weighting on $P(T{=}1 \mid S)$ — are alternative estimators of the same backdoor formula, not an escape from either assumption.)

The collider: why “control for everything” is wrong

Reverse the fork’s arrows and the logic reverses with them. In a collider, $T \to C \leftarrow Y$ , the path through $C$ is already blocked — and conditioning on $C$ opens it, manufacturing dependence where none exists. The canonical instance (Berkson, 1946): talent and looks are independent in the population, but among admitted celebrities — admitted because of either — they correlate negatively, since fame with no talent implies looks. Condition on “hospitalized”, “hired”, “survived”, “responded to the survey”, and you have selected on a collider of many causes; every dataset assembled by such a filter carries these phantom correlations baked in. This is why the backdoor criterion forbids conditioning on descendants of $T$ , and why the reflex “add every available covariate to the regression” is not conservative but destructive: adjustment sets are chosen by diagram, not by abundance.

No measured confounder? Borrow an experiment

Difference-in-differences. Sometimes the world runs the experiment sloppily: one group gets treated at a known time, another never does. Neither naive comparison works — before-vs-after eats any common trend, treated-vs-control eats any baseline gap — but their difference subtracts both nuisances:

\hat{\tau}_{\text{DiD}} = \big( \bar{Y}_{T,\text{post}} - \bar{Y}_{T,\text{pre}} \big) - \big( \bar{Y}_{C,\text{post}} - \bar{Y}_{C,\text{pre}} \big),

valid under parallel trends: absent treatment, both groups would have moved together. That counterfactual is unobservable by definition; the supporting evidence is that pre-treatment trends track (necessary, not sufficient).

The dashed line is the counterfactual — the treated group’s world without treatment — the thing no dataset ever contains. At γ = 0 read the three estimators: before–after is off by the trend, cross-section is off by the baseline gap, DiD alone lands on τ. Then turn γ: the treated group now trends differently for its own reasons, the counterfactual bends away from the control group’s shadow, and DiD absorbs the divergence as if it were treatment. Nothing in the post-period data flags this — the diagnosis lives entirely in the pre-period (do the trends track before the red line?) and in your argument for why they would have continued to.

Instrumental variables. When confounding is unmeasured and there is no clean before/after, look for an instrument $W$ : a variable that (i) moves the treatment (relevance — checkable) and (ii) touches the outcome only through the treatment (exclusion — an untestable structural claim, argued not computed). Then $W$ ‘s wiggle in $Y$ can only have traveled via $T$ , so the ratio of wiggles isolates the causal path:

\hat{\tau}_{IV} = \frac{\operatorname{Cov}(Y, W)}{\operatorname{Cov}(T, W)} .

Classic instruments: randomized encouragement to take the treatment, distance to the nearest facility, lottery-assigned eligibility. Two honest warnings. Weak instruments: the denominator near zero turns the ratio into a noise amplifier — small-sample IV with a weak first stage is often worse than the bias it treats. And local effects: IV estimates the effect among compliers, the units the instrument actually moved — which may not be the population you plan to treat.

The working method

The estimators differ; the discipline is one procedure. Draw the diagram first — every arrow is a claim someone can dispute, which is precisely its value. Choose the adjustment set from the diagram (confounders in, colliders and descendants of treatment out). State the untestable assumption your design leans on — no unmeasured confounding, parallel trends, exclusion — out loud, in writing, where a domain expert can attack it. Then run the sensitivity question: how strong would a violation have to be to erase the conclusion? An effect that survives “an unmeasured confounder twice as strong as any measured one” is a finding; one that dies to a whisper of confounding is a prediction wearing a costume. ML enters as a better estimator of the pieces — flexible models for the outcome and propensity surfaces (double/debiased ML) — but it inherits the same identification assumptions; no architecture launders correlation into causation.

Exercises

Work these before the next module

In the linear fork model $Y = \tau T + bZ + \varepsilon$ with $\Pr(T{=}1\mid Z) = \sigma(aZ)$ , derive the naive contrast’s bias term $b\,(\mathbb{E}[Z\mid T{=}1] - \mathbb{E}[Z\mid T{=}0])$ and explain why each of $a = 0$ , $b = 0$ , and randomization kills it.
Solution
Worked solutions are part of Premium — unlock all of them for £5/month →
5 more exercises — each with a worked solution — are part of Premium. Unlock everything for £5/month →

References

J. Pearl, Causality: Models, Reasoning, and Inference, 2nd ed., 2009 — do-calculus, backdoor criterion, colliders.
J. Angrist & J.-S. Pischke, Mostly Harmless Econometrics, 2009 — DiD, IV, and the design-based mindset.
D. Card & A. Krueger, “Minimum Wages and Employment”, AER 1994 — the difference-in-differences study that launched a thousand replications.
J. Berkson, “Limitations of the Application of Fourfold Table Analysis to Hospital Data”, Biometrics Bulletin 1946 — the collider bias that bears his name.
V. Chernozhukov et al., “Double/Debiased Machine Learning for Treatment and Structural Parameters”, Econometrics Journal 2018 — where ML legitimately enters the causal pipeline.