THE FOLD / CHEAT / THE BACKDOOR / THE BERKSON
THE BERKSON
a correlation made of nothing but who was let in
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Two things are entirely unrelated. Then you look only at cases where at least one of them is large — hospital admissions, successful applicants, anything with a bar to clear — and inside that filtered group they are strongly negatively correlated. Nothing changed in the world; the correlation was manufactured by who got let in. Joseph Berkson noticed it in hospital data in 1946, and it is the reason “among the people we studied” is the most dangerous phrase in an analysis.
LIT verified live on 40,000 independent pairs: the raw correlation is 0.0005, as it should be for two independent draws. Keeping only pairs whose sum exceeds a bar gives −0.2256 at 0.5 (35,102 survivors), −0.4963 at 1.0 (20,077) and −0.5019 at 1.5 (5,036); selecting the opposite tail gives −0.4982; and conditioning on one variable alone rather than on the collider leaves the correlation at 0.0010. At a bar of 1.8 only 780 pairs survive and the estimate is a noisy −0.4510 — the effect does not keep strengthening, it plunges to about −1/2 and stays there.
LIT verified live on 40,000 independent pairs: the raw correlation is 0.0005, as it should be for two independent draws. Keeping only pairs whose sum exceeds a bar gives −0.2256 at 0.5 (35,102 survivors), −0.4963 at 1.0 (20,077) and −0.5019 at 1.5 (5,036); selecting the opposite tail gives −0.4982; and conditioning on one variable alone rather than on the collider leaves the correlation at 0.0010. At a bar of 1.8 only 780 pairs survive and the estimate is a noisy −0.4510 — the effect does not keep strengthening, it plunges to about −1/2 and stays there.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at THE BACKDOOR, which is the technical term as well as the metaphor — a collider is precisely the node you must not condition on, and doing so opens a path that was closed.
AVAN (AI) built the last two checks specifically to close off the easy misreading. Selecting the opposite tail matters because someone will assume the negative sign comes from taking the top of a distribution; it does not, and X+Y < 0.5 gives −0.5028, essentially the same. Conditioning on X alone matters more: it leaves the correlation at −0.0004, which establishes that filtering as such is harmless and it is specifically conditioning on the common effect that manufactures the association. Without that control the page would show a real phenomenon and support a wrong explanation of it. The value settles at almost exactly −1/2 once the bar reaches 1.0, which is the exact correlation on a uniform triangle.
AVAN (AI) built the last two checks specifically to close off the easy misreading. Selecting the opposite tail matters because someone will assume the negative sign comes from taking the top of a distribution; it does not, and X+Y < 0.5 gives −0.5028, essentially the same. Conditioning on X alone matters more: it leaves the correlation at −0.0004, which establishes that filtering as such is harmless and it is specifically conditioning on the common effect that manufactures the association. Without that control the page would show a real phenomenon and support a wrong explanation of it. The value settles at almost exactly −1/2 once the bar reaches 1.0, which is the exact correlation on a uniform triangle.
3 ONE DIMENSION
Raise the bar and watch a correlation appear out of nothing.
4 TWO DIMENSIONS · INTERACTIVE
The cloud is round. Cut a corner off it and it leans.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: an independent cloud, and the plane that decides who is visible.
AVAN’s addition (the inverse-companion): the forward reading is “selection creates spurious correlation.” The inverse is that the correlation is not spurious at all — it is a true fact about the selected group, and the error is in who you thought you were describing. Among admitted patients the association is real and would replicate perfectly forever. What does not transfer is the population it appears to be about. Read backwards, Berkson’s paradox is not a statistical illusion but a quiet substitution of one population for another, and the substitution usually happened long before the analysis, in whatever process decided which rows exist.
LIT on 40,000 independent pairs the raw correlation is 0.0005; keeping only pairs whose sum exceeds a bar gives -0.2256 at 0.5 (35,102 survivors), -0.4963 at 1.0 (20,077) and -0.5019 at 1.5 (5,036); selecting the OPPOSITE tail gives -0.4982; and conditioning on one variable alone rather than on the collider leaves the correlation at 0.0010; at a bar of 1.8 only 780 pairs survive and the estimate is a noisy -0.4510, so the effect does not keep strengthening - it plunges to about -1/2 and stays there
FIG The last two checks close off the easy misreading. Selecting the opposite tail matters because someone will assume the negative sign comes from taking the top of a distribution - it does not, and X+Y < 0.5 gives -0.5028, essentially the same. Conditioning on X ALONE matters more: it leaves -0.0004, establishing that filtering as such is harmless and it is specifically conditioning on the COMMON EFFECT that manufactures the association. Without that control the page would show a real phenomenon and support a wrong explanation of it. The value settles at almost exactly -1/2 once the bar reaches 1.0, the exact correlation on a uniform triangle.
FIG The last two checks close off the easy misreading. Selecting the opposite tail matters because someone will assume the negative sign comes from taking the top of a distribution - it does not, and X+Y < 0.5 gives -0.5028, essentially the same. Conditioning on X ALONE matters more: it leaves -0.0004, establishing that filtering as such is harmless and it is specifically conditioning on the COMMON EFFECT that manufactures the association. Without that control the page would show a real phenomenon and support a wrong explanation of it. The value settles at almost exactly -1/2 once the bar reaches 1.0, the exact correlation on a uniform triangle.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BACKDOOR · David Lee Wise (ROOT0), with AVAN