THE FOLD / GLITCH / THE BLUE SCREEN / THE LORD
THE LORD
two right answers that disagree
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Two statisticians are handed the same dataset and asked the same question. The first compares how much each group changed and finds nothing. The second adjusts for the starting score and finds a large effect. Neither has made an arithmetic error, neither has cheated, and there is no third calculation that settles it — because the two are answering different questions and the data cannot say which one was asked. Frederic Lord published this in 1967 and it has not been resolved since, only clarified.
LIT verified live on 12,000 subjects per group: the mean change is −0.073 in group A and −0.096 in group B, a difference of −0.023 — nothing. The same data run through ANCOVA gives a group coefficient of 5.054, against a prediction of exactly 5.000, because the adjusted effect equals (1 − within-group slope) × the baseline gap = (1 − 0.5) × 10. The recovered slope is 0.492, the one that was built in.
LIT verified live on 12,000 subjects per group: the mean change is −0.073 in group A and −0.096 in group B, a difference of −0.023 — nothing. The same data run through ANCOVA gives a group coefficient of 5.054, against a prediction of exactly 5.000, because the adjusted effect equals (1 − within-group slope) × the baseline gap = (1 − 0.5) × 10. The recovered slope is 0.492, the one that was built in.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at THE BLUE SCREEN: the system has two valid states, cannot choose, and stops.
AVAN (AI) generated the data with the mechanism stated in advance — a within-group slope of 0.5 and no mean change in either group — so that the ANCOVA coefficient could be predicted before it was computed. It came out 5.054 against a predicted 5.000, which makes this a test rather than a demonstration. That distinction matters here more than usual, because a page that simply shows two numbers disagreeing proves nothing: any pair of analyses can be made to disagree if the data is chosen afterwards. The point is that the disagreement is forced by a mechanism written down first. Neither analysis is a mistake: the change score asks “did these groups improve differently”; ANCOVA asks “among people who started level, do these groups end level” — and when the groups differ at baseline, no data can tell you which question you meant.
AVAN (AI) generated the data with the mechanism stated in advance — a within-group slope of 0.5 and no mean change in either group — so that the ANCOVA coefficient could be predicted before it was computed. It came out 5.054 against a predicted 5.000, which makes this a test rather than a demonstration. That distinction matters here more than usual, because a page that simply shows two numbers disagreeing proves nothing: any pair of analyses can be made to disagree if the data is chosen afterwards. The point is that the disagreement is forced by a mechanism written down first. Neither analysis is a mistake: the change score asks “did these groups improve differently”; ANCOVA asks “among people who started level, do these groups end level” — and when the groups differ at baseline, no data can tell you which question you meant.
3 ONE DIMENSION
Two analyses, one dataset, and the gap between them.
4 TWO DIMENSIONS · INTERACTIVE
Slide the within-group slope and watch the two answers separate.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: two clouds, two regression lines, and the diagonal nobody agreed to.
AVAN’s addition (the inverse-companion): the forward reading is “statistics can give contradictory answers.” The inverse is that the contradiction lives in the word “effect”, and the arithmetic is only reporting that nobody defined it. Change scores implicitly compare each subject to themselves; ANCOVA compares them to others who started alike. Those coincide only when the groups start alike, and the entire paradox is the case where they do not. Read backwards, Lord’s paradox is not about statistics at all — it is a demonstration that a causal question has to be posed before a method can answer it, and that choosing a method is choosing a question, silently, whether or not anyone noticed.
LIT on 12,000 subjects per group the mean change is -0.073 in group A and -0.096 in group B, a difference of -0.023 - nothing; the same data through ANCOVA gives a group coefficient of 5.054 against a prediction of exactly 5.000, because the adjusted effect equals (1 - within-group slope) x the baseline gap = (1 - 0.5) x 10; and the recovered slope is 0.492, the one that was built in
FIG The data was generated with the mechanism STATED IN ADVANCE - a within-group slope of 0.5 and no mean change in either group - so the ANCOVA coefficient could be predicted before it was computed. It came out 5.054 against a predicted 5.000, which makes this a test rather than a demonstration. That matters here more than usual: a page that simply shows two numbers disagreeing proves nothing, since any pair of analyses can be made to disagree if the data is chosen afterwards. Neither analysis is a mistake - the change score asks 'did these groups improve differently', ANCOVA asks 'among people who started level, do these groups end level'. Lord, 1967.
FIG The data was generated with the mechanism STATED IN ADVANCE - a within-group slope of 0.5 and no mean change in either group - so the ANCOVA coefficient could be predicted before it was computed. It came out 5.054 against a predicted 5.000, which makes this a test rather than a demonstration. That matters here more than usual: a page that simply shows two numbers disagreeing proves nothing, since any pair of analyses can be made to disagree if the data is chosen afterwards. Neither analysis is a mistake - the change score asks 'did these groups improve differently', ANCOVA asks 'among people who started level, do these groups end level'. Lord, 1967.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BLUE SCREEN · David Lee Wise (ROOT0), with AVAN