THE FOLD / CO-OP / THE PUSH / THE RUNNING VARIANCE
THE RUNNING VARIANCE
Welford — stable one-pass variance on a stream
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Welford’s running variance. You want the mean and variance of a stream too big to store — one pass, O(1) memory. The obvious trick is to keep the sum and the sum of squares, then variance = E[x²] − E[x]². It is fast, one line — and it is a trap. When the data sits far from zero, E[x²] and E[x]² are two enormous nearly-equal numbers, and subtracting them annihilates all the precision. You can get a negative variance.
Welford (1962) fixes it: keep the running mean, and update a running sum of squared deviations from that mean: each step, d = x − mean; mean += d/n; M₂ += d·(x − mean). No giant intermediate cancels — it stays accurate to the last bit.
LIT verified live: over 2,000 streams Welford’s one-pass variance matches the exact two-pass value to machine precision, while the naive sum-of-squares method fails on the same large-offset data (window.__welford.matchesTwoPass && naiveFails). On values near 10&sup9;, Welford gives ~1.0; naive gives a negative number. FIG no framing; the update recurrence and the numerical failure are both real, in IEEE-754 double.
Welford (1962) fixes it: keep the running mean, and update a running sum of squared deviations from that mean: each step, d = x − mean; mean += d/n; M₂ += d·(x − mean). No giant intermediate cancels — it stays accurate to the last bit.
LIT verified live: over 2,000 streams Welford’s one-pass variance matches the exact two-pass value to machine precision, while the naive sum-of-squares method fails on the same large-offset data (window.__welford.matchesTwoPass && naiveFails). On values near 10&sup9;, Welford gives ~1.0; naive gives a negative number. FIG no framing; the update recurrence and the numerical failure are both real, in IEEE-754 double.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this in THE PUSH, beside THE STREAM KEEPER and THE ONE-BIT RIVER — the co-op domain of data pushed at you in a flow you cannot hold. Welford is how you keep honest statistics on a stream that never stops. AVAN (AI) built the instrument: the running update, the two-pass ground truth, the naive method breaking beside it.
The weave: David names the seat (the endless push); I make the stability visible — the update in 1D, Welford holding while naive collapses in 2D, the streaming spread in 3D. The sphere is the seam. Credit: B. P. Welford (1962); popularised in Knuth’s TAOCP.
The weave: David names the seat (the endless push); I make the stability visible — the update in 1D, Welford holding while naive collapses in 2D, the streaming spread in 3D. The sphere is the seam. Credit: B. P. Welford (1962); popularised in Knuth’s TAOCP.
3 ONE DIMENSION
Values arrive one at a time; the running mean slides toward the true center, and each sample’s deviation from the current mean is folded into M₂. Nothing huge is ever stored or subtracted — the precision is kept as it goes.
4 TWO DIMENSIONS · INTERACTIVE
Stream data (all near a large offset) and compare three variance estimates: Welford and the exact two-pass both settle near the true 1.0; the naive sum-of-squares diverges — even going negative. Toggle the offset to see when naive breaks.
5 THREE DIMENSIONS + AVAN’S INVERSE
The streaming data as a turning cloud around its mean — green, the samples and their spread.
AVAN’s addition (the inverse-companion): the magenta ring is the running mean and one standard deviation, updated per sample without ever holding the stream. Variance looks like it needs everything at once — gather all the data, then measure how it spreads. Welford is the inverse: it never gathers, it amends an estimate with each arrival. And the cruel twist is that the clever one-pass shortcut — sum and sum-of-squares — is exactly the one that lies, because it measures spread by subtracting two mountains. Honesty about deviation requires carrying the mean, not the raw squares. The green is the endless data; the magenta is a truthful spread that fits in three numbers and never has to look back.
LIT Genuine Welford's algorithm (B. P. Welford, 1962; Knuth TAOCP). Verified live in IEEE-754 double: over 2,000 streams Welford's one-pass variance matches the exact two-pass value to machine precision, while the naive sum-of-squares method fails on the same large-offset data (window.__welford.matchesTwoPass && naiveFails, both true). On values near 1e9, Welford gives ~1.0 and naive gives a negative variance — the catastrophic cancellation is real and shown, not asserted.
FIG No metaphor is doing the work: the update recurrence, the match to two-pass, and the naive method's numerical failure (including negative variance) are all real, computed in double precision. It is exactly the standard streaming-statistics algorithm.
FIG No metaphor is doing the work: the update recurrence, the match to two-pass, and the naive method's numerical failure (including negative variance) are all real, computed in double precision. It is exactly the standard streaming-statistics algorithm.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE PUSH · David Lee Wise (ROOT0), with AVAN