THE FOLD / RESPAWN / GARBAGE COLLECTION / THE RATE-DISTORTION
THE RATE-DISTORTION
how small it gets if you say what you can lose
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Lossless compression has a floor: the entropy. But almost nothing you actually store is kept perfectly — photographs, audio, video are all thrown away on purpose. So the real question is not “how small can this get” but “how small can this get if I am willing to be wrong by this much”. Shannon answered it in 1959 with a curve, R(D), giving the minimum bits per symbol for any permitted average distortion D. Everything below the curve is impossible. Everything above it is merely engineering.
LIT verified live: for a Bernoulli(0.25) source under Hamming distortion R(D) = H(p) − H(D), with R(0) = 0.811278 bits and R(0.25) = 0 exactly, monotone and convex throughout; the Gaussian bound R(D) = ½log₂(σ²/D) inverts D(R) = σ²2−2R exactly at every rate tested; no quantiser built here beats the bound; a Lloyd–Max quantiser climbs 2.211 → 2.432 → 2.565× the bound at N = 8, 16, 32 without ever crossing the Panter–Dite constant √3π/2 = 2.7207 (4.347 dB), and its measured distortions 0.034548 and 0.002505 reproduce Max’s 1960 published 0.034545 and 0.002499 to within 0.25%; and entropy-coded uniform quantisation converges to 1.4234×, matching the space-filling loss πe/6 = 1.4233 = 1.533 dB = 0.2546 bits per sample.
LIT verified live: for a Bernoulli(0.25) source under Hamming distortion R(D) = H(p) − H(D), with R(0) = 0.811278 bits and R(0.25) = 0 exactly, monotone and convex throughout; the Gaussian bound R(D) = ½log₂(σ²/D) inverts D(R) = σ²2−2R exactly at every rate tested; no quantiser built here beats the bound; a Lloyd–Max quantiser climbs 2.211 → 2.432 → 2.565× the bound at N = 8, 16, 32 without ever crossing the Panter–Dite constant √3π/2 = 2.7207 (4.347 dB), and its measured distortions 0.034548 and 0.002505 reproduce Max’s 1960 published 0.034545 and 0.002499 to within 0.25%; and entropy-coded uniform quantisation converges to 1.4234×, matching the space-filling loss πe/6 = 1.4233 = 1.533 dB = 0.2546 bits per sample.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at GARBAGE COLLECTION, which is the honest name for lossy compression. A collector does not ask “is this needed?” — it asks “is this reachable?”, and frees everything else without apology. R(D) is the same trade written as a law: name what you can afford to lose, and the bit count follows from that and nothing else.
AVAN (AI) got the headline result and the solver wrong in opposite directions. The entropy-coded quantiser landed on πe/6 to four digits immediately. The Lloyd–Max solver did not: at N = 64 it reported 3.376× the bound — above the constant it is supposed to approach from below. That overshoot was the tell. It was not slow convergence but grid resolution: at a step of 0.002 the quantiser cells near the peak were only about 25 grid points wide, and the discretisation inflated the measured distortion by 27.6%. Refining to 0.0002 reproduced Max’s published table. A second temptation was to keep N = 64 anyway and quote a number that had merely stopped moving; the page stops at N = 32, which genuinely converges in 458 sweeps, and says so. Panter–Dite is asymptotic, so the claim here is that the ratio climbs toward the constant and never crosses it — not that it equals it.
AVAN (AI) got the headline result and the solver wrong in opposite directions. The entropy-coded quantiser landed on πe/6 to four digits immediately. The Lloyd–Max solver did not: at N = 64 it reported 3.376× the bound — above the constant it is supposed to approach from below. That overshoot was the tell. It was not slow convergence but grid resolution: at a step of 0.002 the quantiser cells near the peak were only about 25 grid points wide, and the discretisation inflated the measured distortion by 27.6%. Refining to 0.0002 reproduced Max’s published table. A second temptation was to keep N = 64 anyway and quote a number that had merely stopped moving; the page stops at N = 32, which genuinely converges in 458 sweeps, and says so. Panter–Dite is asymptotic, so the claim here is that the ratio climbs toward the constant and never crosses it — not that it equals it.
3 ONE DIMENSION
R(D) for a Bernoulli(0.25) source. Below the curve is not hard — it is impossible.
4 TWO DIMENSIONS · INTERACTIVE
Build the real quantisers and measure how far short of Shannon they land.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: the gap between what Shannon allows and what one dimension can reach.
AVAN’s addition (the inverse-companion): the 1.533 dB has nothing to do with cleverness and everything to do with shape. A scalar quantiser cuts space into cubes because it decides each coordinate separately; the optimal partition wants spheres. πe/6 is precisely the penalty for the cube, and no scalar algorithm can escape it — not because the algorithms are bad but because the question was asked one axis at a time. Read backwards, the constant is a measurement of how much is lost by treating a joint problem as a list of separate ones. That is a statement about decomposition, not about compression, and it is why vector quantisation exists at all.
LIT R(D)=H(p)-H(D) for Bernoulli(0.25) with R(0)=0.811278 and R(0.25)=0 exactly, monotone and convex; the Gaussian bound inverts D(R) exactly at every rate; no quantiser built here beats the bound; Lloyd-Max climbs 2.211, 2.432, 2.565 times the bound at N=8,16,32 without crossing sqrt(3)pi/2 = 2.7207, with measured distortions 0.034548 and 0.002505 reproducing Max's 1960 published 0.034545 and 0.002499 to within 0.25%; entropy-coded uniform converges to 1.4234 against pi*e/6 = 1.4233 = 1.533 dB
FIG The Lloyd-Max solver first reported 3.376 at N=64 — above the constant it approaches from below. The cause was grid resolution, not convergence: cells near the peak spanned only ~25 points and distortion came out 27.6% high. The page stops at N=32, which converges in 458 sweeps, rather than quote an unconverged N=64. Panter-Dite is asymptotic, so the claim is that the ratio climbs toward the constant, not that it reaches it.
FIG The Lloyd-Max solver first reported 3.376 at N=64 — above the constant it approaches from below. The cause was grid resolution, not convergence: cells near the peak spanned only ~25 points and distortion came out 27.6% high. The page stops at N=32, which converges in 458 sweeps, rather than quote an unconverged N=64. Panter-Dite is asymptotic, so the claim is that the ratio climbs toward the constant, not that it reaches it.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GARBAGE COLLECTION · David Lee Wise (ROOT0), with AVAN