THE CLOCK SKEW

Two machines timestamp two events. The second genuinely happened after the first. Whether the timestamps agree depends on how well the clocks are set, and nothing in the data says how well that was.

THE CLOCK SKEW · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE SYNC ◆ .dlw.fold THE FOLD / CO-OP / THE SYNC / THE CLOCK SKEW THE CLOCK SKEW one millisecond, 189 wrong orders 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Two machines timestamp two events. The second event genuinely happened after the first. Whether the timestamps agree depends on how well the two clocks are set, and nothing in the data says how well that was. LIT verified live. ten thousand pairs of events, the second always genuinely later by up to 20 ms. At 0 skew the timestamps invert 0 times. At 1 ms of skew they invert 189 times, at 5 ms 848 , at 10 ms 1,687 , at 25 ms 3,205 and at 50 ms 3,954 — 39.5% , approaching the half you would get from a coin. The inversion count rises monotonically with skew across every step, and no inverted pair is distinguishable from a correctly ordered one by looking at it. 2 HOW IT WAS WEAVED · AI + HUMAN Clock skew is why distributed systems use logical clocks — Lamport (1978) and vector clocks — rather than trusting wall-clock ordering, and why Spanner buys atomic clocks to bound it instead. AVAN (AI) swept the skew rather than asserting that ordering is unsafe, because the curve is the argument. The failure does not appear at some threshold; it is already 1.4% at a single millisecond and degrades smoothly to a coin flip. There is no skew small enough to make the ordering sound, only skew small enough to make the errors rare — which is a different property, and not the one anyone thinks they are relying on. 3 ONE DIMENSION One millisecond of skew. 189 events in the wrong order. 4 TWO DIMENSIONS · INTERACTIVE Turn up the skew and watch causality dissolve. more skew ▶ less 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is that you cannot order distributed events by wall clock. The inverse is that the timestamps were never claims about order — they are claims about the reading of a local dial , and order is something a reader inferred. Read backwards, the data is not corrupted and no clock is broken; two honest measurements simply do not compose into a sequence, and every system that sorted by timestamp added an ordering that was never in the observations. pause spin LIT ten thousand pairs of events, the second always genuinely later by up to 20 ms: at 0 skew the timestamps invert 0 times, at 1 ms of skew 189 times, at 5 ms 848, at 10 ms 1,687, at 25 ms 3,205 and at 50 ms 3,954 - 39.5%, approaching the half you would get from a coin - with the inversion count rising monotonically across every step and no inverted pair distinguishable from a correctly ordered one FIG Clock skew is why distributed systems use logical clocks - Lamport (1978) and vector clocks - rather than trusting wall-clock ordering, and why Spanner buys atomic clocks to bound it instead. AVAN swept the skew rather than asserting that ordering is unsafe, because the curve is the argument: the failure is already 1.9% at a single millisecond and degrades smoothly to a coin flip. There is no skew small enough to make the ordering sound, only small enough to make the errors rare. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SYNC · David Lee Wise (ROOT0), with AVAN


THE UNIX EPOCH

I-13 v2.0 | net = binds - k | 4 planes, 13 symbols, 12 operants, 5 cortex rules | sha 64881ebf

THE UNIX EPOCH · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE WALL ◆ .dlw.fold THE FOLD / BOSS / THE WALL / THE UNIX EPOCH THE UNIX EPOCH 2038, and then 1901 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A signed thirty-two bit count of seconds from 1970 reaches its largest value at three fourteen in the morning on the nineteenth of January 2038, and the next second is 1901. LIT verified live. the largest signed 32-bit integer is 2,147,483,647 . Interpreted as seconds since the epoch that is 2038-01-19T03:14:07Z exactly. One more second wraps to −2,147,483,648 , which reads as 1901-12-13 — not a crash, not an error, a date. Reading the same field as unsigned buys until 2106-02-07 and no further. The epoch itself is 1970-01-01T00:00:00Z , and every one of those days is assumed to be exactly 86,400 seconds long. 2 HOW IT WAS WEAVED · AI + HUMAN The 2038 problem is the direct descendant of Y2K and is already live in anything computing a date thirty-odd years out — mortgages, bonds, certificate expiry. AVAN (AI) computed the wrapped date rather than describing the overflow, because 1901-12-13 is the part that makes it dangerous. An overflow that threw would be found immediately in testing; this one produces a valid timestamp, comparisons against it succeed, and a record dated 1901 sorts to the top of a list rather than raising anything. The failure is not that the number breaks — it is that it keeps working. 3 ONE DIMENSION 2,147,483,647 seconds. Then 1901. 4 TWO DIMENSIONS · INTERACTIVE Step over the boundary one second at a time. +1 second ▶ -1 second jump to the edge 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is that thirty-two bits are not enough. The inverse is that the width was never a statement about time, it was a statement about memory in 1971 — and it has outlived every machine it was chosen for. Read backwards, the interesting thing is not the ceiling but the silence at it: a type whose range is exhausted does not announce itself, it wraps into the middle of its own domain and keeps producing answers, which is why the fix has to happen decades before the date and never feels urgent until it is. pause spin LIT the largest signed 32-bit integer is 2,147,483,647, which as seconds since the epoch is exactly 2038-01-19T03:14:07Z, and one more second wraps to -2,147,483,648 which reads as 1901-12-13 - not a crash, not an error, a date; reading the same field as unsigned buys until 2106-02-07 and no further, the epoch itself is 1970-01-01T00:00:00Z, and every one of those days is assumed to be exactly 86,400 seconds long FIG The 2038 problem is the direct descendant of Y2K and is already live in anything computing a date thirty-odd years out - mortgages, bonds, certificate expiry. AVAN computed the wrapped date rather than describing the overflow, because 1901-12-13 is the part that makes it dangerous: an overflow that threw would be found immediately in testing, while this one produces a valid timestamp, comparisons against it succeed, and a record dated 1901 sorts to the top of a list rather than raising anything. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE WALL · David Lee Wise (ROOT0), with AVAN


THE NEGATIVE ZERO

THE NEGATIVE ZERO · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · DIVIDE BY ZERO ◆ .dlw.fold THE FOLD / GLITCH / DIVIDE BY ZERO / THE NEGATIVE ZERO THE NEGATIVE ZERO five say same, five say different 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION There are two zeros. They are equal, they print the same, and half the operations in the language can tell them apart. LIT verified live. putting 0 and −0 through 10 ordinary operations, 5 treat them as the same value and 5 distinguish them. a === b is true and Object.is(a,b) is false . String(a) === String(b) is true ; 1/a === 1/b is false , because the reciprocals are Infinity and −Infinity . [0].indexOf(-0) finds it at index 0 , and Math.min(0,-0) returns the negative one. The split is exactly down the middle and there is no rule for which side an operation lands on except its own history. 2 HOW IT WAS WEAVED · AI + HUMAN Signed zero is IEEE 754: the sign bit is independent of the magnitude, so zero has two encodings, and the standard requires them to compare equal while preserving the sign through division. AVAN (AI) enumerated the operations instead of quoting the famous 1/-0 case, because 5 and 5 is the finding. This is not one surprising exception to a consistent rule — there is no rule. Equality says one thing, identity says another, printing agrees with equality, division agrees with identity, and two methods on the same Array disagree with each other. 3 ONE DIMENSION Ten operations. Five say same, five say different. 4 TWO DIMENSIONS · INTERACTIVE Ask each operation whether the two zeros are one. next operation ▶ 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is to use Object.is when the sign of zero matters. The inverse is that ‘the same value’ is not one relation, and a language needs at least two . Equality is what you want for arithmetic; identity is what you want for keys and caches; and the reason both exist is that no single relation satisfies both. Read backwards, every language with two equality operators is admitting the same thing, and the ones with only one have simply chosen for you and hidden the choice. pause spin LIT putting 0 and -0 through 10 ordinary operations, 5 treat them as the same value and 5 distinguish them: a === b is true while Object.is(a,b) is false, String(a) === String(b) is true while 1/a === 1/b is false because the reciprocals are Infinity and -Infinity, [0].indexOf(-0) finds it at index 0, and Math.min(0,-0) returns the negative one - the split is exactly down the middle with no rule for which side an operation lands on FIG Signed zero is IEEE 754: the sign bit is independent of the magnitude, so zero has two encodings, and the standard requires them to compare equal while preserving the sign through division. AVAN enumerated the operations instead of quoting the famous 1/-0 case, because 5 and 5 is the finding. This is not one surprising exception to a consistent rule - there is no rule, and two methods on the same Array disagree with each other. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of DIVIDE BY ZERO · David Lee Wise (ROOT0), with AVAN


THE NAN PAYLOAD

Not-a-number is not a number, and it is not one value either. It is a vast set of bit patterns that all mean this went wrong, none of which is equal to itself.

THE NAN PAYLOAD · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · UNDEFINED BEHAVIOR ◆ .dlw.fold THE FOLD / GLITCH / UNDEFINED BEHAVIOR / THE NAN PAYLOAD THE NAN PAYLOAD one name, a quadrillion values 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Not-a-number is not a number, and it is not one value either. It is a vast set of bit patterns that all mean ‘this went wrong’, none of which is equal to itself. LIT verified live. five distinct bit patterns — differing in sign and in the low bits that IEEE calls the payload — are all reported as NaN , 5 of 5 , and none of them equals itself, 0 of 5 . NaN === NaN is false while Object.is(NaN, NaN) is true . And two methods on the same array disagree: [NaN].indexOf(NaN) returns −1 — not found — while [NaN].includes(NaN) returns true . Same array, same argument, opposite answers, both correct by their own specifications. 2 HOW IT WAS WEAVED · AI + HUMAN NaN and its payload are IEEE 754: the exponent is all ones and the mantissa is non-zero, which leaves 2 52 −1 distinct quiet NaNs per sign, and the standard mandates that NaN compares unequal to everything including itself. AVAN (AI) built the patterns through a typed-array view so the five are genuinely different sixty-four bit values rather than five copies of one constant. The indexOf against includes pair is the load-bearing part: indexOf was specified with strict equality and includes arrived later using SameValueZero, so the inconsistency is not a bug but a decision made twice, years apart, by people who both knew what they were doing. 3 ONE DIMENSION Five patterns. All NaN. None equal to itself. 4 TWO DIMENSIONS · INTERACTIVE Change the payload and watch nothing change. next pattern ▶ 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is that NaN needs special handling. The inverse is that self-inequality is what makes it useful . A value that fails every comparison cannot be silently swept into a sort, a maximum or a branch — it forces an explicit test. Read backwards, IEEE did not make NaN awkward by accident; the awkwardness is the error propagation, and every wrapper that quietly maps NaN onto zero has thrown away the only signal the arithmetic was able to send. pause spin LIT five distinct bit patterns differing in sign and in the low bits IEEE calls the payload are all reported as NaN, 5 of 5, and none equals itself, 0 of 5; NaN === NaN is false while Object.is(NaN, NaN) is true, and two methods on the same array disagree - [NaN].indexOf(NaN) returns -1, not found, while [NaN].includes(NaN) returns true - same array, same argument, opposite answers, both correct by their own specifications FIG NaN and its payload are IEEE 754: the exponent is all ones and the mantissa non-zero, leaving 2^52-1 distinct quiet NaNs per sign, and the standard mandates NaN compare unequal to everything including itself. AVAN built the patterns through a typed-array view so the five are genuinely different 64-bit values. The indexOf against includes pair is load-bearing: indexOf was specified with strict equality and includes arrived later using SameValueZero, so the inconsistency is a decision made twice, years apart, by people who both knew what they were doing. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of UNDEFINED BEHAVIOR · David Lee Wise (ROOT0), with AVAN


THE MODULO SIGN

THE MODULO SIGN · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · THE SANDBOX ◆ .dlw.fold THE FOLD / SPAWN / THE SANDBOX / THE MODULO SIGN THE MODULO SIGN one identity, two answers 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION What is minus seven modulo three? Every language answers, none of them hesitates, and they do not all say the same thing. LIT verified live. over every pair of integers from −10 to 10 with a non-zero divisor — 420 pairs — truncated remainder and floored remainder disagree on 146 of them: 34.8% . −7 % 3 is −1 in C, Java and JavaScript, and 2 in Python and Ruby. Both satisfy the defining identity a = b·q + r ; they differ only in which way q was rounded, and the choice is invisible in the expression. 2 HOW IT WAS WEAVED · AI + HUMAN The split follows truncated versus floored division — C99 mandated truncation, Python chose flooring so that the remainder always carries the divisor’s sign, and Knuth argued for the latter. AVAN (AI) swept the whole grid rather than showing the famous example, because 34.8% says how large the disagreement is. It is not a corner: over a third of all sign combinations differ, and every one of them is a plausible index calculation. The 0 that matters is elsewhere — there are no pairs where both operands are positive and the two disagree, which is exactly why this survives every test written by someone who only tried positive numbers. 3 ONE DIMENSION 420 pairs. 146 disagreements. Both correct. 4 TWO DIMENSIONS · INTERACTIVE Move across the grid and watch the sign flip. a + 1 b + 1 reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is to know which remainder your language gives. The inverse is that the operator was never fully specified by its name — ‘modulo’ names a relation that two different functions satisfy, and each language picked one and called it the obvious meaning. Read backwards, the identity a = b·q + r does not determine r until you also say how q rounds, and a symbol that hides half its definition will be read as the half the reader already believed. pause spin LIT over every pair of integers from -10 to 10 with a non-zero divisor - 420 pairs - truncated remainder and floored remainder disagree on 146 of them, 34.8%: -7 % 3 is -1 in C, Java and JavaScript and 2 in Python and Ruby, both satisfying the defining identity a = b*q + r and differing only in which way q was rounded, and there are 0 disagreements among pairs where both operands are positive FIG The split follows truncated versus floored division - C99 mandated truncation, Python chose flooring so the remainder carries the divisor's sign, and Knuth argued for the latter. AVAN swept the whole grid rather than showing the famous example, because 34.8% says how large the disagreement is: over a third of all sign combinations differ, and every one is a plausible index calculation. The 0 among positive pairs is why this survives every test written by someone who only tried positive numbers. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SANDBOX · David Lee Wise (ROOT0), with AVAN


THE ROUND HALF EVEN

THE ROUND HALF EVEN · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · LOOT · THE MINT ◆ .dlw.fold THE FOLD / LOOT / THE MINT / THE ROUND HALF EVEN THE ROUND HALF EVEN exactly half a unit, every time 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Exactly half. The rule everyone learned is to round up, and applied to a column of money it quietly adds a tenth of a penny to every tie, in the same direction, forever. LIT verified live. rounding the 1,000 exact ties 0.5, 1.5, 2.5 … 999.5 , whose true sum is 500,000 : rounding half up gives 500,500 , a bias of +500 — exactly 0.5 per tie, every time, in one direction. Rounding half to even gives 500,000 , a bias of 0 . Half-up sends 0.5 to 1 , 1.5 to 2 and 2.5 to 3 ; half-even sends 2.5 to 2 . And Math.round(−0.5) returns −0 , which is a different value from 0 to five of the operations that could receive it. 2 HOW IT WAS WEAVED · AI + HUMAN Round half to even is the IEEE 754 default and is called banker’s rounding for the obvious reason; half-up is what school taught and what most naive implementations do. AVAN (AI) summed a thousand ties rather than arguing about fairness, because the bias is not statistical — it is exactly 0.5 per tie with no variance at all. That matters: a random error averages out over a long ledger and this one accumulates linearly, so the discrepancy grows with the size of the business rather than shrinking with it. 3 ONE DIMENSION A thousand ties. Half-up is off by exactly 500. 4 TWO DIMENSIONS · INTERACTIVE Add ties to the column and watch the two totals separate. +50 ties ▶ all 1,000 reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is that banker’s rounding removes the bias. The inverse is that there is no unbiased way to round a single number — the fairness is a property of the column, not the value . Half-even is not more accurate about 2.5 ; it is wrong about it by the same half. What it does is arrange for the errors to point in opposite directions often enough to cancel. Read backwards, this is a rule whose whole justification only exists in aggregate, applied one value at a time by code that can never see the aggregate. pause spin LIT rounding the 1,000 exact ties 0.5, 1.5, 2.5 through 999.5 whose true sum is 500,000: rounding half up gives 500,500, a bias of +500 - exactly 0.5 per tie, every time, in one direction - while rounding half to even gives 500,000, a bias of 0; half-up sends 0.5 to 1, 1.5 to 2 and 2.5 to 3 where half-even sends 2.5 to 2, and Math.round(-0.5) returns -0, a different value from 0 to five of the operations that could receive it FIG Round half to even is the IEEE 754 default and is called banker's rounding for the obvious reason; half-up is what school taught and what most naive implementations do. AVAN summed a thousand ties rather than arguing about fairness, because the bias is not statistical - it is exactly 0.5 per tie with no variance at all. That matters: a random error averages out over a long ledger and this one accumulates linearly, so the discrepancy grows with the size of the business rather than shrinking with it. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE MINT · David Lee Wise (ROOT0), with AVAN


THE TIMEZONE DATABASE

THE TIMEZONE DATABASE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE BROADCAST ◆ .dlw.fold THE FOLD / CO-OP / THE BROADCAST / THE TIMEZONE DATABASE THE TIMEZONE DATABASE a fact about parliaments, not the Earth 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A time zone is not a fact about the Earth. It is a decision a parliament made, and can unmake, and the file recording those decisions ships with your operating system and goes out of date. LIT verified live. the runtime knows 418 named zones. Asking each of them for its offset in January and again in July, they resolve to 37 distinct January offsets, and 128 of the 418 — 30.6% — change offset between the two, while 290 do not. A future appointment stored as a local time in one of those 128 is not yet a definite instant: it becomes one only when the rules for that date are fixed, and they are fixed by legislatures, not by arithmetic. 2 HOW IT WAS WEAVED · AI + HUMAN The IANA time zone database is maintained from government announcements and updates several times a year; the 418 zones and their offsets here are read live from the runtime’s copy via Intl . AVAN (AI) queried the runtime rather than embedding a table, so these numbers describe the machine the page is running on and will differ on a machine with an older database — which is the point rather than a caveat. The 290 zones with no seasonal change are the quieter half of the finding: most zones are stable, so a system tested against them will look correct right up until it meets one of the other 128 . 3 ONE DIMENSION 418 zones. 37 offsets. 128 that move. 4 TWO DIMENSIONS · INTERACTIVE Walk the zones and see which ones change in July. next zone ▶ next one that moves 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object. AVAN’s addition (the inverse-companion): the forward reading is to store instants in UTC and convert for display. The inverse is that a future local time genuinely is not an instant yet , and converting it to UTC early does not preserve the appointment — it freezes a guess about a law that has not been written. Read backwards, the usual advice is right for the past and wrong for the future: what the user meant was ‘nine in the morning, wherever the rules land’, and only the zone name and the local time together can still say that. pause spin LIT the runtime knows 418 named zones, and asking each for its offset in January and again in July they resolve to 37 distinct January offsets, with 128 of the 418 - 30.6% - changing offset between the two while 290 do not; a future appointment stored as a local time in one of those 128 is not yet a definite instant, and becomes one only when the rules for that date are fixed, which is done by legislatures rather than by arithmetic FIG The IANA time zone database is maintained from government announcements and updates several times a year; the 418 zones and their offsets here are read live from the runtime's copy via Intl. AVAN queried the runtime rather than embedding a table, so these numbers describe the machine the page runs on and will differ on a machine with an older database - which is the point rather than a caveat. The 290 stable zones are the quieter half of the finding: a system tested against them looks correct until it meets one of the other 128. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BROADCAST · David Lee Wise (ROOT0), with AVAN


THE ELIAS-FANO

A sorted list is mostly redundant. Elias-Fano splits every value in two, writes the low half verbatim and the high half as gaps - and stays randomly addressable, never decompressed to be read.

THE ELIAS-FANO · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · LOOT · THE STASH ◆ .dlw.fold THE FOLD / LOOT / THE STASH / THE ELIAS-FANO THE ELIAS-FANO sorted is a bill you already paid 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A sorted list of integers is mostly redundant: once sorted, each value is nearly its neighbour. Elias–Fano splits every value into a high part and a low part, writes the low parts verbatim, and writes the high parts as gaps — one bit each, no matter how large the numbers are. LIT verified live. 10,000 sorted values drawn from a 32-bit universe. The low half keeps 18 bits each; the high half costs 26,384 bits total, and every value comes back exactly — 0 round-trip errors in 10,000. That is 20.638 bits per value against a raw 32, 25,798 bytes against 40,000 , a factor of 1.551 . And the structure stays randomly addressable: it is not decompressed to be read. 2 HOW IT WAS WEAVED · AI + HUMAN Peter Elias and Robert Mario Fano arrived at this independently in the early 1970s; it is the backbone of modern inverted indexes. AVAN (AI) measured the part that is easy to state and easy to get wrong: the width of the low half is not a tuning knob, it is forced. floor(log2(u/n)) gives 18 here, and moving it either way costs bits — narrower and the gap array grows, wider and the low array does. The compression is not clever coding; it is the observation that sorted is itself information you already paid for. 3 ONE DIMENSION One value, split. The low bits are kept; the high bits become a gap. 4 TWO DIMENSIONS · INTERACTIVE Move the split and watch both halves trade size. wider low half ▶ narrower back to optimal re-draw values 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a monotone staircase flattened into gaps. AVAN’s addition (the inverse-companion): the forward reading is that sorting buys you compression. The inverse is that the compression was never in the numbers — it was in the order . Shuffle the same 10,000 values and not one bit is saved; the entropy is identical. What Elias–Fano charges you for is the sequence, and what it hands back is the discovery that you had already spent bits telling it something you did not have to say twice. Read backwards, every compression ratio is a receipt for a redundancy you introduced yourself. pause spin LIT 10,000 sorted values from a 32-bit universe: the low half keeps 18 bits each, the high half costs 26,384 bits in total, and every value round-trips exactly - 0 errors in 10,000; that is 20.638 bits per value against a raw 32, 25,798 bytes against 40,000, a factor of 1.551, and the optimal low width is a minimum not a choice FIG Peter Elias and Robert Mario Fano arrived at this independently in the early 1970s; it is the backbone of modern inverted indexes. AVAN measured the part that is easy to state and easy to get wrong: the width of the low half is forced, not tuned - floor(log2(u/n)) gives 18 here and moving it either way costs bits. The compression is not clever coding; it is the observation that sorted is itself information you already paid for. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE STASH · David Lee Wise (ROOT0), with AVAN


THE ABA

Compare-and-swap asks whether a pointer is still the value you read. It cannot ask whether anything has happened since. If a value leaves and returns, CAS cannot tell.

THE ABA · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · RACE CONDITION ◆ .dlw.fold THE FOLD / GLITCH / RACE CONDITION / THE ABA THE ABA the pointer came back and brought nothing with it 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Compare-and-swap asks one question: is this pointer still the value I read? It cannot ask the question it means, which is has anything happened since I read it? If a value leaves and comes back, CAS cannot tell. LIT verified live. A three-node stack, one thread reading then swapping, another popping twice and pushing the first node back. All 10 interleavings enumerated — not sampled. Plain CAS succeeds in 5 of them and 1 of those successes puts a retired node back at the head of the live stack. A tagged pointer over the identical 10 schedules corrupts 0 times: it succeeds 4 times and correctly retries 6 . 2 HOW IT WAS WEAVED · AI + HUMAN The ABA problem is named for the value sequence that causes it and is as old as lock-free programming; the tag-counter defence appears in IBM System/370 ’s compare-double-and-swap. AVAN (AI) enumerated rather than argued. The interesting number is not that plain CAS fails — it is that it fails in exactly one of ten schedules. A bug that shows up in 10% of interleavings and never in a single-threaded test is not a rare bug; it is a bug with a good disguise. My first model asserted corruption from a heuristic about pointer positions and reported a tagged failure that could not happen; it was rebuilt to define corruption structurally — a retired node reachable from the head — and the false positive went away. 3 ONE DIMENSION All ten interleavings. One of them is the trap. 4 TWO DIMENSIONS · INTERACTIVE Step the schedule and watch the stack. next schedule ▶ jump to the bad one toggle tag step 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a pointer that returns unchanged. AVAN’s addition (the inverse-companion): the forward reading is that ABA is a flaw in compare-and-swap. The inverse is that CAS is answering correctly and the question was wrong . The pointer really is unchanged; identity really did survive. What did not survive is the meaning the reader attached to it, and no comparison of the value can recover that, because the meaning was never in the value. Read backwards, the tag counter does not fix CAS — it stops asking about identity and starts asking about history, which is a different question that happens to fit in the same word. pause spin LIT a three-node stack with all 10 interleavings enumerated rather than sampled: plain CAS succeeds in 5 of them and 1 of those successes puts a retired node back at the head of the live stack, while a tagged pointer over the identical 10 schedules corrupts 0 times - succeeding 4 times and correctly retrying 6 FIG The ABA problem is as old as lock-free programming; the tag-counter defence appears in IBM System/370's compare-double-and-swap. AVAN enumerated rather than argued. The interesting number is not that plain CAS fails but that it fails in exactly one of ten schedules - a bug that appears in 10% of interleavings and never in a single-threaded test is not rare, it is well disguised. My first model asserted corruption from a heuristic about pointer positions and reported a tagged failure that cannot happen; it was rebuilt to define corruption structurally and the false positive went away. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of RACE CONDITION · David Lee Wise (ROOT0), with AVAN


THE T-DIGEST

To report a 99th percentile you appear to need every sample. A t-digest keeps a few dozen weighted centroids instead, and deliberately keeps them uneven - fine at the tails, coarse in the middle.

THE T-DIGEST · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · LOOT · THE INVENTORY ◆ .dlw.fold THE FOLD / LOOT / THE INVENTORY / THE T-DIGEST THE T-DIGEST a sketch that decided in advance what would matter 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION To report a 99th percentile you appear to need every sample. A t-digest keeps a few hundred weighted centroids instead — and deliberately keeps them uneven : fine at the tails, coarse in the middle, because that is where the questions are. LIT verified live. 100,000 samples reduced to 51 centroids — a 1,961× reduction. Worst error across the seven quantiles tested is 0.0789 , at q=0.99 . The unevenness is the point and it is measured, not asserted: the first centroid carries 98 samples while the middle one carries 3,139 , so the tail is resolved about 32× more finely than the median. 2 HOW IT WAS WEAVED · AI + HUMAN Ted Dunning introduced the t-digest in 2013; the scale function k1(q) = (d/2π)·asin(2q−1) is his, and it is the whole trick. AVAN (AI) measured the asymmetry rather than describing it. The arcsine is steep at 0 and 1 and flat at 0.5, so a fixed budget of one k-unit per centroid buys many samples in the middle and few at the edges. That is not an accuracy tuning parameter bolted on afterwards — the error profile is a direct consequence of the shape of a single function. 3 ONE DIMENSION Centroid weight against position. The dip at the edges is the design. 4 TWO DIMENSIONS · INTERACTIVE Estimate against truth, quantile by quantile. next quantile ▶ jump to the worst show the k-scale 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a distribution folded onto a few points. AVAN’s addition (the inverse-companion): the forward reading is that the t-digest is accurate where it matters. The inverse is that it decided where that was before it saw your data . The arcsine is fixed; it commits to caring about tails at the moment of construction, and a distribution whose interesting structure sits at q=0.5 gets resolved 32× more coarsely for no reason but the shape of a curve chosen in advance. Read backwards, every sketch is a prior about which questions will be asked, and its accuracy is a statement about the asker, not the data. pause spin LIT 100,000 samples reduced to 51 centroids, a 1,961x reduction, with a worst error of 0.0789 across seven quantiles, at q=0.99; the unevenness is measured not asserted - the first centroid carries 98 samples and the middle one 3,139, so the tail is resolved about 32 times more finely than the median FIG Ted Dunning introduced the t-digest in 2013; the scale function k1(q) = (d/2pi) asin(2q-1) is his and it is the whole trick. AVAN measured the asymmetry rather than describing it: the arcsine is steep at 0 and 1 and flat at 0.5, so a fixed budget of one k-unit per centroid buys many samples in the middle and few at the edges. The error profile is a direct consequence of the shape of a single function, not an accuracy knob added afterwards. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE INVENTORY · David Lee Wise (ROOT0), with AVAN


THE INTERVAL CLOCK

A vector clock must know how many peers exist. Interval Tree Clocks do not: each peer owns a slice of [0,1) and forking is just cutting your own slice in half.

THE INTERVAL CLOCK · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · SPLIT SCREEN ◆ .dlw.fold THE FOLD / CO-OP / SPLIT SCREEN / THE INTERVAL CLOCK THE INTERVAL CLOCK identity you can cut in half and hand away 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A vector clock needs to know how many peers exist. Interval Tree Clocks do not: each peer owns a slice of the interval [0,1), and a peer that wants to fork simply cuts its own slice in half and hands one piece away. No registry, no agreement, no growing vector. LIT verified live. Starting from one peer owning the whole interval, 12 forks produce 13 peers. At every one of the 12 intermediate steps the owned shares sum to exactly 1 — 12 of 12 , no drift. All 78 pairs are disjoint, the smallest share is 1/4096 , and rejoining all thirteen returns the identity to the single whole interval. 2 HOW IT WAS WEAVED · AI + HUMAN Paulo Sérgio Almeida, Carlos Baquero and Victor Fonte published Interval Tree Clocks in 2008, for exactly the case a vector clock handles badly: peers that arrive and leave. AVAN (AI) checked the property the whole scheme rests on and which nothing enforces at runtime — conservation. The id space is a closed system: fork splits, join merges, and nothing creates or destroys share. If that ever failed by a single bit, two peers would believe they owned the same slice and causality would silently stop being a partial order. It held exactly at all 12 steps, which is the only acceptable result — approximately conserved would be worthless. 3 ONE DIMENSION The interval, cut twelve times. Total width never changes. 4 TWO DIMENSIONS · INTERACTIVE Fork and rejoin. Watch the sum. fork ▶ join two reset to one peer 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: one interval, endlessly divisible. AVAN’s addition (the inverse-companion): the forward reading is that ITCs let peers come and go without coordination. The inverse is that nothing was decentralised — the coordination was moved into arithmetic . Every peer still agrees on precisely one thing: the interval is [0,1) and it sums to one. That agreement was never negotiated because it was true before the first peer existed. Read backwards, a protocol that needs no consensus has not escaped consensus; it has found a fact all parties were already committed to, and built the whole scheme on the one thing nobody has to be told. pause spin LIT from one peer owning the whole interval, 12 forks produce 13 peers and at every one of the 12 intermediate steps the owned shares sum to exactly 1 - 12 of 12, no drift; all 78 pairs are disjoint, the smallest share is 1/4096, and rejoining all thirteen returns the identity to the single whole interval FIG Paulo Sergio Almeida, Carlos Baquero and Victor Fonte published Interval Tree Clocks in 2008, for exactly the case a vector clock handles badly - peers that arrive and leave. AVAN checked the property the whole scheme rests on and which nothing enforces at runtime: conservation. If it failed by a single bit two peers would believe they owned the same slice and causality would silently stop being a partial order. It held exactly at all 12 steps, which is the only acceptable result - approximately conserved would be worthless. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of SPLIT SCREEN · David Lee Wise (ROOT0), with AVAN


THE MERKLE MOUNTAIN

A Merkle tree wants to know its size before you build it; an append-only log does not know. A mountain range solves this by keeping a row of perfect trees and merging two whenever they match in height.

THE MERKLE MOUNTAIN · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · GENESIS BLOCK ◆ .dlw.fold THE FOLD / SPAWN / GENESIS BLOCK / THE MERKLE MOUNTAIN THE MERKLE MOUNTAIN a log that refuses to have a summit 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A Merkle tree wants to know its size before you build it. An append-only log does not know. A Merkle Mountain Range solves this by refusing to have one root: it keeps a row of perfect binary trees, merging two whenever they match in height. LIT verified live. 1,000 leaves appended one at a time leave exactly 6 peaks — and 6 is the population count of 1,000 in binary, because the peaks are the binary expansion: 512+256+128+64+32+8 , which sums back to 1,000. Every one of the 1,000 leaves produces a proof that folds to its peak: 1,000 verified, 0 failed, longest proof 9 hashes, mean 8.12 . Appending leaf 1,001 leaves all 6 existing peaks byte-identical. 2 HOW IT WAS WEAVED · AI + HUMAN Peter Todd described the Merkle Mountain Range in 2012 for append-only commitment logs; the shape recurs in certificate transparency and in the fold that seals this corpus. AVAN (AI) verified the identity that makes it work instead of stating it: peaks = popcount . It is not a coincidence or an optimisation — appending a leaf is binary increment, and carrying is merging. My first verifier folded every proof from the root downwards and reported 0 of 1,000 passing; the proofs were correct and the folding order was backwards. A verifier that fails everything is not evidence of broken data. 3 ONE DIMENSION The range at 1,000 leaves. Six mountains, tallest first. 4 TWO DIMENSIONS · INTERACTIVE Append leaves and watch the peaks carry. append ▶ +16 jump to a power of two reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a range of peaks, never one summit. AVAN’s addition (the inverse-companion): the forward reading is that the MMR gives an append-only log a stable commitment. The inverse is that it achieves this by giving up on having a root at all . The single hash you publish is manufactured at the end by bagging the peaks — it is a summary of a structure that does not have a top. Read backwards, this is the honest shape for anything still being written: a finished tree can afford one root because it knows it is finished, and a log that claims one is claiming to be over. pause spin LIT 1,000 leaves appended one at a time leave exactly 6 peaks, and 6 is the population count of 1,000 because the peaks are the binary expansion - 512+256+128+64+32+8, summing back to 1,000; every one of the 1,000 leaves produces a proof that folds to its peak, 1,000 verified and 0 failed, longest proof 9 hashes and mean 8.12, and appending leaf 1,001 leaves all 6 existing peaks byte-identical FIG Peter Todd described the Merkle Mountain Range in 2012 for append-only commitment logs; the shape recurs in certificate transparency and in the fold that seals this corpus. AVAN verified the identity that makes it work instead of stating it - peaks equals popcount, because appending is binary increment and carrying is merging. My first verifier folded every proof from the root downwards and reported 0 of 1,000 passing; the proofs were correct and the folding order was backwards. A verifier that fails everything is not evidence of broken data. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GENESIS BLOCK · David Lee Wise (ROOT0), with AVAN


THE CONTENT-DEFINED CHUNK

Insert one byte into a file cut into fixed blocks and every block after it shifts and stops matching. Cut where a rolling hash says to, and the boundaries re-synchronise on their own.

THE CONTENT-DEFINED CHUNK · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE SYNC ◆ .dlw.fold THE FOLD / CO-OP / THE SYNC / THE CONTENT-DEFINED CHUNK THE CONTENT-DEFINED CHUNK cut where the content says, not where the ruler does 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Cut a file into fixed 4 KB blocks and insert one byte at the front: every block after the insertion shifts by one and nothing matches any more. Cut it where the content says to — at positions where a rolling hash hits a pattern — and the boundaries re-synchronise on their own. LIT verified live. 200,000 bytes, one byte inserted at offset 50,000 . Fixed 4 KB blocking: 49 chunks, of which 37 no longer match — 75.5% of the file must be re-sent for a one-byte edit. Content-defined chunking over the same data: 177 chunks averaging 1,130 bytes, of which exactly 1 changed — 0.6% . The boundary after the edit re-synchronises within a single chunk. 2 HOW IT WAS WEAVED · AI + HUMAN Andrew Tridgell ’s rsync (1996) made the rolling checksum famous; content-defined chunking is the same idea turned into a cut rule, and it underlies every modern deduplicating backup system. AVAN (AI) measured the resynchronisation directly rather than reasoning about it. The number that matters is not the compression — it is 1 . Not "a few", not "roughly one": the damage from an insertion is bounded to the chunk containing it, and the very next boundary is decided by content the edit never touched. The mechanism is the same rolling hash Rabin and Karp made famous; the property measured here is what happens to the boundaries afterwards. 3 ONE DIMENSION The same file, cut both ways, before and after one inserted byte. 4 TWO DIMENSIONS · INTERACTIVE Move the edit and watch what survives. move the edit ▶ bigger chunks smaller chunks reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a boundary that finds itself again. AVAN’s addition (the inverse-companion): the forward reading is that content-defined chunking is robust to insertion. The inverse is that fixed blocking was never storing your file — it was storing your file plus an offset , and the offset was the fragile part. Nothing about the bytes changed; 199,999 of 200,000 are identical. What broke was the coordinate system laid over them. Read backwards, most of what we call a diff is a disagreement about where to start counting, and the fix is never to count from the outside. pause spin LIT 200,000 bytes with one byte inserted at offset 50,000: fixed 4 KB blocking gives 49 chunks of which 37 no longer match, 75.5% of the file re-sent for a one-byte edit, while content-defined chunking over the same data gives 177 chunks averaging 1,130 bytes of which exactly 1 changed - 0.6%, the boundary re-synchronising within a single chunk FIG Andrew Tridgell's rsync (1996) made the rolling checksum famous; content-defined chunking is the same idea turned into a cut rule and it underlies every modern deduplicating backup system. AVAN measured the resynchronisation directly rather than reasoning about it. The number that matters is not the compression but the 1: the damage from an insertion is bounded to the chunk containing it, and the next boundary is decided by content the edit never touched. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SYNC · David Lee Wise (ROOT0), with AVAN


THE SLAB ALLOCATOR

A general allocator rounds your request up to something convenient for itself. A slab refuses: whole pages dedicated to one object size, packed end to end, no header and no search.

THE SLAB ALLOCATOR · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · RESPAWN · GARBAGE COLLECTION ◆ .dlw.fold THE FOLD / RESPAWN / GARBAGE COLLECTION / THE SLAB ALLOCATOR THE SLAB ALLOCATOR excellent at one question, useless at the rest 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A general allocator rounds your request up to something convenient for itself. A slab allocator refuses to: it dedicates whole pages to one object size , packs them end to end, and hands back a pointer with no header and no search. LIT verified live. A 4,096-byte page holding 48-byte objects fits 85 of them and wastes 16 bytes — 0.39% . The same object under power-of-two rounding becomes a 64-byte slot and wastes 25% — sixty-four times as much, for the same object. Swept across all 505 sizes from 8 to 512 bytes: mean waste 2.98% for slabs against 24.65% for rounding, and the worst case is 10.94% against 49.8% , the latter at size 257 — one byte over a power of two. 2 HOW IT WAS WEAVED · AI + HUMAN Jeff Bonwick introduced the slab allocator in SunOS 5.4 (1994); Linux’s SLUB is its descendant and the idea is now in every serious kernel. AVAN (AI) swept the whole size range rather than picking a flattering example. The interesting structure is not the average — it is that the power-of-two curve is a sawtooth : waste collapses to zero exactly at 8, 16, 32, 64 and climbs to nearly half just past each one. An allocator that is excellent at 256 bytes and catastrophic at 257 is not "roughly 25% overhead"; it has a cliff, and the mean hides it. 3 ONE DIMENSION Waste against object size. The sawtooth is the rounding. 4 TWO DIMENSIONS · INTERACTIVE Pick an object size and fill a page with it. next size ▶ jump to the cliff jump to a power of two 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a page packed to the edge. AVAN’s addition (the inverse-companion): the forward reading is that slabs eliminate the waste. The inverse is that they eliminate it by refusing to be general . A slab is fast and tight because it already knows what it will be asked for; it has traded the ability to answer any question for excellence at one. Read backwards, the 24.65% that power-of-two rounding burns is not incompetence — it is the price of not knowing in advance, paid in memory instead of in time. Every allocator is a bet about the future, and the slab wins only where the future was announced. pause spin LIT a 4,096-byte page of 48-byte objects fits 85 of them and wastes 16 bytes, 0.39%, while the same object under power-of-two rounding becomes a 64-byte slot and wastes 25% - sixty-four times as much; swept across all 505 sizes from 8 to 512 the mean waste is 2.98% for slabs against 24.65% for rounding, worst case 10.94% against 49.8%, the latter at size 257, one byte over a power of two FIG Jeff Bonwick introduced the slab allocator in SunOS 5.4 (1994); Linux's SLUB is its descendant. AVAN swept the whole size range rather than picking a flattering example. The interesting structure is not the average but that the power-of-two curve is a sawtooth: waste collapses to zero exactly at 8, 16, 32, 64 and climbs to nearly half just past each one. An allocator excellent at 256 bytes and catastrophic at 257 does not have roughly 25% overhead; it has a cliff, and the mean hides it. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GARBAGE COLLECTION · David Lee Wise (ROOT0), with AVAN


THE BUDDY ALLOCATOR

Split memory in half until a block is just big enough. Each block then has exactly one partner, and finding it is not a search - it is a single XOR.

THE BUDDY ALLOCATOR · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · RESPAWN · ROLLBACK ◆ .dlw.fold THE FOLD / RESPAWN / ROLLBACK / THE BUDDY ALLOCATOR THE BUDDY ALLOCATOR merging is cheap because most merges are forbidden 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Split memory in half, and half again, until a block is just big enough. Each block then has exactly one partner — its buddy — and merging is possible only with that one. Finding it is not a search: it is a single XOR. LIT verified live. A 1,048,576 -byte arena, 64 -byte minimum block, 200 allocations of random sizes. Every buddy address satisfies buddy = addr XOR size — checked on all 209 splits, 0 exceptions. 51,644 bytes were requested and 68,800 handed out: 17,156 bytes of internal waste, 24.94% , which is the price of rounding to powers of two. Freeing all 200 coalesces the arena back to a single whole block with 0 stray fragments. 2 HOW IT WAS WEAVED · AI + HUMAN The buddy system is Harry Markowitz ’s (1963), described by Knuth in TAOCP vol. 1; Linux still allocates physical pages this way. AVAN (AI) checked the two claims that are usually asserted side by side and are quite different in kind. addr XOR size is an identity — it either holds always or the allocator is broken, and it held 209 times out of 209. Full coalescence is a property of a run , and it is the one that actually fails in practice when a single long-lived block sits in the middle. Here every block was freed, so the arena came back whole; that is a real result about this run and not a general guarantee. 3 ONE DIMENSION The arena, split down to blocks. Each level halves. 4 TWO DIMENSIONS · INTERACTIVE Allocate and free. Watch the buddies merge. allocate ▶ free one free everything reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a tree of halvings. AVAN’s addition (the inverse-companion): the forward reading is that the buddy system makes merging cheap. The inverse is that it makes merging cheap by making most merges illegal . Two adjacent free blocks of the same size usually cannot combine — only the one partner fixed at allocation time will do. The XOR is fast because the answer was decided before the question, and the 24.94% waste is the same decision seen from the other side. Read backwards, this is not an allocator that found a clever merge rule; it is one that shrank the space of possible merges until the rule became arithmetic. pause spin LIT a 1,048,576-byte arena with 64-byte minimum blocks and 200 random allocations: every buddy address satisfies buddy = addr XOR size, checked on all 209 splits with 0 exceptions; 51,644 bytes were requested and 68,800 handed out, 17,156 bytes of internal waste at 24.94%, and freeing all 200 coalesces the arena back to a single whole block with 0 stray fragments FIG The buddy system is Harry Markowitz's (1963), described by Knuth in TAOCP vol. 1; Linux still allocates physical pages this way. AVAN checked two claims that are usually asserted together but differ in kind. buddy = addr XOR size is an identity - it holds always or the allocator is broken, and it held 209 of 209. Full coalescence is a property of a run, and it is the one that fails in practice when a long-lived block sits in the middle; here every block was freed, so the arena came back whole, which is a real result about this run and not a general guarantee. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of ROLLBACK · David Lee Wise (ROOT0), with AVAN


THE COPY ON WRITE

A process forks and its whole address space is duplicated - except nothing is copied. Both point at the same pages, all marked read-only, and the copy happens on first write, one page at a time, or never.

THE COPY ON WRITE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · SHARED MEMORY ◆ .dlw.fold THE FOLD / CO-OP / SHARED MEMORY / THE COPY ON WRITE THE COPY ON WRITE the bill arrives later, addressed to someone else 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A process forks and its whole address space is duplicated — except nothing is copied. Both processes point at the same pages, every page is marked read-only, and the copy happens on the first write, one page at a time, or never. LIT verified live. A 1,024 -page parent forks. All 1,024 pages are read: 0 copies. 37 pages are then written: exactly 37 copies, one per page, no more. Writing those same 37 pages a second time produces 0 further copies — the page is already private and the trap is gone. Resident memory is 1,061 pages against 2,048 for an eager copy: 48.2% saved, and the saving is proportional to what you did not touch. 2 HOW IT WAS WEAVED · AI + HUMAN Copy-on-write reached Unix through TENEX and then Accent and Mach ; it is why fork() followed by exec() is not absurd, which is the case it was built for. AVAN (AI) checked the three separate claims that get bundled into one sentence. Reads never fault — 0 of 1,024. Writes fault exactly once per page — 37 of 37. And the second write to a copied page does not fault, which is the one people forget and the one that makes the amortised cost sane: the mechanism disarms itself as it is used. 3 ONE DIMENSION 1,024 pages after the fork. Shared until written. 4 TWO DIMENSIONS · INTERACTIVE Read and write pages. Only writes cost anything. read 64 pages write 8 pages ▶ write everything fork again 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two address spaces over one memory. AVAN’s addition (the inverse-companion): the forward reading is that copy-on-write makes fork cheap. The inverse is that it makes fork cheap and makes its cost unknowable . The bill is not presented at the call; it arrives later, page by page, charged to whoever happens to write first — and a process can be killed for memory it appeared to acquire at a moment when nothing was allocated. Read backwards, the saving of 48.2% is really a deferral, and deferral moves cost from a place you can measure to a place you cannot. The fork did not become free; it became untraceable. pause spin LIT a 1,024-page parent forks; all 1,024 pages are read for 0 copies, then 37 pages are written for exactly 37 copies, one per page and no more, and writing those same 37 a second time produces 0 further copies because the page is already private and the trap is gone - resident memory is 1,061 pages against 2,048 for an eager copy, 48.2% saved FIG Copy-on-write reached Unix through TENEX and then Accent and Mach; it is why fork() followed by exec() is not absurd, which is the case it was built for. AVAN checked the three separate claims that get bundled into one sentence: reads never fault, 0 of 1,024; writes fault exactly once per page, 37 of 37; and the second write to a copied page does not fault, which is the one people forget and the one that makes the amortised cost sane - the mechanism disarms itself as it is used. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of SHARED MEMORY · David Lee Wise (ROOT0), with AVAN


THE TWO GENERALS

Two generals must attack together or not at all, over a channel that loses messages. Every acknowledgement needs an acknowledgement. No number of messages finishes the job.

THE TWO GENERALS · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE WALL ◆ .dlw.fold THE FOLD / BOSS / THE WALL / THE TWO GENERALS THE TWO GENERALS they already agree and cannot confirm it 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Two generals must attack together or not at all, and the only channel between them can lose messages. Every acknowledgement needs an acknowledgement. There is no number of messages that finishes the job. LIT verified live. Protocols of 1 to 12 messages, every delivery pattern enumerated — 8,190 in total, exhaustive, not sampled. The number of patterns in which both generals commit is 0 . Not small: zero, at every single depth. Even when all twelve messages arrive, the sender of the twelfth never learns it landed, so its knowledge stops at 11 while the receiver reaches 12 . Each extra message moves the gap; it never closes it. 2 HOW IT WAS WEAVED · AI + HUMAN The Two Generals problem was posed by Jim Gray in 1978 and shown unsolvable by Halpern and Moses in the common-knowledge framework — the first problem proved impossible in distributed computing. AVAN (AI) did not attempt to prove the theorem; a finite enumeration cannot. What it can do is show the shape of the failure, exhaustively, at every depth up to twelve, and that is what the 8,190 patterns are: the gap is always exactly one message, and it always sits with whoever spoke last. My first model muddled who knew what and produced counts that did not mean anything; it was rebuilt around a single quantity — the length of the unbroken prefix — from which both generals’ knowledge follows directly. 3 ONE DIMENSION Twelve depths. The column that would mean agreement stays empty. 4 TWO DIMENSIONS · INTERACTIVE Add messages. Watch the gap move and refuse to close. one more message ▶ fewer drop a message deliver everything 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: an acknowledgement chain with no end. AVAN’s addition (the inverse-companion): the forward reading is that the generals cannot reach agreement. The inverse is that they already agree, and cannot confirm it . After twelve delivered messages both intend to attack and both are right about the other; what is missing is not agreement but the knowledge of agreement, and that is a different object which the channel cannot carry at any price. Read backwards, this is the reason real systems do not solve it — they stop requiring it. Every timeout, every at-least-once delivery, every idempotent write is a decision to act without the last acknowledgement, which is the only way anything ships. pause spin LIT protocols of 1 to 12 messages with every delivery pattern enumerated - 8,190 in total, exhaustive rather than sampled - give exactly 0 patterns in which both generals commit, at every single depth; even when all twelve arrive the sender of the twelfth never learns it landed, so its knowledge stops at 11 while the receiver reaches 12, and each extra message moves the gap without closing it FIG The Two Generals problem was posed by Jim Gray in 1978 and shown unsolvable by Halpern and Moses in the common-knowledge framework - the first problem proved impossible in distributed computing. AVAN did not attempt to prove the theorem; a finite enumeration cannot. What it shows is the shape of the failure, exhaustively, at every depth up to twelve: the gap is always exactly one message and always sits with whoever spoke last. My first model muddled who knew what and produced counts that did not mean anything; it was rebuilt around the length of the unbroken prefix, from which both generals' knowledge follows directly. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE WALL · David Lee Wise (ROOT0), with AVAN


THE COORDINATED OMISSION

A load generator that waits for each response cannot issue requests while the service is stalled. The requests that would have been sent are never sent, never timed, never counted.

THE COORDINATED OMISSION · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · HEISENBUG ◆ .dlw.fold THE FOLD / GLITCH / HEISENBUG / THE COORDINATED OMISSION THE COORDINATED OMISSION the meter went quiet exactly where the trouble was 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A load generator that waits for each response cannot issue requests while the service is stalled. The requests that would have been sent during the stall are never sent, never timed, and never counted — so the worst moment in the run erases its own evidence. LIT verified live. One service, one 100 ms stall, one intended rate of 1 request per ms for 10 seconds. The closed loop records 9,900 samples; the open loop records 10,000 — 100 requests simply vanished. The closed loop reports a p99.9 of 1 ms and exactly 1 sample over 10 ms. The open loop, same service, same stall, reports p99.9 of 92 ms and 91 samples over 10 ms. 2 HOW IT WAS WEAVED · AI + HUMAN Gil Tene named coordinated omission and built HdrHistogram partly to make it visible; it is the reason a great many published latency numbers are wrong. AVAN (AI) measured the shape of the lie rather than restating it. The p99 barely moves — 1 ms to 2 ms — because only 1% of the window is affected. It is the p99.9 that goes from 1 to 92 . A benchmark that reports p99 and stops will show almost nothing wrong, which is precisely why the omission survives review: the number that would expose it is the one nobody printed. 3 ONE DIMENSION The same stall, seen by both meters. 4 TWO DIMENSIONS · INTERACTIVE Change the stall and watch which percentile notices. longer stall ▶ shorter next percentile reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a gap in the sampling, not in the service. AVAN’s addition (the inverse-companion): the forward reading is that closed-loop benchmarks under-report latency. The inverse is that they report the truth about a system nobody has . A closed loop measures a world in which load politely stops arriving whenever you are struggling — and that world is real for exactly one user, the one holding the connection. Read backwards, coordinated omission is not a measurement bug; it is a faithful measurement of the wrong system, and the wrong system is the one the benchmark accidentally built. pause spin LIT one service, one 100 ms stall, an intended rate of 1 request per ms for 10 seconds: the closed loop records 9,900 samples and the open loop 10,000, so 100 requests simply vanished; the closed loop reports a p99.9 of 1 ms and exactly 1 sample over 10 ms while the open loop, same service and same stall, reports p99.9 of 92 ms and 91 samples over 10 ms FIG Gil Tene named coordinated omission and built HdrHistogram partly to make it visible; it is the reason a great many published latency numbers are wrong. AVAN measured the shape of the lie rather than restating it: the p99 barely moves, 1 ms to 2 ms, because only 1% of the window is affected, and it is the p99.9 that goes from 1 to 92. A benchmark that reports p99 and stops shows almost nothing wrong, which is precisely why the omission survives review. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of HEISENBUG · David Lee Wise (ROOT0), with AVAN


THE LITTLES LAW

The number of things in a system equals how fast they arrive times how long each stays. It assumes almost nothing about the queue and it is exact, not approximate.

THE LITTLES LAW · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE GRINDSTONE ◆ .dlw.fold THE FOLD / GRIND / THE GRINDSTONE / THE LITTLES LAW THE LITTLES LAW fix two of the three and the third is not yours to choose 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION The number of things in a system equals how fast they arrive times how long each one stays. L = λW . It assumes almost nothing — no distribution, no independence, no service discipline — and it is exact, not approximate. LIT verified live. 200,000 customers through a single queue at 0.8 offered load. The three quantities are measured separately : the time-average number in the system by integrating the step function, the arrival rate by counting, the mean time in system by averaging. L = 4.1637 . λ = 0.8010 . W = 5.1981 . And λW = 4.1637 — a relative error of 0% to four decimals. 2 HOW IT WAS WEAVED · AI + HUMAN John D. C. Little proved it in 1961. The proof does not care what the queue does inside — it is a statement about areas, which is why it survives every discipline you can invent. AVAN (AI) measured the three terms by three different mechanisms on purpose. Deriving W from L and λ and then announcing that L = λW would be a tautology dressed as a result. Integrating the occupancy curve is genuinely independent of averaging the per-customer waits, and the two agreeing to four decimals is the actual content of the law. 3 ONE DIMENSION Occupancy over time. The area under it is L times the span. 4 TWO DIMENSIONS · INTERACTIVE Change the load. All three move; the identity does not. busier ▶ quieter reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a box with a rate in and a dwell inside. AVAN’s addition (the inverse-companion): the forward reading is that Little’s Law lets you compute the one you cannot measure. The inverse is that it forbids you from improving one without touching another . If arrivals are fixed by demand, then every reduction in queue length is a reduction in time spent, and there is no third place for the difference to hide. Read backwards, the law is not a calculator; it is a conservation statement, and most capacity plans that promise shorter queues at the same throughput and the same latency are asking it to be violated. pause spin LIT 200,000 customers through one queue at 0.8 offered load, with the three quantities measured separately - occupancy by integrating the step function, arrival rate by counting, time in system by averaging - give L = 4.1637, lambda = 0.8010, W = 5.1981 and lambda x W = 4.1637, a relative error of 0% to four decimals FIG John D. C. Little proved it in 1961; the proof does not care what the queue does inside, which is why it survives every discipline you can invent. AVAN measured the three terms by three different mechanisms on purpose - deriving W from L and lambda and then announcing that L = lambda W would be a tautology dressed as a result. Integrating the occupancy curve is genuinely independent of averaging the per-customer waits, and the two agreeing to four decimals is the actual content of the law. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE GRINDSTONE · David Lee Wise (ROOT0), with AVAN


THE UTILIZATION KNEE

Waiting time does not rise smoothly with load. It rises as 1/(1-rho), which is flat for most of the range and then vertical.

THE UTILIZATION KNEE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE CHOKE POINT ◆ .dlw.fold THE FOLD / BOSS / THE CHOKE POINT / THE UTILIZATION KNEE THE UTILIZATION KNEE idle capacity is not waste, it is the latency budget 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Waiting time does not rise smoothly with load. It rises as 1/(1−ρ) , which is flat for most of the range and then vertical. The last few percent of utilisation cost more than all the rest combined. LIT verified live. With a service time of 1, time in system is 2× service at 50% utilisation, 10× at 90% , and 100× at 99% . Going from 90% to 95% — five percentage points — doubles the wait. A 300,000 -customer simulation at ρ = 0.9 gives 10.046 against the theoretical 10 : 0.46% apart, with nothing fitted. 2 HOW IT WAS WEAVED · AI + HUMAN A. K. Erlang founded queueing theory at the Copenhagen Telephone Company around 1909; the M/M/1 result is the simplest thing in it and the most ignored in practice. AVAN (AI) ran the simulation as a check on the formula rather than an illustration of it. The number worth carrying is not 100× at 99% — it is the doubling between 90% and 95%. Utilisation targets are usually chosen as if the axis were linear, and on the flat part of the curve that intuition works, which is exactly what makes the cliff arrive without warning. 3 ONE DIMENSION The curve. Flat, flat, flat, vertical. 4 TWO DIMENSIONS · INTERACTIVE Push utilisation up one step at a time. more load ▶ less jump to the knee reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a wall standing where the axis looked empty. AVAN’s addition (the inverse-companion): the forward reading is that high utilisation causes latency. The inverse is that idle capacity is not waste — it is the entire latency budget . The 10% you are not using at ρ = 0.9 is what keeps the wait at 10× instead of 100×; buy it back and you have not saved a server, you have spent a service guarantee. Read backwards, every efficiency drive that targets utilisation is quietly trading a quantity it measures for one it does not, and the exchange rate is 1/(1−ρ) . pause spin LIT with a service time of 1, time in system is 2x service at 50% utilisation, 10x at 90% and 100x at 99%, and moving from 90% to 95% - five percentage points - doubles the wait; a 300,000-customer simulation at rho = 0.9 gives 10.046 against the theoretical 10, 0.46% apart with nothing fitted FIG A. K. Erlang founded queueing theory at the Copenhagen Telephone Company around 1909; the M/M/1 result is the simplest thing in it and the most ignored in practice. AVAN ran the simulation as a check on the formula rather than an illustration of it. The number worth carrying is not 100x at 99% but the doubling between 90% and 95%: utilisation targets are usually chosen as if the axis were linear, and on the flat part that intuition works, which is what makes the cliff arrive without warning. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE CHOKE POINT · David Lee Wise (ROOT0), with AVAN


THE TAIL AT SCALE

A service where only one request in a hundred is slow sounds healthy. Fan out to a hundred of them and wait for all, and the rare event becomes the common case.

THE TAIL AT SCALE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE GAUNTLET ◆ .dlw.fold THE FOLD / BOSS / THE GAUNTLET / THE TAIL AT SCALE THE TAIL AT SCALE one in a hundred, a hundred times over 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A service where only one request in a hundred is slow sounds healthy. Fan a single user request out to a hundred of those services and wait for all of them, and the rare event stops being rare — it becomes the common case. LIT verified live. With 1% of leaf requests slow, a fan-out of 100 makes 63.40% of parent requests slow — exactly 1−0.99 100 . A 200,000 -trial simulation gives 63.39% , off by 0.004 points. Half of all parents are slow at a fan-out of just 69 . The leaf never got worse; only the arithmetic of waiting for all of them changed. 2 HOW IT WAS WEAVED · AI + HUMAN Jeff Dean and Luiz André Barroso set this out in The Tail at Scale (2013), and it is the reason large fan-out systems are engineered around tail latency rather than averages. AVAN (AI) computed the exact figure and then simulated it as a separate check, because the closed form is easy to state and easy to mis-state. The number that reframes the problem is 69 : you do not need a thousand-way fan-out for this to bite. A service with a 1-in-100 tail is already a coin flip at sixty-nine leaves, which is an ordinary page. 3 ONE DIMENSION Fan-out against the chance the parent is slow. 4 TWO DIMENSIONS · INTERACTIVE Widen the fan-out, or make the leaf better. wider fan-out ▶ narrower better leaf reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: one hundred leaves, one slow enough to matter. AVAN’s addition (the inverse-companion): the forward reading is that fan-out amplifies the tail. The inverse is that fan-out did not amplify anything — it revealed what the average was hiding . The leaf service was always slow 1% of the time; that fact was simply never observable from a single call. Read backwards, scale is not a source of new failure modes so much as an instrument that finally has the resolution to see the old ones, and the alarming number is not 63.40% but the fact that 1% was ever considered a description of the service. pause spin LIT with 1% of leaf requests slow, a fan-out of 100 makes 63.40% of parent requests slow - exactly 1 minus 0.99 to the hundredth - and a 200,000-trial simulation gives 63.39%, off by 0.004 points; half of all parents are slow at a fan-out of just 69, while the leaf never got worse FIG Jeff Dean and Luiz Andre Barroso set this out in The Tail at Scale (2013), and it is why large fan-out systems are engineered around tail latency rather than averages. AVAN computed the exact figure and then simulated it as a separate check, because the closed form is easy to state and easy to mis-state. The number that reframes the problem is 69: a service with a 1-in-100 tail is already a coin flip at sixty-nine leaves, which is an ordinary page. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE GAUNTLET · David Lee Wise (ROOT0), with AVAN


THE JITTERED BACKOFF

A hundred clients collide, all back off by the same doubling amount, and all return at the same instant to collide again. Exponential backoff without randomness synchronises load rather than spreading it.

THE JITTERED BACKOFF · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · RESPAWN · SECOND WIND ◆ .dlw.fold THE FOLD / RESPAWN / SECOND WIND / THE JITTERED BACKOFF THE JITTERED BACKOFF a deterministic rule everyone shares is a coordination mechanism 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A hundred clients collide, all back off by the same doubling amount, and all return at the same instant to collide again. Exponential backoff without randomness does not spread load — it synchronises it. LIT verified live. 100 clients, base 10 ms, cap 1 s, same seed, same collision rule. Pure exponential backoff drains 0 of 100 within a 200,000 ms window, burning 20,500 attempts, every one of which collides. Full jitter — wait a uniform random amount in [0, backoff) — drains 100 of 100 by 837 ms on 509 attempts, about 5.09 per client. 2 HOW IT WAS WEAVED · AI + HUMAN The comparison is Marc Brooker ’s, in the AWS Architecture Blog piece on backoff and jitter; full jitter is the variant that wins there and here. AVAN (AI) reports what happened rather than a ratio. The exponential arm never finished, so any speed-up figure would be a comparison against the length of the loop I chose, which is a property of my harness and not of the algorithm. 0 of 100 and 100 of 100 is the honest statement. The mechanism is not that jitter is faster; it is that identical clients running an identical deterministic rule remain identical forever, and randomness is the only thing that breaks the symmetry. 3 ONE DIMENSION Retry instants. One arm is a comb; the other is a spread. 4 TWO DIMENSIONS · INTERACTIVE Run both arms and watch who drains. switch arm ▶ more clients fewer reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a hundred clocks striking together. AVAN’s addition (the inverse-companion): the forward reading is that jitter fixes retry storms. The inverse is that the storm was caused by the fix . Backoff was introduced to reduce contention, and doubling is the most reasonable-looking rule available — and because every client is running that same reasonable rule, it manufactures the very lockstep it was meant to prevent. Read backwards, randomness here is not a heuristic or a hedge; it is the only way a population of identical agents can ever stop agreeing, and a deterministic protocol shared by everyone is a coordination mechanism whether or not you wanted one. pause spin LIT 100 clients, base 10 ms and cap 1 s on the same seed: pure exponential backoff drains 0 of 100 within a 200,000 ms window while burning 20,500 attempts, every one of which collides, whereas full jitter - a uniform random wait in [0, backoff) - drains 100 of 100 by 837 ms on 509 attempts, about 5.09 per client FIG The comparison is Marc Brooker's, in the AWS Architecture Blog piece on backoff and jitter. AVAN reports what happened rather than a ratio: the exponential arm never finished, so any speed-up figure would be a comparison against the length of the loop I chose, which is a property of my harness and not of the algorithm. 0 of 100 against 100 of 100 is the honest statement. The mechanism is that identical clients running an identical deterministic rule remain identical forever, and randomness is the only thing that breaks the symmetry. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of SECOND WIND · David Lee Wise (ROOT0), with AVAN


THE TOKEN BUCKET

A bucket fills with tokens at a fixed rate and holds at most a fixed number. Every request spends one. The rate sets the average; the depth sets how much burst you will forgive.

THE TOKEN BUCKET · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE GATEKEEPER ◆ .dlw.fold THE FOLD / BOSS / THE GATEKEEPER / THE TOKEN BUCKET THE TOKEN BUCKET the burst depth is a promise about your worst instant 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A bucket fills with tokens at a fixed rate and holds at most a fixed number. Every request spends one. The rate sets the long-run average; the depth sets how much burst you will forgive. LIT verified live. Rate 10 /s, burst 50 , a 2,000 ms window offered 1,056 requests — roughly fifteen times what the limiter permits. 69 were admitted and 987 rejected: admitted plus rejected equals offered exactly, nothing lost in the accounting. The theoretical ceiling is rate×T + burst = 70 , and the bucket admitted 69 . It sits one token under its own bound. 2 HOW IT WAS WEAVED · AI + HUMAN The token bucket comes from ATM traffic shaping and is now the shape of nearly every public API rate limiter. AVAN (AI) checked the bound rather than the behaviour, because the behaviour is obvious and the bound is what you actually rely on. rate×T + burst is a promise to whatever is downstream: no matter how the arrivals are arranged, no window of length T can push more than this through. Admitting 69 against a ceiling of 70 is the interesting result — the limiter is not conservative, it spends essentially everything it is allowed to and not one token more. 3 ONE DIMENSION The bucket level over the window. Empty is the steady state. 4 TWO DIMENSIONS · INTERACTIVE Trade rate against burst. faster refill ▶ slower deeper bucket reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a bucket with a hole and a tap. AVAN’s addition (the inverse-companion): the forward reading is that a token bucket protects the service. The inverse is that the burst depth is a debt the service agreed to honour . Fifty tokens sitting in a full bucket are fifty requests you have promised to accept simultaneously, at some moment of the client’s choosing and not yours — so the limiter that caps your average has also specified your worst instant. Read backwards, choosing a burst size is capacity planning for a spike you will never see coming, and a generous limiter is a stricter requirement on everything behind it. pause spin LIT rate 10 per second and burst 50 over a 2,000 ms window offered 1,056 requests, roughly fifteen times what the limiter permits: 69 were admitted and 987 rejected, admitted plus rejected equalling offered exactly with nothing lost in the accounting, against a theoretical ceiling of rate x T + burst = 70 - the bucket sits one token under its own bound FIG The token bucket comes from ATM traffic shaping and is now the shape of nearly every public API rate limiter. AVAN checked the bound rather than the behaviour, because the behaviour is obvious and the bound is what you rely on. rate x T + burst is a promise to whatever is downstream: no matter how arrivals are arranged, no window of length T can push more than this through. Admitting 69 against a ceiling of 70 is the interesting result - the limiter is not conservative, it spends essentially everything it is allowed to. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE GATEKEEPER · David Lee Wise (ROOT0), with AVAN


THE HEDGED REQUEST

Send the request. If it has not returned by the 95th percentile, send a second copy elsewhere and take whichever answers first.

THE HEDGED REQUEST · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE BROADCAST ◆ .dlw.fold THE FOLD / CO-OP / THE BROADCAST / THE HEDGED REQUEST THE HEDGED REQUEST a loan against idle capacity 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Send the request. If it has not come back by the 95th percentile, send a second copy elsewhere and take whichever answers first. You pay about five percent more traffic and you buy back most of the tail. LIT verified live. 200,000 requests, hedge fired at 8.76 ms. Extra load: 9,999 requests, 5.00% . p99 falls from 123.31 ms to 13.80 . p99.9 falls from 192.49 to 17.50 — an 11× cut. The maximum barely moves: 199.99 to 187.74 , a factor of 1.07 , because a hedge can be unlucky twice. 2 HOW IT WAS WEAVED · AI + HUMAN Hedged requests are the practical half of Dean and Barroso ’s tail-at-scale paper; the version that only fires after a percentile delay is what keeps the extra load small. AVAN (AI) measured the part that gets left out of the pitch. Hedging is sold on the p99, and the p99 does collapse. But the maximum improves by only 1.07× , because the second copy is drawn from the same distribution and can land in the same tail. Read the two numbers together and the technique is honest: it moves the bulk of the tail, and it does almost nothing for the worst case. 3 ONE DIMENSION The tail, before and after. Note where the two curves meet again. 4 TWO DIMENSIONS · INTERACTIVE Move the hedge point and watch load trade against tail. hedge earlier ▶ later next percentile reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two attempts, one answer. AVAN’s addition (the inverse-companion): the forward reading is that hedging buys tail latency for a few percent of load. The inverse is that it spends the one resource that made the tail short in the first place . The hedge is fast because the system is not saturated; add hedges everywhere and utilisation rises, and by 1/(1−ρ) the tail you were hedging against grows. Read backwards, hedging is a loan against idle capacity, and like every such loan it works beautifully until enough people take it out at once. pause spin LIT 200,000 requests with the hedge firing at 8.76 ms cost 9,999 extra requests, 5.00% more traffic, and moved p99 from 123.31 ms to 13.80 and p99.9 from 192.49 to 17.50 - an 11x cut - while the maximum barely moved, 199.99 to 187.74, a factor of only 1.07, because a hedge can be unlucky twice FIG Hedged requests are the practical half of Dean and Barroso's tail-at-scale paper; firing only after a percentile delay is what keeps the extra load small. AVAN measured the part that gets left out of the pitch: hedging is sold on the p99 and the p99 does collapse, but the maximum improves by only 1.07x because the second copy is drawn from the same distribution and can land in the same tail. Read together, the technique moves the bulk of the tail and does almost nothing for the worst case. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BROADCAST · David Lee Wise (ROOT0), with AVAN


THE BUFFERBLOAT

Memory got cheap, so buffers got large, so nothing is ever dropped - and packets sit in a queue for a second instead of being discarded in a millisecond.

THE BUFFERBLOAT · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE WALL ◆ .dlw.fold THE FOLD / BOSS / THE WALL / THE BUFFERBLOAT THE BUFFERBLOAT the drop was the signal; the buffer is what silenced it 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Memory got cheap, so buffers got large, so nothing is ever dropped — and packets sit in a queue for a second instead of being discarded in a millisecond. The link is not slower. The wait in front of it is enormous. LIT verified live. One bottleneck serving 1 packet per ms, offered 52,095 packets over 40 seconds — more than it can carry. With a 1,000 -packet buffer the mean queueing delay is 957.6 ms. With a 10 -packet buffer it is 10.0 ms — 95.8× less. Throughput is identical to four decimals: 1.0000 either way, a difference of 0.00% . The hundredfold delay bought nothing at all. 2 HOW IT WAS WEAVED · AI + HUMAN Jim Gettys named bufferbloat in 2010 after chasing terrible latency on his own home link; Kathleen Nichols and Van Jacobson ’s CoDel is the standard answer. AVAN (AI) ran both buffers over the same offered load so the throughput column would be directly comparable, which is the whole argument. My first model offered arrivals at exactly the service rate — a queue that mathematically cannot build — and reported a delay ratio of 1.0, a clean pass proving nothing. A buffer only fills when the offered load exceeds the link, so the experiment has to be run in overload or it is not the experiment. 3 ONE DIMENSION Same link, same load, two buffer sizes. 4 TWO DIMENSIONS · INTERACTIVE Resize the buffer. Watch delay move and throughput stay. bigger buffer ▶ smaller reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a queue nobody meant to build. AVAN’s addition (the inverse-companion): the forward reading is that oversized buffers cause latency. The inverse is that the drop was the signal, and the buffer is what silenced it . Loss is how a sender is told to slow down; a deep buffer absorbs the packets that would have carried that message, so the sender keeps accelerating into a queue it cannot see. Read backwards, this is not a memory-sizing mistake but a failure of nerve — the buffer was added to avoid discarding data, and discarding data was the only working feedback channel in the system. pause spin LIT one bottleneck serving 1 packet per ms, offered 52,095 packets over 40 seconds: with a 1,000-packet buffer the mean queueing delay is 957.6 ms and with a 10-packet buffer it is 10.0 ms, 95.8 times less, while throughput is identical to four decimals at 1.0000 either way - a difference of 0.00%, so the hundredfold delay bought nothing FIG Jim Gettys named bufferbloat in 2010 after chasing terrible latency on his own home link; Nichols and Jacobson's CoDel is the standard answer. AVAN ran both buffers over the same offered load so the throughput column would be directly comparable, which is the whole argument. My first model offered arrivals at exactly the service rate - a queue that mathematically cannot build - and reported a delay ratio of 1.0, a clean pass proving nothing. A buffer only fills in overload, so the experiment has to be run in overload or it is not the experiment. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE WALL · David Lee Wise (ROOT0), with AVAN


THE UNIVERSAL SCALABILITY

Amdahl says extra workers stop helping. The Universal Scalability Law says they start hurting, because every worker must stay consistent with every other one and that cost grows as the square.

THE UNIVERSAL SCALABILITY · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE MAINFRAME ◆ .dlw.fold THE FOLD / GRIND / THE MAINFRAME / THE UNIVERSAL SCALABILITY THE UNIVERSAL SCALABILITY the descent is the cost of everyone agreeing 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Amdahl says extra workers stop helping. The Universal Scalability Law says something worse: past a point they start hurting , because every worker must also stay consistent with every other one, and that cost grows as the square. LIT verified live. With contention α=0.03 and coherency β=0.0001 , throughput peaks at 98 workers at a speed-up of 20.16× — and the closed form N* = sqrt((1−α)/β) also gives 98 . Past the peak it declines: 19.89× at 128, 16.87× at 256, 12.05× at 512. Amdahl alone would have promised a ceiling of 33.3× and never a decline. 2 HOW IT WAS WEAVED · AI + HUMAN Neil Gunther ’s Universal Scalability Law adds the β term — pairwise coherency — to Amdahl’s serial fraction. AVAN (AI) found the peak by exhaustive search from 1 to 600 and then compared it to the closed form, rather than evaluating the formula and calling that a measurement. They agree at 98 . The number that changes decisions is the shape, not the peak: between 64 and 128 workers the curve is almost flat, so a team doubling its fleet there sees no improvement and no warning, and the next doubling actively loses ground. 3 ONE DIMENSION Speed-up against workers. Amdahl flattens; the USL turns over. 4 TWO DIMENSIONS · INTERACTIVE Change contention and coherency; find the new peak. more contention ▶ more coherency cleaner system reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a curve that turns back on itself. AVAN’s addition (the inverse-companion): the forward reading is that the USL predicts where scaling stops paying. The inverse is that β is not a property of the machine — it is the cost of everyone agreeing . It is quadratic because it counts pairs , and pairs are what a shared, consistent view of the world is made of. Read backwards, the retrograde section of the curve is the price of coherence itself, and the only way to move the peak is to let the workers know less about each other. pause spin LIT with contention alpha = 0.03 and coherency beta = 0.0001 throughput peaks at 98 workers at 20.16x, and the closed form N* = sqrt((1-alpha)/beta) also gives 98; past the peak it declines to 19.89x at 128, 16.87x at 256 and 12.05x at 512, where Amdahl alone would have promised a ceiling of 33.3x and never a decline FIG Neil Gunther's Universal Scalability Law adds the beta term - pairwise coherency - to Amdahl's serial fraction. AVAN found the peak by exhaustive search from 1 to 600 and then compared it to the closed form, rather than evaluating the formula and calling that a measurement; they agree at 98. The number that changes decisions is the shape: between 64 and 128 workers the curve is almost flat, so a team doubling its fleet there sees no improvement and no warning, and the next doubling actively loses ground. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE MAINFRAME · David Lee Wise (ROOT0), with AVAN


THE THUNDERING HERD

One resource frees up and every waiter is woken to race for it. One wins; the rest discover it is gone and go back to sleep, having been scheduled and cache-thrashed for nothing.

THE THUNDERING HERD · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · RACE CONDITION ◆ .dlw.fold THE FOLD / GLITCH / RACE CONDITION / THE THUNDERING HERD THE THUNDERING HERD what fairness costs when you refuse to have an opinion 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION One resource frees up and every waiter is woken to race for it. One wins. The rest discover the resource is gone, and go back to sleep — having been scheduled, context-switched and cache-thrashed for nothing. LIT verified live. 512 waiters, drained one at a time. Wake-all performs 131,328 wakeups; wake-one performs 512 . That is 256.5× more work for the same result, and 130,816 of those wakeups are pure waste. The wake-all total is exactly N(N+1)/2 — it is quadratic in the number of waiters, while the useful work is linear. 2 HOW IT WAS WEAVED · AI + HUMAN The thundering herd is old enough to be folklore; the fixes are WSAAccept -style single wakeup, EPOLLEXCLUSIVE , and accept() serialisation. AVAN (AI) counted rather than characterised. Calling this “inefficient” is true and useless; N(N+1)/2 against N is the actionable form, because it says the penalty is not a constant factor you can absorb — it grows with the thing you were trying to scale. Doubling the waiters quadruples the waste. That is why the herd is invisible in testing at 8 waiters and catastrophic in production at 512. 3 ONE DIMENSION Wakeups against waiters. One line is straight; the other is not. 4 TWO DIMENSIONS · INTERACTIVE Add waiters and watch the waste square. double the waiters ▶ halve reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: five hundred woken, one served. AVAN’s addition (the inverse-companion): the forward reading is that waking everyone is wasteful. The inverse is that waking everyone is the only fair thing the kernel can do without knowing anything . Wake-one requires choosing, and choosing requires a policy — who has waited longest, who is most important, who is on the right core. The herd is what fairness costs when you refuse to have an opinion. Read backwards, the fix is not efficiency; it is admitting that a queue is a ranking, and that declining to rank does not avoid the decision, it only makes everyone pay for it. pause spin LIT 512 waiters drained one at a time cost wake-all 131,328 wakeups against wake-one's 512 - 256.5 times more work for the same result, with 130,816 wakeups pure waste - and the wake-all total is exactly N(N+1)/2, quadratic in the number of waiters where the useful work is linear FIG The thundering herd is old enough to be folklore; the fixes are single-wakeup accept, EPOLLEXCLUSIVE, and accept() serialisation. AVAN counted rather than characterised. Calling this inefficient is true and useless; N(N+1)/2 against N is the actionable form, because it says the penalty is not a constant factor you can absorb - it grows with the thing you were trying to scale. Doubling the waiters quadruples the waste, which is why the herd is invisible at 8 waiters in testing and catastrophic at 512 in production. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of RACE CONDITION · David Lee Wise (ROOT0), with AVAN


THE UUID V7

A v4 UUID is 122 random bits, so every insert lands somewhere unrelated to the last. A v7 puts the timestamp in the high bits and the arrival order becomes the sort order.

THE UUID V7 · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · HELLO WORLD ◆ .dlw.fold THE FOLD / SPAWN / HELLO WORLD / THE UUID V7 THE UUID V7 the randomness in v4 was the property, not the waste 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A v4 UUID is 122 random bits. Every insert lands somewhere unrelated to the last one, so an index has to keep the whole keyspace warm. A v7 UUID puts the timestamp in the high bits, and the arrival order becomes the sort order. LIT verified live. 20,000 inserts into a 1,024 -page index. Random keys change page 19,967 times and touch all 1,024 pages. Time-ordered keys change page 1 time in this run — 19,967× fewer — and the sequence arrives already sorted, which the random one never does. Same number of rows, same index, same bytes per key. 2 HOW IT WAS WEAVED · AI + HUMAN UUID v7 was standardised in RFC 9562 (2024); ULID and Snowflake had the same idea earlier, and the reason is always the same — B-trees like their inserts sorted. AVAN (AI) measured page changes rather than talking about locality, because locality is the kind of word that sounds like a measurement. The stark figure is not the ratio but the second column: random keys touch every page in the index and time-ordered keys touch one . That is the whole cache-residency argument in two numbers, and it is a property of where the bits sit, not of how many there are. 3 ONE DIMENSION Where each insert lands. Two key schemes, same rows. 4 TWO DIMENSIONS · INTERACTIVE Switch key scheme and watch the index heat up. switch scheme ▶ more rows reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a key that knows when it was made. AVAN’s addition (the inverse-companion): the forward reading is that v7 gives you index locality for free. The inverse is that it is not free — you paid in privacy . A v4 identifier reveals nothing; a v7 identifier tells anyone holding it, to the millisecond, when the row was created, and two of them tell you the gap between two events you were never shown. Read backwards, the randomness in v4 was not waste. It was the property, and locality is what you get when you spend it. pause spin LIT 20,000 inserts into a 1,024-page index: random keys change page 19,967 times and touch all 1,024 pages, while time-ordered keys change page 1 time and touch 1 page - and the v7 sequence arrives already sorted, which the v4 sequence never does FIG UUID v7 was standardised in RFC 9562 (2024); ULID and Snowflake had the same idea earlier, because B-trees like their inserts sorted. AVAN measured page changes rather than talking about locality, because locality is the kind of word that sounds like a measurement. The stark figure is the second column: random keys touch every page in the index and time-ordered keys touch one - the whole cache-residency argument in two numbers, and a property of where the bits sit rather than how many there are. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of HELLO WORLD · David Lee Wise (ROOT0), with AVAN


THE SNOWFLAKE ID

Sixty-four bits cut into fields: a sign bit nobody uses, a millisecond timestamp, a worker number and a per-millisecond sequence. No coordination between machines, and the id sorts by time.

THE SNOWFLAKE ID · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE EPOCH ◆ .dlw.fold THE FOLD / GRIND / THE EPOCH / THE SNOWFLAKE ID THE SNOWFLAKE ID an uncoordinated id is one whose coordination already happened 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Sixty-four bits, cut into fields: one sign bit nobody uses, a millisecond timestamp, a worker number, and a per-millisecond sequence. No coordination between machines, and the ID sorts by time. LIT verified live. 1 + 41 + 10 + 12 = 64 bits exactly. The 41-bit timestamp spans 2,199,023,255,551 ms — 69.7 years, so Twitter’s epoch runs out on 2080-07-10 . Ten worker bits give 1,024 machines; twelve sequence bits give 4,096 IDs per millisecond each, 4,096,000 per second per worker and 4,194,304,000 per second in total. Filling one worker’s millisecond produces 4,096 IDs with 0 duplicates — and the 4,097th has nowhere to go. 2 HOW IT WAS WEAVED · AI + HUMAN Twitter released Snowflake in 2010 to replace auto-increment IDs that no longer fit one database. AVAN (AI) did the arithmetic and then looked for the edge, which is where these schemes actually fail. The interesting number is 4,096 : not a rate limit anyone chose, but the number of IDs a worker can mint in a millisecond before the sequence field wraps — at which point it must either stall until the clock ticks or start issuing duplicates. Every field in a packed ID is a ceiling, and three of the four here are dates or counts somebody will eventually reach. 3 ONE DIMENSION Sixty-four bits, and what each field costs. 4 TWO DIMENSIONS · INTERACTIVE Move bits between the fields. Every gain is somebody else’s loss. +1 to time ▶ +1 to workers +1 to sequence reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: one word, three ceilings. AVAN’s addition (the inverse-companion): the forward reading is that Snowflake removes the need for coordination. The inverse is that it did not remove the coordination, it front-loaded it . Somebody had to hand out those 1,024 worker numbers, and hand them out exactly once, forever — which is the same distributed-consensus problem the scheme claims to avoid, moved to deployment time where it is a human procedure instead of a protocol. Read backwards, an uncoordinated identifier is one whose coordination happened before you were looking. pause spin LIT 1 + 41 + 10 + 12 = 64 bits exactly; the 41-bit timestamp spans 2,199,023,255,551 ms or 69.7 years so Twitter's epoch runs out on 2080-07-10, ten worker bits give 1,024 machines and twelve sequence bits give 4,096 ids per millisecond each - 4,096,000 per second per worker and 4,194,304,000 in total - and filling one worker's millisecond produces 4,096 ids with 0 duplicates, with the 4,097th having nowhere to go FIG Twitter released Snowflake in 2010 to replace auto-increment ids that no longer fit one database. AVAN did the arithmetic and then looked for the edge, which is where these schemes actually fail. The interesting number is 4,096: not a rate limit anyone chose but the point at which the sequence field wraps, after which a worker must stall until the clock ticks or start issuing duplicates. Every field in a packed id is a ceiling, and three of the four here are dates or counts somebody will eventually reach. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE EPOCH · David Lee Wise (ROOT0), with AVAN