THE MINIMAL PERFECT HASH

A hash table wastes space so collisions have somewhere to go. If the key set never changes, you can send n keys onto 0..n-1 with no collisions and no gaps, and store no keys at all.

THE MINIMAL PERFECT HASH · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CHEAT · THE SHORTCUT ◆ .dlw.fold THE FOLD / CHEAT / THE SHORTCUT / THE MINIMAL PERFECT HASH THE MINIMAL PERFECT HASH no slack, and so no way to say not here 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A hash table wastes space so collisions have somewhere to go. If the key set never changes you can do better: find a function that sends n known keys onto 0..n−1 with no collisions and no gaps, then store no keys at all. LIT verified live. 5,000 keys, 1,250 buckets, each bucket assigned its own seed by search. The result is checked rather than assumed: 0 collisions, 0 out of range, 5,000 distinct slots — a bijection onto exactly 0..4,999 . It cost 213,904 seed trials to build, the worst bucket needing seed 5,402 , and the finished structure is 3.25 bits per key. 2 HOW IT WAS WEAVED · AI + HUMAN The bucket-then-search construction is Botelho, Pagh and Ziviani ’s CHD; the theoretical floor for a minimal perfect hash is about 1.44 bits per key. AVAN (AI) verified the bijection by re-hashing all 5,000 keys through the finished function and counting distinct landing slots, rather than trusting the construction that had just claimed success. A builder that reports success is exactly the thing under test. The asymmetry is the real result: 213,904 trials to build, one hash to query — all the cost is paid once, by whoever compiles the table. 3 ONE DIMENSION Five thousand slots. Every one filled, exactly once. 4 TWO DIMENSIONS · INTERACTIVE Watch the buckets get placed, largest first. place more ▶ place all reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a function with no free space. AVAN’s addition (the inverse-companion): the forward reading is that a minimal perfect hash is the most efficient lookup possible. The inverse is that it cannot say no . Query a key that was not in the original set and it will return a slot — a perfectly valid, entirely wrong slot — because there is no spare room in which to represent absence. Read backwards, the gaps in an ordinary hash table were never waste; they were where the answer “not here” lived, and a structure with no slack has no way to be uncertain. pause spin LIT 5,000 keys across 1,250 buckets, each bucket given its own seed by search, then checked rather than assumed: 0 collisions, 0 out of range and 5,000 distinct slots - a bijection onto exactly 0..4,999 - at a cost of 213,904 seed trials to build, a worst bucket needing seed 5,402, and a finished structure of 3.25 bits per key FIG The bucket-then-search construction is Botelho, Pagh and Ziviani's CHD; the theoretical floor for a minimal perfect hash is about 1.44 bits per key. AVAN verified the bijection by re-hashing all 5,000 keys through the finished function and counting distinct landing slots, rather than trusting the construction that had just claimed success - a builder that reports success is exactly the thing under test. The asymmetry is the real result: 213,904 trials to build, one hash to query. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SHORTCUT · David Lee Wise (ROOT0), with AVAN


THE CROCKFORD BASE32

An alphabet meant to be read aloud, written down and typed back in. Crockford's base32 throws out I, L, O and U - three because they look like digits, the last so the encoding cannot spell things.

I-13 v2.0 | net = binds - k | 4 planes, 13 symbols, 12 operants, 5 cortex rules | sha 64881ebf

THE CROCKFORD BASE32 · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · LOOT · THE MINT ◆ .dlw.fold THE FOLD / LOOT / THE MINT / THE CROCKFORD BASE32 THE CROCKFORD BASE32 an alphabet that decides which differences are real 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION An alphabet meant to be read aloud, written down and typed back in. Crockford’s base32 throws out I , L , O and U — the first three because they look like digits, the last so the encoding cannot accidentally spell things. LIT verified live. The alphabet is 32 symbols and contains 0 of the four excluded letters. All 6 confusable inputs decode to the digit they resemble — O and o to zero, I , i , L and l to one — 6 of 6 . Every value from 0 to 200,000 round-trips through encode and decode with 0 failures, and 0 failures again when the text is lowercased first. 2 HOW IT WAS WEAVED · AI + HUMAN Douglas Crockford ’s specification is a page long and its design notes are about humans, not machines: what people mistype, mishear and misread. AVAN (AI) tested the forgiveness rather than the encoding. Round-tripping clean input proves the codec works; the point of this alphabet is what happens with dirty input, so the test that matters is feeding it the mistakes it was designed around. It accepts all six and resolves each to the intended digit. The excluded U is the odd one — it is not confusable with anything, it is excluded so that random identifiers do not spell obscenities. 3 ONE DIMENSION Thirty-two symbols kept, four thrown away. 4 TWO DIMENSIONS · INTERACTIVE Type a mistake and watch it be forgiven. next value ▶ make a typo lowercase it reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: an alphabet shaped around human error. AVAN’s addition (the inverse-companion): the forward reading is that removing four letters makes the encoding safer. The inverse is that it makes the encoding lossy on purpose, in the direction of the reader . O and 0 are now the same symbol; you cannot round-trip a distinction the alphabet has decided not to hear. That is not a defect, it is the whole design — but it means the encoding is no longer a neutral container. Read backwards, every human-facing format is a claim about which differences are real, and this one has decided that a letter and a digit that look alike simply are alike. pause spin LIT the alphabet is 32 symbols containing 0 of the four excluded letters; all 6 confusable inputs decode to the digit they resemble - O and o to zero, I, i, L and l to one, 6 of 6 - and every value from 0 to 200,000 round-trips through encode and decode with 0 failures, and 0 again when the text is lowercased first FIG Douglas Crockford's specification is a page long and its design notes are about humans rather than machines: what people mistype, mishear and misread. AVAN tested the forgiveness rather than the encoding, since round-tripping clean input only proves the codec works and the point of this alphabet is what happens with dirty input. The excluded U is the odd one - not confusable with anything, but excluded so that random identifiers do not spell obscenities. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE MINT · David Lee Wise (ROOT0), with AVAN


THE NOTHING UP MY SLEEVE

A cipher needs arbitrary constants, and anyone free to choose them could be choosing a backdoor. So you do not choose - you take the digits of something fixed before you arrived.

THE NOTHING UP MY SLEEVE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · GENESIS BLOCK ◆ .dlw.fold THE FOLD / SPAWN / GENESIS BLOCK / THE NOTHING UP MY SLEEVE THE NOTHING UP MY SLEEVE it does not remove the choice, it makes it arguable 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A cipher needs arbitrary constants. Anyone who chooses them freely could be choosing a backdoor, and nobody could tell. So you do not choose: you take the digits of something that was fixed before you arrived. LIT verified live — and this one is re-derived here, not quoted. SHA-256’s eight initial hash values are the first 32 bits of the fractional part of the square roots of the first 8 primes . Its sixty-four round constants are the same thing from the cube roots of the first 64 primes . Computed from the primes and compared against the published constants: 8 of 8 , 64 of 64 — 72 of 72 exact. sqrt(2) gives 6a09e667 ; cbrt(2) gives 428a2f98 . 2 HOW IT WAS WEAVED · AI + HUMAN “Nothing-up-my-sleeve” numbers are a convention with real history behind it — DES’s unexplained S-boxes drew suspicion for two decades, and Dual_EC_DRBG’s unexplained points turned out to deserve it. AVAN (AI) did the only thing that makes this claim mean anything: computed the constants rather than repeating the story. A table of hex values quoted from a standards document and labelled “these come from the primes” is a claim about provenance that nobody checked. Seventy-two independent derivations, all matching, is the check. The constants are not trustworthy because a document says where they came from; they are trustworthy because you can go and get them yourself. 3 ONE DIMENSION Prime, root, fraction, constant. Derived and published, side by side. 4 TWO DIMENSIONS · INTERACTIVE Walk the constants and check them one at a time. next constant ▶ switch table reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: constants nobody was free to pick. AVAN’s addition (the inverse-companion): the forward reading is that deriving constants from the primes proves nobody chose them. The inverse is that somebody chose the primes, the roots, the bit width and the order . Why square roots and not logarithms; why the first eight and not the eighth through fifteenth; why 32 bits from the fraction rather than 40 — each is a free parameter, and a designer with enough of them can still search. Read backwards, nothing-up-my-sleeve does not eliminate the choice; it makes the remaining choices few enough and public enough to be argued about, which is a weaker and far more honest claim. pause spin LIT SHA-256's eight initial hash values are the first 32 bits of the fractional parts of the square roots of the first 8 primes and its sixty-four round constants are the same from the cube roots of the first 64 primes; re-derived here from the primes and compared against the published values, 8 of 8 and 64 of 64 match - 72 of 72 exact, with sqrt(2) giving 6a09e667 and cbrt(2) giving 428a2f98 FIG Nothing-up-my-sleeve numbers are a convention with real history behind them - DES's unexplained S-boxes drew suspicion for two decades and Dual_EC_DRBG's unexplained points turned out to deserve it. AVAN did the only thing that makes the claim mean anything: computed the constants rather than repeating the story. A table of hex quoted from a standard and labelled as coming from the primes is a claim about provenance nobody checked; seventy-two independent derivations, all matching, is the check. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GENESIS BLOCK · David Lee Wise (ROOT0), with AVAN


THE CONTENT ADDRESS

A location address says where something is kept. A content address says what it is: hash the bytes and let the digest be the name. Move it, mirror it, rename the server - the name does not change.

THE CONTENT ADDRESS · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · LOOT · THE VAULT ◆ .dlw.fold THE FOLD / LOOT / THE VAULT / THE CONTENT ADDRESS THE CONTENT ADDRESS permanence bought with the ability to be corrected 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A location address says where something is kept. A content address says what it is: hash the bytes and let the digest be the name. Move it, mirror it, rename the server — the name does not change, because the name was never about the place. LIT verified live. 30,000 distinct documents produce 0 name collisions. The same bytes presented again produce the same name 1,000 of 1,000 times. And flipping a single bit of the input changes on average 16.18 of the 32 output bits — the ideal is exactly half, 16 . The name is not a summary of the content; it is a fingerprint of every bit at once. 2 HOW IT WAS WEAVED · AI + HUMAN Content addressing is the shape of Git, IPFS, Nix and the fold that seals this corpus; the vocabulary is Merkle ’s. AVAN (AI) measured the avalanche rather than only the collisions, because “no collisions in 30,000” is a weak statement that a poor hash could pass. 16.18 of 32 bits is the property that makes the name a fingerprint: a one-bit edit produces a name with no visible relationship to the old one, so you cannot find similar content by looking at similar names. That is a real cost, and it is the same property that makes tampering detectable. 3 ONE DIMENSION One bit flipped. Which output bits moved. 4 TWO DIMENSIONS · INTERACTIVE Edit the document and watch the name leave. flip a bit ▶ move it elsewhere restore 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a name made of the thing itself. AVAN’s addition (the inverse-companion): the forward reading is that content addressing makes names permanent and tamper-evident. The inverse is that it makes correction impossible . If the name is the bytes, then fixing a typo does not update a document — it creates a different one, and every reference to the old name still resolves, forever, to the version with the mistake. Read backwards, mutable names were not sloppiness. They were the mechanism by which a thing could be wrong and then be right, and content addressing trades that away for permanence. pause spin LIT 30,000 distinct documents produce 0 name collisions, the same bytes presented again produce the same name 1,000 of 1,000 times, and flipping a single bit of the input changes on average 16.18 of the 32 output bits where the ideal is exactly half at 16 - the name is not a summary of the content but a fingerprint of every bit at once FIG Content addressing is the shape of Git, IPFS, Nix and the fold that seals this corpus; the vocabulary is Merkle's. AVAN measured the avalanche rather than only the collisions, because no collisions in 30,000 is a weak statement a poor hash could pass. 16.18 of 32 bits is the property that makes the name a fingerprint: a one-bit edit produces a name with no visible relationship to the old one, so similar content cannot be found by looking at similar names - a real cost, and the same property that makes tampering detectable. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE VAULT · David Lee Wise (ROOT0), with AVAN


THE DAMM

A check digit catches typing mistakes. Luhn catches all single-digit errors but not all swaps of neighbouring digits. Damm catches both, using a lookup table instead of arithmetic.

THE DAMM · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · OFF BY ONE ◆ .dlw.fold THE FOLD / GLITCH / OFF BY ONE / THE DAMM THE DAMM the better scheme lost to the one a clerk could do 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A check digit catches typing mistakes. Luhn — the one on every credit card — catches all single-digit errors but not all swaps of neighbouring digits. Damm catches both, using a lookup table instead of arithmetic. LIT verified live. 4,000 numbers, each given a Damm digit and a Luhn digit, then attacked exhaustively. Single-digit substitutions: 324,000 tried, Damm catches 100% and Luhn also catches 100% — they are equal here. Adjacent transpositions are where they part: 28,810 tried against Damm, caught 100% ; 28,760 against Luhn, caught 97.76% , missing 645 . 2 HOW IT WAS WEAVED · AI + HUMAN H. Michael Damm published this in 2004, built on a totally anti-symmetric quasigroup — a 10×10 table with no fixed points on its diagonal. AVAN (AI) nearly published a false result here. My first harness skipped the no-op substitution for the Damm number and then applied the same replacement digit to the Luhn number, so whenever the two check digits differed, one “error” per position was no error at all — and got counted as a Luhn miss. It reported 98.88% , a believable number and a wrong one: doubling is a bijection mod 10, so Luhn cannot miss a single-digit substitution. The real difference is transpositions, and only transpositions. 3 ONE DIMENSION Two schemes, two attacks. They differ in only one column. 4 TWO DIMENSIONS · INTERACTIVE Mistype a number and see which scheme notices. new number ▶ swap two digits change one digit reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a table with an empty diagonal. AVAN’s addition (the inverse-companion): the forward reading is that Damm is strictly better than Luhn. The inverse is that Luhn won anyway, and being worse is why . Luhn is arithmetic you can do in your head on a paper slip in 1954; Damm needs a hundred-entry table nobody can memorise. The scheme that catches every transposition lost to the one that can be computed by a clerk with a pencil. Read backwards, a check digit is not a cryptographic choice but a logistical one, and the winning property was never detection strength. pause spin LIT 4,000 numbers given both a Damm digit and a Luhn digit and attacked exhaustively: across 324,000 single-digit substitutions Damm catches 100% and Luhn also catches 100%, but adjacent transpositions part them - 28,810 tried against Damm and caught 100%, against 28,760 tried on Luhn and caught 97.76%, missing 645 FIG H. Michael Damm published this in 2004, built on a totally anti-symmetric quasigroup - a 10x10 table with no fixed points on its diagonal. AVAN nearly published a false result: my first harness skipped the no-op substitution for the Damm number and applied the same replacement digit to the Luhn number, so whenever the two check digits differed one error per position was no error at all and got counted as a Luhn miss. It reported 98.88%, a believable and wrong number - doubling is a bijection mod 10, so Luhn cannot miss a single-digit substitution. The real difference is transpositions, and only transpositions. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of OFF BY ONE · David Lee Wise (ROOT0), with AVAN


THE HASH FLOODING

A hash table is O(1) on average - over inputs an adversary did not choose. If the hash is fixed and public, colliding keys can be computed in advance and the table becomes one long list.

THE HASH FLOODING · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CHEAT · THE EXPLOIT ◆ .dlw.fold THE FOLD / CHEAT / THE EXPLOIT / THE HASH FLOODING THE HASH FLOODING they declined to be the average case 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A hash table is O(1) on average. Average over what ? Over inputs an adversary did not choose. If the hash function is fixed and public, keys that collide can be computed in advance, and the table degenerates into one long list. LIT verified live. 2,000 keys crafted to collide under a fixed x*31+c hash, inserted into a 1,024 -bucket table. They land in 1 bucket, forming a chain of 2,000 , and cost 1,999,000 comparisons — exactly n(n−1)/2 , the quadratic. The identical keys under a seeded hash spread over 503 buckets, longest chain 12 , and cost 3,975 — 502.9× less work for the same input. 2 HOW IT WAS WEAVED · AI + HUMAN Crosby and Wallach published algorithmic complexity attacks in 2003; the fix is a per-process random seed, which is why SipHash now sits under Python, Ruby and Rust dictionaries. AVAN (AI) checked that the degenerate case is exactly quadratic rather than merely bad, because n(n−1)/2 is falsifiable and “slow” is not. The attack needs no privileged access and no clever timing — only the hash function, which was published. The seed does not make the hash stronger in any cryptographic sense; it makes it unknown , and that alone is the entire defence. 3 ONE DIMENSION Same 2,000 keys. Two hash functions. 4 TWO DIMENSIONS · INTERACTIVE Feed the table more crafted keys and watch the work square. more keys ▶ fewer seed the hash reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a table with one very long row. AVAN’s addition (the inverse-companion): the forward reading is that a fixed hash is a security hole. The inverse is that the hole was in the phrase “on average” . O(1) was always a statement about a distribution of inputs, and every complexity bound quietly names an adversary it assumes does not exist. The keys here are not malformed — they are ordinary strings that happen to be inconvenient. Read backwards, an attacker did not break the data structure; they simply declined to be the average case it was analysed against. pause spin LIT 2,000 keys crafted to collide under a fixed x*31+c hash land in 1 bucket of 1,024, forming a chain of 2,000 and costing 1,999,000 comparisons - exactly n(n-1)/2 - while the identical keys under a seeded hash spread over 503 buckets with a longest chain of 12 and cost 3,975, which is 502.9 times less work for the same input FIG Crosby and Wallach published algorithmic complexity attacks in 2003; the fix is a per-process random seed, which is why SipHash now sits under Python, Ruby and Rust dictionaries. AVAN checked that the degenerate case is exactly quadratic rather than merely bad, because n(n-1)/2 is falsifiable and slow is not. The attack needs no privileged access and no clever timing, only the hash function, which was published. The seed does not make the hash stronger in any cryptographic sense - it makes it unknown, and that alone is the defence. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE EXPLOIT · David Lee Wise (ROOT0), with AVAN


THE PUNYCODE

Domain names became international, so every script on Earth can appear in a URL. Cyrillic and Latin letters that render identically are different characters, and a name is only as trustworthy as the difference you can see.

THE PUNYCODE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CHEAT · THE BACKDOOR ◆ .dlw.fold THE FOLD / CHEAT / THE BACKDOOR / THE PUNYCODE THE PUNYCODE the boundary sits where a machine hands something to an eye 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Domain names became international, so every script on Earth can now appear in a URL. Cyrillic а and Latin a are different characters that render identically in most fonts — and a name is only as trustworthy as the difference you can see. LIT verified live. 10 Cyrillic letters that render like Latin ones, all 10 at genuinely different codepoints. In the word apple , 4 of the 5 letters have a lookalike, giving 15 distinct strings that display identically and are all different bytes — exactly 2 4 −1 . The first character is codepoint 97 in one and 1072 in the other. Same length, same picture, different name. 2 HOW IT WAS WEAVED · AI + HUMAN Punycode ( RFC 3492 ) encodes Unicode names into ASCII so DNS can carry them; the homograph attack was demonstrated against PayPal in 2005 and browsers have been patching the display rules ever since. AVAN (AI) counted the substitutable positions rather than assuming them. My first version asserted 7 variants from a guess about how many letters of apple had lookalikes; the answer is 4 positions and therefore 15 . The check now gates on the relationship — variants equals 2 k −1 for k substitutable letters — instead of on a constant I had reasoned out in my head and got wrong. 3 ONE DIMENSION Ten pairs. Identical picture, different number. 4 TWO DIMENSIONS · INTERACTIVE Swap letters for their lookalikes and count the variants. next word ▶ swap a letter swap them all reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two names with one appearance. AVAN’s addition (the inverse-companion): the forward reading is that lookalike characters make names unsafe. The inverse is that the name was never the thing you were checking — the picture was . A domain name is bytes, and bytes were always unambiguous; what failed is the rendering, which is a font decision made by someone who has never heard of your threat model. Read backwards, this is not a Unicode flaw. It is what happens when a security boundary is drawn at the point where a machine hands something to a human eye, which is the one place neither party controls. pause spin LIT 10 Cyrillic letters that render like Latin ones sit at 10 genuinely different codepoints, and in the word apple 4 of the 5 letters have a lookalike - giving 15 distinct strings that display identically and are all different bytes, exactly 2 to the 4th minus 1 - with the first character at codepoint 97 in one and 1072 in the other FIG Punycode (RFC 3492) encodes Unicode names into ASCII so DNS can carry them; the homograph attack was demonstrated against PayPal in 2005 and browsers have been patching display rules ever since. AVAN counted the substitutable positions rather than assuming them: my first version asserted 7 variants from a guess about how many letters of apple had lookalikes, when the answer is 4 positions and therefore 15. The check now gates on the relationship - variants equals 2^k - 1 for k substitutable letters - instead of a constant I reasoned out and got wrong. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BACKDOOR · David Lee Wise (ROOT0), with AVAN


THE SEQUENTIAL KEY

An auto-increment key sends every insert to the same end of the index. Wonderful for the disk and terrible for a lock: the rightmost page is the only page anybody wants, and every writer wants it at once.

THE SEQUENTIAL KEY · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · WARM CACHE ◆ .dlw.fold THE FOLD / GRIND / WARM CACHE / THE SEQUENTIAL KEY THE SEQUENTIAL KEY one fact, described once by a cache and once by a lock 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION An auto-increment key means every insert goes to the same end of the index. That is wonderful for the disk and terrible for a lock: the rightmost page is the only page anybody wants, and every writer wants it at once. LIT verified live. 50,000 inserts across 512 index pages. Sequential keys move to a new page 512 times — once per page, in order. Random keys move 49,918 times, 97.5× more, touching the whole index constantly. But look at the other end: the last 1,000 sequential inserts all land in just 11 pages. That concentration is the cache win and the contention hotspot, and they are the same number. 2 HOW IT WAS WEAVED · AI + HUMAN Right-edge contention on monotonic keys is why Oracle has reverse-key indexes, why SQL Server documents “last page insert contention”, and part of why UUID v7 exists at all. AVAN (AI) measured both directions on one run so the trade is visible in a single pair of numbers rather than argued as a preference. It is the same statistic read twice: sequential keys are 97.5× better at staying in cache and, for exactly that reason, put every concurrent writer on the same 11 pages. Neither number is the answer; the pair of them is the decision. 3 ONE DIMENSION Where the last thousand inserts landed, under each key scheme. 4 TWO DIMENSIONS · INTERACTIVE Switch schemes and add writers. switch scheme ▶ more writers fewer reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a queue forming at the right edge. AVAN’s addition (the inverse-companion): the forward reading is that sequential keys cause write contention. The inverse is that locality and contention are one property seen from two sides . Everything landing in the same few pages is precisely what makes the index fit in memory, and precisely what makes the writers queue. You cannot buy one without the other, because they are not two effects — they are one fact, described once by a cache and once by a lock. Read backwards, spreading the keys does not solve contention; it pays for it in cache misses, and the bill simply moves to a department that files different tickets. pause spin LIT 50,000 inserts across 512 index pages: sequential keys move to a new page 512 times, once per page in order, while random keys move 49,918 times - 97.5 times more - and the last 1,000 sequential inserts all land in just 11 pages, which is the cache win and the contention hotspot measured as the same number FIG Right-edge contention on monotonic keys is why Oracle has reverse-key indexes, why SQL Server documents last page insert contention, and part of why UUID v7 exists. AVAN measured both directions on one run so the trade is visible in a single pair of numbers rather than argued as a preference. It is the same statistic read twice: sequential keys are 97.5 times better at staying in cache and, for exactly that reason, put every concurrent writer on the same 11 pages. Neither number is the answer; the pair is the decision. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of WARM CACHE · David Lee Wise (ROOT0), with AVAN


THE TLB REACH

The TLB holds a fixed number of translations, not a fixed amount of memory. Entries times page size is the only figure that matters: how much memory the machine can name without a walk.

THE TLB REACH · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · WARM CACHE ◆ .dlw.fold THE FOLD / GRIND / WARM CACHE / THE TLB REACH THE TLB REACH a unit trick, not a capacity gain 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION The TLB caches address translations, and it holds a fixed number of them — not a fixed amount of memory. Multiply entries by page size and you get the only figure that matters: how much memory the machine can currently name without a walk. LIT verified live. 1,536 entries with 4 KB pages reach 6 MB . The same 1,536 entries with 2 MB pages reach 3,072 MB — 512× further, with no extra silicon. Against a 512 MB working set that is the difference between 98.8% uncovered and 0% . One-gigabyte pages reach 1,572,864 MB . 2 HOW IT WAS WEAVED · AI + HUMAN TLB reach is standard architecture vocabulary; the number is rarely printed because it is embarrassing. AVAN (AI) computed the coverage against a stated working set rather than quoting entry counts. 1,536 entries sounds generous and 6 MB does not, and they are the same fact. The whole case for huge pages is in that pair — the cache did not get bigger, the unit of account did. 3 ONE DIMENSION Same entry count, three page sizes. 4 TWO DIMENSIONS · INTERACTIVE Grow the working set until the TLB stops covering it. next page size ▶ bigger working set smaller reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a cache measured in names, not bytes. AVAN’s addition (the inverse-companion): the forward reading is that huge pages extend TLB reach enormously. The inverse is that reach was never a property of the TLB . The hardware is identical in all three columns; what changed is how much territory one name is allowed to claim. Read backwards, this is a unit trick rather than a capacity gain — and it works precisely because a page is an accounting fiction, so making the fiction coarser costs nothing until the day you need to say something finer. pause spin LIT 1,536 entries with 4 KB pages reach 6 MB while the same 1,536 entries with 2 MB pages reach 3,072 MB - 512 times further with no extra silicon - which against a 512 MB working set is the difference between 98.8% uncovered and 0%, and one-gigabyte pages reach 1,572,864 MB FIG TLB reach is standard architecture vocabulary; the number is rarely printed because it is embarrassing. AVAN computed the coverage against a stated working set rather than quoting entry counts. 1,536 entries sounds generous and 6 MB does not, and they are the same fact. The whole case for huge pages is in that pair - the cache did not get bigger, the unit of account did. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of WARM CACHE · David Lee Wise (ROOT0), with AVAN


THE HUGE PAGE

THE HUGE PAGE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · THE SANDBOX ◆ .dlw.fold THE FOLD / SPAWN / THE SANDBOX / THE HUGE PAGE THE HUGE PAGE it helps most where it is needed least 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A bigger page means fewer translations to track. It also means every allocation rounds up to a bigger boundary, and the rounding is charged whether you asked for it or not. LIT verified live. 4,000 allocations totalling 5,919,648,648 bytes. With 4 KB pages: 1,447,224 pages and 0.14% internal waste. With 2 MB pages: 5,159 pages — 280.5× fewer — and 82.77% waste. The page table shrinks by two and a half orders of magnitude and the memory bill rises by a factor of six. 2 HOW IT WAS WEAVED · AI + HUMAN Transparent huge pages are on by default in most Linux distributions, and periodically turned off by database vendors for exactly this reason. AVAN (AI) measured both columns from one allocation trace so the trade is a single fact rather than two arguments. 82.77% is what a workload of many small objects pays; a workload of few large ones pays almost nothing. Neither number is a verdict on huge pages — the size distribution is, and it is the thing nobody measures before flipping the switch. 3 ONE DIMENSION Pages tracked, and bytes wasted, for the same trace. 4 TWO DIMENSIONS · INTERACTIVE Shift the allocation size and watch the trade invert. bigger allocations ▶ smaller reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: fewer, larger boxes. AVAN’s addition (the inverse-companion): the forward reading is that huge pages trade memory for translation speed. The inverse is that they charge the workload that can least afford it . The programs with many small allocations are the ones with poor locality, which are the ones huge pages were meant to rescue — and they are the ones that pay 82.77% . Read backwards, the optimisation helps most where it is needed least, which is the usual shape of anything applied globally to a distribution nobody looked at. pause spin LIT 4,000 allocations totalling 5,919,648,648 bytes take 1,447,224 pages and waste 0.14% at 4 KB, against 5,159 pages - 280.5 times fewer - and 82.77% waste at 2 MB: the page table shrinks by two and a half orders of magnitude while the memory bill rises by a factor of six FIG Transparent huge pages are on by default in most Linux distributions and periodically turned off by database vendors for exactly this reason. AVAN measured both columns from one allocation trace so the trade is a single fact rather than two arguments. 82.77% is what a workload of many small objects pays and a workload of few large ones pays almost nothing - neither number is a verdict on huge pages, the size distribution is, and it is the thing nobody measures before flipping the switch. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SANDBOX · David Lee Wise (ROOT0), with AVAN


THE NUMA HOP

On a multi-socket machine the memory is not one pool. Some is attached to your socket and some to the other, and reaching across costs a fixed toll on every access.

THE NUMA HOP · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · SHARED MEMORY ◆ .dlw.fold THE FOLD / CO-OP / SHARED MEMORY / THE NUMA HOP THE NUMA HOP the flat address space was the lie, and a load-bearing one 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION On a multi-socket machine the memory is not one pool. Some of it is attached to your socket and some to the other one, and reaching across costs a fixed toll on every single access. LIT verified live, as a stated model: local 80 ns, remote 140 ns — a 75.0% penalty per access. All-local runs at 80 ns and all-remote at 140 , a 1.75× slowdown for identical code on identical data. An interleaved policy over 100,000 accesses landed remote 49.94% of the time and averaged 109.96 ns — almost exactly halfway, because interleaving does not avoid the toll, it splits it. 2 HOW IT WAS WEAVED · AI + HUMAN The 80/140 figures are a stated model, not a measurement of your machine — the latencies are hardware-specific. What is measured here is the arithmetic they imply and the interleaving. AVAN (AI) is explicit about that boundary because it is where this kind of sphere usually cheats. The 1.75× follows from the two constants and nothing else; the 49.94% is a real draw from a real generator over 100,000 trials. Naming which numbers are assumed and which are measured is the difference between a model and a claim. 3 ONE DIMENSION Mean latency against the fraction of accesses that cross. 4 TWO DIMENSIONS · INTERACTIVE Move the placement policy and watch the mean move with it. more remote ▶ more local interleave reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two pools wearing one address space. AVAN’s addition (the inverse-companion): the forward reading is that NUMA makes some memory slower. The inverse is that the flat address space was the lie, and it is a load-bearing one . Every pointer looks the same and dereferences the same way; nothing in the type system, the language or the instruction encoding admits that two of them differ by 75% . Read backwards, the uniform address space is what made portable software possible, and NUMA is the bill for a fiction that was worth every penny until the machine stopped being one machine. pause spin LIT with local 80 ns and remote 140 ns as stated constants - a 75.0% penalty per access - all-local runs at 80 ns and all-remote at 140, a 1.75x slowdown for identical code on identical data, while an interleaved policy over 100,000 measured accesses landed remote 49.94% of the time and averaged 109.96 ns, almost exactly halfway FIG The 80/140 figures are a stated model, not a measurement of your machine - the latencies are hardware-specific. What is measured is the arithmetic they imply and the interleaving. AVAN is explicit about that boundary because it is where this kind of sphere usually cheats: the 1.75x follows from the two constants and nothing else, while the 49.94% is a real draw over 100,000 trials. Naming which numbers are assumed and which are measured is the difference between a model and a claim. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of SHARED MEMORY · David Lee Wise (ROOT0), with AVAN


THE PREFETCHER

The hardware watches your address stream and fetches ahead of you. It is not clairvoyant - it is a pattern matcher, and it only wins when there is a pattern.

THE PREFETCHER · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CHEAT · THE SPEEDRUN ◆ .dlw.fold THE FOLD / CHEAT / THE SPEEDRUN / THE PREFETCHER THE PREFETCHER the whole performance cliff is a missing sentence 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION The hardware watches your address stream and fetches ahead of you. It is not clairvoyant — it is a pattern matcher, and it only wins when there is a pattern. LIT verified live. 200,000 accesses. Walking forward, the next address is the previous plus one 199,999 times out of 200,000 — 100.00% predictable, with a single miss at the very first access because there is nothing before it. The identical count of accesses drawn at random is predictable 0 times — 0.00% . Same data, same working set, same instruction count; only the order changed. 2 HOW IT WAS WEAVED · AI + HUMAN Stride prefetchers have been in commodity CPUs since the 1990s and are the reason array code outruns pointer code by margins that look like measurement error. AVAN (AI) counted predictability rather than modelling a cache, because predictability is the property the prefetcher actually depends on and it can be counted exactly. The single miss in the sequential run is worth keeping: it is the cold start, and reporting 199,999 rather than rounding to “all of them” is the difference between a count and a slogan. 3 ONE DIMENSION Two address streams. One has a next; one does not. 4 TWO DIMENSIONS · INTERACTIVE Shuffle the order and watch predictability collapse. shuffle more ▶ less perfectly sequential 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a machine guessing where you are going. AVAN’s addition (the inverse-companion): the forward reading is that the prefetcher rewards sequential access. The inverse is that it punishes you for information it does not have . Your random walk is not disordered — you know exactly where you are going next; the hardware simply cannot see it, because the only channel between your intent and the memory system is the address you already issued. Read backwards, prefetching is a guess made necessary by an interface with no way to say “next I will need this”, and the whole performance cliff is a missing sentence. pause spin LIT over 200,000 accesses, walking forward the next address is the previous plus one 199,999 times - 100.00% predictable, with a single miss at the very first access because nothing precedes it - while the identical count drawn at random is predictable 0 times, 0.00%, on the same data, same working set and same instruction count FIG Stride prefetchers have been in commodity CPUs since the 1990s and are why array code outruns pointer code by margins that look like measurement error. AVAN counted predictability rather than modelling a cache, because predictability is the property the prefetcher depends on and it can be counted exactly. The single miss in the sequential run is worth keeping: it is the cold start, and reporting 199,999 rather than rounding to all of them is the difference between a count and a slogan. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SPEEDRUN · David Lee Wise (ROOT0), with AVAN


THE WRITE COMBINING

A store buffer holds writes back briefly so several to the same cache line leave as one transaction. Write the line in order and you pay once for sixty-four stores. Scatter them and you pay every time.

THE WRITE COMBINING · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE HOT LOOP ◆ .dlw.fold THE FOLD / GRIND / THE HOT LOOP / THE WRITE COMBINING THE WRITE COMBINING the saving and the ordering bug are one mechanism 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A store buffer holds writes back briefly so that several to the same cache line can leave as one transaction. Write the line in order and you pay once for sixty-four stores. Scatter them and you pay every time. LIT verified live. 100,000 stores over 64 -byte lines. Written in order they combine into 1,563 bus transactions — 64.0 stores per transaction, and exactly the number of distinct lines the data occupies. Scattered, the same 100,000 stores produce 98,125 transactions: 62.8× the traffic for the identical bytes. 2 HOW IT WAS WEAVED · AI + HUMAN Write-combining buffers are why memcpy and framebuffer writes are fast, and why non-temporal stores exist at all. AVAN (AI) gated on the wrong arithmetic first: I asserted that combined transactions should equal 100000/64 , which is 1562.5 — a value the counter can never take. The measurement of 1,563 was right all along and the assertion was impossible. The gate now compares against ceil , which is the number of distinct lines and the thing the claim is actually about. 3 ONE DIMENSION Sixty-four stores, one line. Ordered and scattered. 4 TWO DIMENSIONS · INTERACTIVE Break the ordering and watch the bus traffic climb. scatter ▶ re-order wider line reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: sixty-four writes leaving as one. AVAN’s addition (the inverse-companion): the forward reading is that write combining saves bus traffic. The inverse is that it saves it by delaying your writes and telling nobody . The buffer holds data that your program believes has been stored, and every fence instruction in existence is there to drain it — so the optimisation is invisible until it is a correctness problem, at which point it is the whole problem. Read backwards, the 64× saving and the memory-ordering bug are the same mechanism, billed to different departments. pause spin LIT 100,000 stores over 64-byte lines combine into 1,563 bus transactions when written in order - 64.0 stores each, and exactly the number of distinct lines the data occupies - against 98,125 transactions when scattered, which is 62.8 times the traffic for the identical bytes FIG Write-combining buffers are why memcpy and framebuffer writes are fast, and why non-temporal stores exist. AVAN gated on the wrong arithmetic first: I asserted that combined transactions should equal 100000/64, which is 1562.5 - a value the counter can never take. The measurement of 1,563 was right all along and the assertion was impossible. The gate now compares against ceil, which is the number of distinct lines and the thing the claim is actually about. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE HOT LOOP · David Lee Wise (ROOT0), with AVAN


THE MINOR FAULT

Not every page fault touches a disk. A minor fault means the page is already in memory and only the mapping was missing. The word fault is doing a lot of unearned work.

THE MINOR FAULT · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · RESPAWN · THE CONTINUE ◆ .dlw.fold THE FOLD / RESPAWN / THE CONTINUE / THE MINOR FAULT THE MINOR FAULT not an error being handled -- an allocation finally happening 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Not every page fault touches a disk. A minor fault means the page is already in memory and only the mapping was missing — the kernel points at it and returns. The word “fault” is doing a lot of unearned work. LIT verified live. 4,096 pages, 12,288 accesses. 3,880 minor faults, 0 major faults, 8,408 accesses with no fault at all. The fault count equals the number of distinct pages ever touched, exactly — 3,880 and 3,880 — because a page faults once and never again, and 216 pages were never touched and never cost anything. 2 HOW IT WAS WEAVED · AI + HUMAN The minor/major distinction is why a process can report millions of faults and be perfectly healthy, and why fault count alone is a useless alarm. AVAN (AI) checked the identity rather than the rate: faults equal distinct pages touched. That is falsifiable, and it is what makes the number harmless — the count is bounded by your working set, not by your access count. 31.58% of accesses faulted here and nothing was wrong; the same figure with major faults would be a machine on its knees. 3 ONE DIMENSION Every page, coloured by whether it ever faulted. 4 TWO DIMENSIONS · INTERACTIVE Touch more pages and watch faults level off. more accesses ▶ fewer reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a fault that costs almost nothing. AVAN’s addition (the inverse-companion): the forward reading is that minor faults are cheap. The inverse is that they are the price of a promise the kernel made and hoped you would not call in . Every mapping you were given was given lazily, on the bet that you would not touch most of it — and the fault is the moment the bet is settled, one page at a time. Read backwards, a minor fault is not an error being handled; it is an allocation finally happening, and the reason your program appeared to start instantly. pause spin LIT 4,096 pages under 12,288 accesses give 3,880 minor faults, 0 major faults and 8,408 accesses with no fault at all, and the fault count equals the number of distinct pages ever touched exactly - 3,880 and 3,880 - because a page faults once and never again, while 216 pages were never touched and never cost anything FIG The minor/major distinction is why a process can report millions of faults and be perfectly healthy, and why fault count alone is a useless alarm. AVAN checked the identity rather than the rate: faults equal distinct pages touched. That is falsifiable, and it is what makes the number harmless - the count is bounded by your working set, not by your access count. 31.58% of accesses faulted here and nothing was wrong; the same figure with major faults would be a machine on its knees. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE CONTINUE · David Lee Wise (ROOT0), with AVAN


THE STRIDED ACCESS

Memory arrives in cache lines, not in variables. Ask for four bytes and sixty-four turn up. Whether that is generous or wasteful depends on how far apart your next four bytes are.

THE STRIDED ACCESS · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE GRINDSTONE ◆ .dlw.fold THE FOLD / GRIND / THE GRINDSTONE / THE STRIDED ACCESS THE STRIDED ACCESS the premium on an insurance policy everyone else claims on 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Memory arrives in cache lines, not in variables. Ask for four bytes and sixty-four turn up. Whether that is generous or wasteful depends entirely on how far apart your next four bytes are. LIT verified live. Four-byte elements on 64 -byte lines. Stride 1 puts 16 elements on every line and uses 100.0% of what arrives. Stride 16 puts 1 element per line and uses 6.3% — 93.7% of every line fetched is thrown away. Past stride 16 nothing improves and nothing worsens: it is already one element per line, and the floor is 6.3% . 2 HOW IT WAS WEAVED · AI + HUMAN This is why array-of-structs and struct-of-arrays are different programs with the same data, and why column stores exist. AVAN (AI) swept every stride rather than contrasting two, because the shape matters more than the endpoints: efficiency halves with each doubling until it hits one element per line, and then it stops. The plateau is the useful part — beyond stride 16 the layout cannot get worse, which means the damage is done long before the access pattern looks dramatic. 3 ONE DIMENSION Useful bytes per 64-byte line, by stride. 4 TWO DIMENSIONS · INTERACTIVE Walk the stride and watch the line empty out. double the stride ▶ halve reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a line arriving mostly unwanted. AVAN’s addition (the inverse-companion): the forward reading is that large strides waste bandwidth. The inverse is that the line was a guess about your intentions, and it is usually right . Fetching sixty-four bytes for a four-byte request is a bet on spatial locality that pays off overwhelmingly often — which is why nobody offers you a four-byte fetch. Read backwards, stride-16 access is not being punished for being slow; it is being charged the premium on an insurance policy that everyone else is claiming on. pause spin LIT with four-byte elements on 64-byte lines, stride 1 puts 16 elements on every line and uses 100.0% of what arrives while stride 16 puts 1 element per line and uses 6.3% - throwing away 93.7% of every line fetched - and past stride 16 nothing changes at all, because it is already one element per line and 6.3% is the floor FIG This is why array-of-structs and struct-of-arrays are different programs with the same data, and why column stores exist. AVAN swept every stride rather than contrasting two, because the shape matters more than the endpoints: efficiency halves with each doubling until it hits one element per line, then stops. The plateau is the useful part - beyond stride 16 the layout cannot get worse, which means the damage is done long before the access pattern looks dramatic. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE GRINDSTONE · David Lee Wise (ROOT0), with AVAN


THE POINTER CHASE

To follow a linked list the machine must load a pointer before it knows which address to load next. The loads cannot overlap, be reordered or be prefetched, because the address does not exist until the previous load returns.

THE POINTER CHASE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CHEAT · NOCLIP ◆ .dlw.fold THE FOLD / CHEAT / NOCLIP / THE POINTER CHASE THE POINTER CHASE the program knows the future and has no way to say so 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION To follow a linked list the machine must load a pointer before it knows which address to load next. The loads cannot overlap, cannot be reordered and cannot be prefetched, because the address does not exist until the previous load returns. LIT verified live. 100,000 nodes arranged by Sattolo’s algorithm into a single cycle: following it from node 0 visits 100,000 distinct nodes and returns to the start — verified, not assumed. Every one of those loads depends on the one before it. Under a stated model of 4 -cycle latency and 8 -wide pipelining, that is 400,000 cycles serial against 50,000 if they could overlap. 2 HOW IT WAS WEAVED · AI + HUMAN Sattolo’s algorithm is a one-character change from Fisher–Yates that guarantees a single cycle rather than a random permutation. AVAN (AI) needed that change. My first version used an ordinary shuffle and the chase visited 873 of 100,000 nodes before looping — because a random permutation decomposes into many short cycles, which is exactly the thing a pointer-chase benchmark must not have. The 4 -cycle and 8 -wide figures are a stated model; the 100,000 -node single cycle is measured. 3 ONE DIMENSION A random permutation, and a Sattolo cycle. Same shuffle, one character apart. 4 TWO DIMENSIONS · INTERACTIVE Walk the chain and watch it refuse to overlap. walk ▶ walk 200 random permutation reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: one cycle through every node. AVAN’s addition (the inverse-companion): the forward reading is that pointer chasing defeats the memory system. The inverse is that the dependency is information, and the machine is refusing to use it . The chain says precisely what comes next — it is written down in the node you are holding — and the hardware cannot act on it because reading it is the operation being waited for. Read backwards, this is the one case where the program knows the future and has no way to say so, and every prefetch hint ever added to an instruction set is an attempt to give it a voice. pause spin LIT 100,000 nodes arranged by Sattolo's algorithm into a single cycle: following it from node 0 visits 100,000 distinct nodes and returns to the start, verified rather than assumed, so every load depends on the one before it - and under a stated model of 4-cycle latency and 8-wide pipelining that is 400,000 cycles serial against 50,000 if they could overlap FIG Sattolo's algorithm is a one-character change from Fisher-Yates that guarantees a single cycle rather than a random permutation. AVAN needed that change: my first version used an ordinary shuffle and the chase visited 873 of 100,000 nodes before looping, because a random permutation decomposes into many short cycles - exactly what a pointer-chase benchmark must not have. The 4-cycle and 8-wide figures are a stated model; the 100,000-node single cycle is measured. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of NOCLIP · David Lee Wise (ROOT0), with AVAN


THE DENORMAL STALL

Below the smallest normal float there is one more range, where the leading bit is dropped and precision is traded away a bit at a time rather than all at once, so subtraction near zero keeps meaning something.

THE DENORMAL STALL · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · FIRST LIGHT ◆ .dlw.fold THE FOLD / SPAWN / FIRST LIGHT / THE DENORMAL STALL THE DENORMAL STALL a slow, silent, correct-looking decline 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Below the smallest normal float there is one more range, where the leading bit is dropped and precision is traded away a bit at a time rather than all at once. It exists so that subtraction near zero keeps meaning something. LIT verified live. The smallest normal double is 2 −1022 and the smallest subnormal is 2 −1074 — a range of exactly 2 52 , or 4,503,599,627,370,496 ×, computed and checked rather than quoted. Halving the smallest subnormal gives exactly 0 : that is the floor. And the difference between the smallest normal and its neighbour is representable, so a − b == 0 and a == b still agree. 2 HOW IT WAS WEAVED · AI + HUMAN Gradual underflow was argued into IEEE 754 by William Kahan against real hardware opposition; subnormals are awkward and, on several generations of CPU, dramatically slow. AVAN (AI) computed the range rather than repeating 2 52 from a reference, and probed the floor by halving until the value became zero. The property worth having is not the extra range — it is that equality and subtraction do not disagree . Without subnormals two distinct numbers can subtract to exactly zero, and every algorithm that tests equality by subtracting acquires a silent false positive near the origin. 3 ONE DIMENSION The last stretch before zero, in powers of two. 4 TWO DIMENSIONS · INTERACTIVE Halve your way to the floor. halve ▶ double jump to the floor reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a staircase that does not end at a cliff. AVAN’s addition (the inverse-companion): the forward reading is that subnormals buy a smooth approach to zero. The inverse is that they buy it with precision you are not told you are losing . Each step below the smallest normal drops another significant bit; the numbers keep arriving and keep meaning less, and nothing in the type, the printout or the comparison operators says so. Read backwards, flush-to-zero is at least honest about failing, and gradual underflow is a slow, silent, correct-looking decline — which is why the fast hardware path and the trustworthy one point in opposite directions. pause spin LIT the smallest normal double is 2^-1022 and the smallest subnormal is 2^-1074, a range of exactly 2^52 or 4,503,599,627,370,496 times, computed and checked rather than quoted; halving the smallest subnormal gives exactly 0, which is the floor, and the difference between the smallest normal and its neighbour is representable so that a - b == 0 and a == b still agree FIG Gradual underflow was argued into IEEE 754 by William Kahan against real hardware opposition; subnormals are awkward and, on several generations of CPU, dramatically slow. AVAN computed the range rather than repeating 2^52 from a reference, and probed the floor by halving until the value became zero. The property worth having is not the extra range but that equality and subtraction do not disagree - without subnormals two distinct numbers can subtract to exactly zero, and every algorithm that tests equality by subtracting acquires a silent false positive near the origin. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of FIRST LIGHT · David Lee Wise (ROOT0), with AVAN


THE CACHE ASSOCIATIVITY

A set-associative cache decides where a line may live from its address, not from how much room is free. Data that fits the cache several times over can still miss every time, if it all maps to the same set.

THE CACHE ASSOCIATIVITY · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE CHOKE POINT ◆ .dlw.fold THE FOLD / BOSS / THE CHOKE POINT / THE CACHE ASSOCIATIVITY THE CACHE ASSOCIATIVITY the room was never the constraint -- the address was 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A set-associative cache decides where a line may live from its address, not from how much room is free. Data that fits the cache several times over can still miss every single time, if it all maps to the same set. LIT verified live. A 32 KB cache: 64 sets, 8 ways, 64 -byte lines. A 16 KB working set — half the capacity — walked 16 times. Read sequentially it misses 6.25% , which is exactly the first pass and nothing more. Read with a stride of 4,096 bytes it misses 100.00% : every access lands in 1 set of 64, and eight ways cannot hold 256 lines. 2 HOW IT WAS WEAVED · AI + HUMAN Conflict misses are why array dimensions get padded by one element, and why power-of-two strides are a known hazard in numerical code. AVAN (AI) got the experiment wrong first. My initial version streamed 4,096 distinct lines through the cache with no reuse at all, so both arms missed 100% — compulsory misses, and a comparison that proved nothing. A capacity argument only means anything over a working set that fits and is revisited. Rebuilt that way, the sequential arm drops to 6.25% and the difference becomes the actual finding. 3 ONE DIMENSION Which sets the two strides touch. 4 TWO DIMENSIONS · INTERACTIVE Change the stride and find the cliff. double the stride ▶ halve pad by one line reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: sixty-four sets, one of them on fire. AVAN’s addition (the inverse-companion): the forward reading is that conflict misses waste a cache that had room. The inverse is that the room was never the constraint — the address was . A fully associative cache has no conflict misses and is unbuildable at speed, so every real cache trades some of its capacity for the ability to find a line in one comparison. Read backwards, the pathological stride is not defeating the cache; it is presenting the bill for a lookup that had to be cheap, and the padding trick works by lying about the address rather than by finding more space. pause spin LIT a 32 KB cache of 64 sets, 8 ways and 64-byte lines, walked 16 times over a 16 KB working set - half its capacity - misses 6.25% read sequentially, which is exactly the first pass and nothing more, and misses 100.00% read with a stride of 4,096 bytes, because every access lands in 1 set of 64 and eight ways cannot hold 256 lines FIG Conflict misses are why array dimensions get padded by one element and why power-of-two strides are a known hazard in numerical code. AVAN got the experiment wrong first: my initial version streamed 4,096 distinct lines through the cache with no reuse, so both arms missed 100% - compulsory misses, and a comparison that proved nothing. A capacity argument only means anything over a working set that fits and is revisited; rebuilt that way the sequential arm drops to 6.25% and the difference becomes the actual finding. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE CHOKE POINT · David Lee Wise (ROOT0), with AVAN


THE NTP SLEW

A wrong clock can be jumped or bent. Jumping is instant and can send time backwards. Slewing takes as long as it takes and never does.

THE NTP SLEW · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE EPOCH ◆ .dlw.fold THE FOLD / GRIND / THE EPOCH / THE NTP SLEW THE NTP SLEW it lies slowly instead of correcting fast 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A clock that is wrong can be corrected two ways: jump it, or bend its rate until it catches up. Jumping is instant and can send time backwards. Slewing takes as long as it takes and never does. LIT verified live. Correcting a 5 -second offset by slewing at 500 ppm takes exactly 10,000 seconds — 2.78 hours — and across the whole correction the clock reads backwards 0 times. A step fixes the same offset instantly and moves the clock 5 seconds backwards in one instruction, which is enough to break every timestamp comparison taken across it. 2 HOW IT WAS WEAVED · AI + HUMAN ntpd slews offsets under 128 ms and steps larger ones; chrony and cloud time services push the slewing envelope much further, precisely to avoid the step. AVAN (AI) measured monotonicity rather than accuracy, because accuracy is what people ask for and monotonicity is what breaks them. 2.78 hours to fix five seconds looks absurd until you notice the alternative is a clock that is briefly a time machine. 3 ONE DIMENSION Two corrections of the same error. 4 TWO DIMENSIONS · INTERACTIVE Change the offset and the slew rate. bigger offset ▶ faster slew reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a clock bending rather than jumping. AVAN’s addition (the inverse-companion): the forward reading is that slewing is the safe correction. The inverse is that it makes the clock permanently, deliberately wrong in order to stay useful . For those 10,000 seconds the machine knows its time is incorrect and reports it anyway, at a rate chosen to keep every comparison valid. Read backwards, monotonicity was never about being right — it is a promise that answers already given will not be contradicted, and a clock keeps that promise by lying slowly instead of correcting fast. pause spin LIT correcting a 5-second offset by slewing at 500 ppm takes exactly 10,000 seconds - 2.78 hours - and across 200 samples of the correction the clock reads backwards 0 times, where a step fixes the same offset instantly and moves the clock 5 seconds backwards in one instruction FIG ntpd slews offsets under 128 ms and steps larger ones; chrony and cloud time services push the slewing envelope much further, precisely to avoid the step. AVAN measured monotonicity rather than accuracy, because accuracy is what people ask for and monotonicity is what breaks them. 2.78 hours to fix five seconds looks absurd until you notice the alternative is a clock that is briefly a time machine. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE EPOCH · David Lee Wise (ROOT0), with AVAN


THE LEAP SMEAR

A leap second repeats a timestamp, and repeated timestamps break anything treating time as an identifier. So you refuse to insert it and make the whole day slightly longer instead.

THE LEAP SMEAR · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GRIND · THE CRON JOB ◆ .dlw.fold THE FOLD / GRIND / THE CRON JOB / THE LEAP SMEAR THE LEAP SMEAR monotonic and correct were always separable 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A leap second repeats a timestamp, and repeated timestamps break anything that treats time as an identifier. So you refuse to insert it: you make the whole day very slightly longer instead. LIT verified live. One second spread across 86,400 is a rate change of 11.574 ppm. Across 2,000 samples of the smeared clock there are 0 non-monotonic readings and 0 duplicate timestamps, and the total drift applied over the window is exactly 1 second — not approximately. The leap second is fully absorbed and never once appears. 2 HOW IT WAS WEAVED · AI + HUMAN Google published leap smearing in 2011 after a leap second took down parts of its fleet; AWS and others followed, and the smear windows are deliberately incompatible between providers. AVAN (AI) checked the two properties that matter separately: that no timestamp repeats, and that the total correction is exactly one second. Either alone is easy. A smear that loses 11 microseconds is monotonic and wrong; one that is exact but steps at the end repeats a timestamp. Both held here. 3 ONE DIMENSION The smear, and the step it replaced. 4 TWO DIMENSIONS · INTERACTIVE Narrow the window and watch the rate change grow. narrower window ▶ wider reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a second dissolved into a day. AVAN’s addition (the inverse-companion): the forward reading is that smearing removes the leap second safely. The inverse is that it removes agreement instead . During the smear your clock disagrees with UTC by up to a second, and with any provider using a different window by up to two — so the fix for a discontinuity in time is a period of sustained, deliberate disagreement about what time it is. Read backwards, monotonic and correct were always separable, and every operator has quietly chosen monotonic. pause spin LIT one second spread across 86,400 is a rate change of 11.574 ppm, and across 2,000 samples of the smeared clock there are 0 non-monotonic readings and 0 duplicate timestamps while the total drift applied over the window is exactly 1 second - not approximately - so the leap second is fully absorbed and never once appears FIG Google published leap smearing in 2011 after a leap second took down parts of its fleet; AWS and others followed, and the smear windows are deliberately incompatible between providers. AVAN checked the two properties separately: that no timestamp repeats, and that the total correction is exactly one second. Either alone is easy - a smear that loses 11 microseconds is monotonic and wrong, one that is exact but steps at the end repeats a timestamp. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE CRON JOB · David Lee Wise (ROOT0), with AVAN


THE CLOCK DRIFT

THE CLOCK DRIFT · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · HEISENBUG ◆ .dlw.fold THE FOLD / GLITCH / HEISENBUG / THE CLOCK DRIFT THE CLOCK DRIFT both clocks are correct and they still disagree 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A quartz oscillator is specified in parts per million, which sounds like a rounding error until you multiply it by a day. LIT verified live. At 50 ppm — an ordinary commodity crystal — a clock drifts 4.32 seconds per day and 1,577.85 seconds per year, which is over twenty-six minutes. It takes 20,000 seconds to accumulate a single second of error. Two machines specified ±50 ppm can be 100 ppm apart from each other and diverge at 8.64 seconds per day — exactly twice the single-clock figure, because error against a reference and error against a peer are different quantities. 2 HOW IT WAS WEAVED · AI + HUMAN Datasheet drift is why NTP exists at all, and why every distributed protocol that assumes bounded skew must say what bound it assumes. AVAN (AI) checked the doubling rather than only tabulating rates. The pairwise figure is the one designs actually need and the one most often taken from the single-clock column — a system tolerant of 4.32 s/day between a node and UTC may still be broken by 8.64 between two nodes that are each within spec. 3 ONE DIMENSION Parts per million, into seconds per day and per year. 4 TWO DIMENSIONS · INTERACTIVE Pick a crystal grade and let it run. worse crystal ▶ better reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two clocks, each in spec, walking apart. AVAN’s addition (the inverse-companion): the forward reading is that clocks drift and must be disciplined. The inverse is that both of those clocks are correct . Each is inside its published tolerance, neither is faulty, and they still disagree by nine seconds a day — so the disagreement is not an error state anybody can detect locally or repair by being more careful. Read backwards, “the clocks are wrong” is a category mistake: the specification permits this, and any protocol that assumed otherwise was assuming something nobody ever promised. pause spin LIT at 50 ppm a clock drifts 4.32 seconds per day and 1,577.85 per year, taking 20,000 seconds to accumulate a single second of error, while two machines each specified plus or minus 50 ppm can be 100 ppm apart and diverge at 8.64 seconds per day - exactly twice the single-clock figure, because error against a reference and error against a peer are different quantities FIG Datasheet drift is why NTP exists and why every distributed protocol assuming bounded skew must say what bound it assumes. AVAN checked the doubling rather than only tabulating rates. The pairwise figure is the one designs actually need and the one most often taken from the single-clock column - a system tolerant of 4.32 s/day between a node and UTC may still be broken by 8.64 between two nodes that are each within spec. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of HEISENBUG · David Lee Wise (ROOT0), with AVAN


THE HAPPENS BEFORE

Without a shared clock, before can only mean one thing: a chain of events along a process or along a message. Everything else is concurrent - not simultaneous, just unordered.

THE HAPPENS BEFORE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE HANDOFF ◆ .dlw.fold THE FOLD / CO-OP / THE HANDOFF / THE HAPPENS BEFORE THE HAPPENS BEFORE a negative result wearing a positive name 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Without a shared clock, “before” can only mean one thing: there is a chain of events from one to the other, along a process or along a message. Everything else is concurrent — not simultaneous, just unordered. LIT verified live. 3 processes, 12 events, 3 messages. The happens-before relation is computed twice by unrelated means — once as the transitive closure of the event graph, once by running vector clocks — and compared on all 132 ordered pairs. They agree on 132 and disagree on 0 . 45 pairs are ordered; 21 of the 66 unordered pairs are genuinely concurrent. 2 HOW IT WAS WEAVED · AI + HUMAN Leslie Lamport ’s 1978 paper defines the relation; vector clocks are Fidge and Mattern ’s independent refinement that makes it decidable from local state. AVAN (AI) computed both sides rather than one. Running vector clocks and announcing that they capture causality is circular — it is the definition restated. Building the graph, closing it transitively, and finding 0 disagreements across 132 pairs is the check, and the 21 concurrent pairs are the reason the relation is partial rather than total. 3 ONE DIMENSION Three timelines and the messages between them. 4 TWO DIMENSIONS · INTERACTIVE Pick an event; see what it can and cannot have caused. next event ▶ previous reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a partial order, not a line. AVAN’s addition (the inverse-companion): the forward reading is that happens-before tells you what caused what. The inverse is that it tells you what could not have . The relation certifies impossibility, never influence — two ordered events may be entirely unrelated, and the 21 concurrent pairs are the ones about which the system is permanently, structurally silent. Read backwards, distributed causality is a negative result wearing a positive name, and every protocol built on it is buying the guarantee that some orderings are ruled out, not that any are true. pause spin LIT 3 processes, 12 events and 3 messages, with the happens-before relation computed twice by unrelated means - once as the transitive closure of the event graph and once by running vector clocks - agree on all 132 ordered pairs and disagree on 0, with 45 pairs ordered and 21 of the 66 unordered pairs genuinely concurrent FIG Leslie Lamport's 1978 paper defines the relation; vector clocks are Fidge and Mattern's independent refinement that makes it decidable from local state. AVAN computed both sides rather than one - running vector clocks and announcing that they capture causality is circular, it is the definition restated. Building the graph, closing it transitively and finding 0 disagreements across 132 pairs is the check, and the 21 concurrent pairs are why the relation is partial rather than total. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE HANDOFF · David Lee Wise (ROOT0), with AVAN


THE CAUSAL CUT

THE CAUSAL CUT · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE SYNC ◆ .dlw.fold THE FOLD / CO-OP / THE SYNC / THE CAUSAL CUT THE CAUSAL CUT the snapshot manufactures a present rather than finding one 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A snapshot of a distributed system is a line drawn across every timeline at once. Most such lines are nonsense: they catch a message arriving that has not yet been sent. LIT verified live. Two processes of 4 events each and 2 messages crossing between them give 25 possible cuts — exactly (4+1)×(4+1) , enumerated rather than sampled. 20 are consistent and 5 are not: 80.0% . The 5 impossible ones are not unlikely or rare, they are states the system can never have been in, and a naive snapshot will happily record one. 2 HOW IT WAS WEAVED · AI + HUMAN Chandy and Lamport ’s 1985 algorithm exists to take a cut that is consistent by construction, without stopping the system. AVAN (AI) enumerated the whole cut lattice rather than arguing about it, because the count is small enough to be exhaustive and exhaustive is a different kind of claim. The useful shape is that inconsistency is a corner of the space: cuts go wrong specifically where one process has advanced past a receive that the other has not yet sent. 3 ONE DIMENSION Every cut in the lattice, marked consistent or impossible. 4 TWO DIMENSIONS · INTERACTIVE Slide the cut and watch it become impossible. advance A ▶ advance B reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a lattice of possible presents. AVAN’s addition (the inverse-companion): the forward reading is that a consistent cut is a valid snapshot of the system. The inverse is that there are twenty of them and no way to prefer one . Each consistent cut is an equally real account of “now”, and the system was in all of them and none of them; the snapshot you take is a state that may never have existed at any single instant, only one that could have. Read backwards, a distributed system does not have a present that a snapshot discovers — the snapshot manufactures one, and consistency only means it manufactured a plausible one. pause spin LIT two processes of 4 events each with 2 messages crossing give 25 possible cuts - exactly 5 times 5, enumerated rather than sampled - of which 20 are consistent and 5 are not, and those 5 are not unlikely or rare but states the system can never have been in, which a naive snapshot will happily record FIG Chandy and Lamport's 1985 algorithm exists to take a cut that is consistent by construction, without stopping the system. AVAN enumerated the whole cut lattice rather than arguing about it, because the count is small enough to be exhaustive and exhaustive is a different kind of claim. The useful shape is that inconsistency is a corner of the space: cuts go wrong specifically where one process has advanced past a receive the other has not yet sent. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SYNC · David Lee Wise (ROOT0), with AVAN


THE TOTAL ORDER BROADCAST

Causal delivery guarantees you never see an effect before its cause. It says nothing about two messages with no cause between them - and replicas that disagree on those diverge while all behave correctly.

THE TOTAL ORDER BROADCAST · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE BROADCAST ◆ .dlw.fold THE FOLD / CO-OP / THE BROADCAST / THE TOTAL ORDER BROADCAST THE TOTAL ORDER BROADCAST a way of making everyone wrong in the same direction 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Causal delivery guarantees you never see an effect before its cause. It says nothing whatever about two messages that have no cause between them — and replicas that disagree on those diverge while every one of them is behaving correctly. LIT verified live. Two concurrent messages, 5 receivers, every delivery order enumerated: 32 in total. Causal order permits all 32 . Only 2 of them have every receiver agreeing — 6.25% . The other 30 are causally legal and leave the replicas in different states, which is exactly the gap total-order broadcast exists to close. 2 HOW IT WAS WEAVED · AI + HUMAN Total order broadcast is equivalent to consensus — it is not a stronger delivery guarantee bolted on, it is the same problem wearing different clothes. AVAN (AI) enumerated rather than reasoned, because 2 of 32 is a number and “causal is weaker than total” is a slogan. The ratio is the argument: agreement is not the common case that occasionally fails, it is 6.25% of the space, and everything else is a legal execution that has quietly forked your state. 3 ONE DIMENSION All thirty-two delivery orders. Two of them agree. 4 TWO DIMENSIONS · INTERACTIVE Add receivers and watch agreement collapse. more receivers ▶ fewer reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: one broadcast, many arrivals. AVAN’s addition (the inverse-companion): the forward reading is that total order broadcast fixes replica divergence. The inverse is that it fixes it by inventing an order that does not exist . The two messages are genuinely concurrent; no fact about the world says which came first, so agreeing on one is not discovering the truth, it is manufacturing a convention and committing to it. Read backwards, consensus is not a way of learning what happened — it is a way of making everyone wrong in the same direction, which turns out to be the only useful thing available. pause spin LIT two concurrent messages across 5 receivers give 32 delivery orders when every one is enumerated, all 32 permitted by causal order, and only 2 of them have every receiver agreeing - 6.25% - leaving 30 that are causally legal and leave the replicas in different states FIG Total order broadcast is equivalent to consensus - not a stronger delivery guarantee bolted on but the same problem wearing different clothes. AVAN enumerated rather than reasoned, because 2 of 32 is a number and causal is weaker than total is a slogan. The ratio is the argument: agreement is not the common case that occasionally fails, it is 6.25% of the space. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BROADCAST · David Lee Wise (ROOT0), with AVAN


THE FENCING TOKEN

A lock does not stop a client that holds it and then freezes. It wakes after its lease has gone, still believing it is the writer, and writes over whoever took over.

THE FENCING TOKEN · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · BOSS · THE GATEKEEPER ◆ .dlw.fold THE FOLD / BOSS / THE GATEKEEPER / THE FENCING TOKEN THE FENCING TOKEN a lock that needs fencing was never a lock 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A lock does not stop a client that already holds it and then freezes. It wakes up after its lease has gone, still believing it is the writer, and writes over whoever took over. A fencing token makes the storage refuse it. LIT verified live. Two clients, two steps each, all 6 interleavings enumerated. Without tokens, 2 of the 6 let a stale writer land a write over a newer one — schedules 0110 and 1001 . With monotonic tokens the storage rejects those writes: 2 rejections, 0 corruptions. The lock is identical in both; only the storage changed. 2 HOW IT WAS WEAVED · AI + HUMAN Fencing tokens are Martin Kleppmann ’s standard answer to distributed locks that assume a client is either alive or gone. AVAN (AI) defined corruption structurally after getting it wrong. My first model tracked which client “held” the lock, which is not what fencing protects — the holder field is exactly the thing that is unreliable. Corruption is a write LANDING with a token older than the highest the store has accepted, and once it is stated that way the guard makes it unreachable, which is the claim. 3 ONE DIMENSION All six interleavings, fenced and unfenced. 4 TWO DIMENSIONS · INTERACTIVE Step a schedule and watch the token decide. next schedule ▶ jump to a bad one toggle fencing 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a number that only ever goes up. AVAN’s addition (the inverse-companion): the forward reading is that fencing tokens make distributed locks safe. The inverse is that they move the lock into the storage and leave the lock service holding nothing . The guard is enforced at the write, by the resource, comparing numbers — the lock manager is now an advisory number-issuer whose failure cannot cause corruption because it was never the thing preventing it. Read backwards, a lock that needs fencing was never a lock; it was a hint, and the fence is where the mutual exclusion actually lives. pause spin LIT two clients of two steps each give 6 interleavings when all are enumerated, and without tokens 2 of them let a stale writer land a write over a newer one - schedules 0110 and 1001 - while with monotonic tokens the storage rejects those writes for 2 rejections and 0 corruptions, the lock being identical in both and only the storage changed FIG Fencing tokens are Martin Kleppmann's standard answer to distributed locks that assume a client is either alive or gone. AVAN defined corruption structurally after getting it wrong: my first model tracked which client held the lock, which is not what fencing protects - the holder field is exactly the thing that is unreliable. Corruption is a write LANDING with a token older than the highest the store accepted, and stated that way the guard makes it unreachable. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE GATEKEEPER · David Lee Wise (ROOT0), with AVAN


THE LEASE

A lease is a lock with an expiry, so a dead holder releases it without anyone asking. That works only if both parties agree what time it is - and they do not.

THE LEASE · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · LOOT · THE BOUNTY ◆ .dlw.fold THE FOLD / LOOT / THE BOUNTY / THE LEASE THE LEASE an assumption about clocks wearing the costume of a constant 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A lease is a lock with an expiry, so a dead holder releases it without anyone asking. That works only if both parties agree what time it is — and they do not. LIT verified live, by counting rather than asserting. A 10 -second lease with 0.5 s of skew each way: running both disciplines over one timeline at 10 ms resolution, the naive version has 1.00 seconds where two holders both believe they hold it. Waiting out the skew before taking over gives 0.00 . The guard costs 1.00 second of every lease — 90.0% usable instead of 100%. 2 HOW IT WAS WEAVED · AI + HUMAN Leases come from the Frangipani and Chubby lineage, and every implementation carries a skew constant somewhere. AVAN (AI) nearly shipped this asserted. My first version wrote 0 into the guarded-overlap column for every row rather than measuring it — the same mistake as declaring a fairness result true. Rebuilt to simulate both disciplines over one timeline and count overlapping instants, the zero is now a measurement, and it means something because the other column is 1.00 . 3 ONE DIMENSION Overlap against skew, both disciplines. 4 TWO DIMENSIONS · INTERACTIVE Increase the skew and watch the safe window shrink. more skew ▶ less toggle guard reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two holders and a gap between them. AVAN’s addition (the inverse-companion): the forward reading is that waiting out the skew makes leases safe. The inverse is that you are paying for a number nobody can verify . The guard is exactly as good as the skew bound you assumed, and no participant can check that bound from inside — a clock that is further out than promised produces overlap silently, with every component behaving correctly. Read backwards, a lease does not convert time into safety; it converts an assumption about clocks into an assumption about correctness, and hides the substitution inside a constant. pause spin LIT a 10-second lease with 0.5 s of skew each way, run as both disciplines over one timeline at 10 ms resolution, gives 1.00 seconds where two holders both believe they hold it under the naive rule and 0.00 under the rule that waits out the skew - a guard costing 1.00 second of every lease, leaving 90.0% usable instead of 100% FIG Leases come from the Frangipani and Chubby lineage, and every implementation carries a skew constant somewhere. AVAN nearly shipped this asserted: my first version wrote 0 into the guarded-overlap column for every row rather than measuring it, the same mistake as declaring a fairness result true. Rebuilt to simulate both disciplines over one timeline and count overlapping instants, the zero is now a measurement, and it means something because the other column is 1.00. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BOUNTY · David Lee Wise (ROOT0), with AVAN


THE QUORUM INTERSECTION

Quorums work for one reason and it is arithmetic: any two sets larger than half of a whole must share a member. That member is the only thing carrying information between one decision and the next.

THE QUORUM INTERSECTION · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CO-OP · THE MERGE ◆ .dlw.fold THE FOLD / CO-OP / THE MERGE / THE QUORUM INTERSECTION THE QUORUM INTERSECTION the guarantee is one node wide 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Quorums work for one reason and it is arithmetic, not engineering: any two sets larger than half of a whole must share a member. That shared member is the only thing carrying information between one decision and the next. LIT verified live and exhaustively. 9 nodes, majority 5 , giving 126 possible quorums and 7,875 pairs of them. Every single pair intersects — 7,875 of 7,875 — and the smallest overlap found is 1 , matching 2k−n exactly. Drop to half rather than a majority and it fails: of 2,415 pairs of 4-node sets over 8 nodes, 35 are completely disjoint. 2 HOW IT WAS WEAVED · AI + HUMAN Quorum intersection underpins Paxos, Raft and every replicated log; Gifford ’s weighted voting (1979) is the general form. AVAN (AI) enumerated both the property and its boundary. Showing that majorities intersect is easy and unconvincing on its own — the claim only has content if the neighbouring case fails, so the 35 disjoint pairs at half-size are the load-bearing number. The minimum overlap of exactly 1 is the other half: a majority quorum guarantees one witness, not a comfortable margin. 3 ONE DIMENSION Quorum size against guaranteed overlap. 4 TWO DIMENSIONS · INTERACTIVE Shrink the quorum until the guarantee breaks. smaller quorum ▶ bigger more nodes reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two sets forced to share. AVAN’s addition (the inverse-companion): the forward reading is that majority quorums guarantee consistency. The inverse is that the guarantee is one node wide . 2k−n is 1 here: consistency between two decisions rests on a single machine remembering correctly, and every additional node you add buys availability rather than overlap. Read backwards, a quorum system is not redundant where it matters most — it is maximally redundant about failure and minimally redundant about truth. pause spin LIT 9 nodes with a majority of 5 give 126 possible quorums and 7,875 pairs, every single pair intersecting - 7,875 of 7,875 - with a smallest overlap of 1 matching 2k-n exactly; drop to half rather than a majority and it fails, since of 2,415 pairs of 4-node sets over 8 nodes, 35 are completely disjoint FIG Quorum intersection underpins Paxos, Raft and every replicated log; Gifford's weighted voting (1979) is the general form. AVAN enumerated both the property and its boundary. Showing that majorities intersect is easy and unconvincing alone - the claim only has content if the neighbouring case fails, so the 35 disjoint pairs at half-size are the load-bearing number. The minimum overlap of exactly 1 is the other half: a majority quorum guarantees one witness, not a comfortable margin. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE MERGE · David Lee Wise (ROOT0), with AVAN


THE HYBRID CLOCK

A logical clock captures causality and drifts from wall time. A physical clock reads like wall time and captures no causality. A hybrid clock refuses to choose.

THE HYBRID CLOCK · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · SPAWN · CHECKPOINT ZERO ◆ .dlw.fold THE FOLD / SPAWN / CHECKPOINT ZERO / THE HYBRID CLOCK THE HYBRID CLOCK the honesty lives in the part nobody prints 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION A logical clock captures causality and drifts arbitrarily far from wall time. A physical clock reads like wall time and captures no causality. A hybrid clock refuses to choose. LIT verified live. 3 processes, 600 events, physical clocks advancing unevenly and messages crossing between them. The hybrid clock records 0 causality violations — every receive strictly follows its send — while never diverging from physical time by more than 7 units. The logical counter stays in single digits: it only increments when the physical clock fails to, so it never runs away. 2 HOW IT WAS WEAVED · AI + HUMAN Hybrid Logical Clocks are Kulkarni, Demirbas, Madappa, Avva and Leone (2014), and they are what CockroachDB and MongoDB use to timestamp transactions. AVAN (AI) checked both halves on the same run, because either alone is trivially achievable and worthless. A clock that never violates causality can be a plain Lamport counter with no relation to wall time; one that tracks wall time can ignore causality entirely. 0 violations and a bounded divergence of 7 is the only interesting statement, and both come from one execution. 3 ONE DIMENSION Physical time, and the hybrid clock tracking it. 4 TWO DIMENSIONS · INTERACTIVE Run the execution and watch both properties hold. more events ▶ fewer reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: two clocks in one word. AVAN’s addition (the inverse-companion): the forward reading is that a hybrid clock gives you causality and wall time together. The inverse is that it gives you a timestamp that is not quite either, and says so in a field most readers drop . The counter is the confession — it is nonzero exactly when the physical part is a fiction the clock was forced to keep, and it is the first thing truncated when the value is logged or compared as a number. Read backwards, the honesty of this clock lives entirely in the part nobody prints. pause spin LIT 3 processes over 600 events with unevenly advancing physical clocks and messages crossing record 0 causality violations - every receive strictly following its send - while never diverging from physical time by more than 7 units, and the logical counter stays in single digits because it only increments when the physical clock fails to FIG Hybrid Logical Clocks are Kulkarni, Demirbas, Madappa, Avva and Leone (2014), and they are what CockroachDB and MongoDB use to timestamp transactions. AVAN checked both halves on the same run, because either alone is trivially achievable and worthless: a clock that never violates causality can be a plain Lamport counter with no relation to wall time, and one that tracks wall time can ignore causality entirely. 0 violations AND a bounded divergence of 7 is the only interesting statement. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of CHECKPOINT ZERO · David Lee Wise (ROOT0), with AVAN


THE FMA

Multiply then add and the machine rounds twice. A fused multiply-add keeps the exact product and rounds only at the end.

THE FMA · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · CHEAT · THE SHORTCUT ◆ .dlw.fold THE FOLD / CHEAT / THE SHORTCUT / THE FMA THE FMA accuracy bought with reproducibility 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION Multiply then add, and the machine rounds twice: once to store the product, once to store the sum. A fused multiply-add keeps the exact product and rounds only at the end. LIT verified live. Over 200,000 random triples, a*b+c and the fused result differ 22,469 times — 11.23% . The clearest case is a 2×2 determinant: for [1e8+1, 1e8; 1e8, 1e8-1] the true answer is −1 . Fused arithmetic returns −1 . Naive arithmetic returns 0 — not close to wrong, but the wrong sign of nothing at all. 2 HOW IT WAS WEAVED · AI + HUMAN FMA is in IEEE 754-2008 and in every modern instruction set; the exact product here is recovered with Dekker ’s splitting, which is how you get it without hardware help. AVAN (AI) chose a determinant rather than a percentage as the headline, because 11.23% only says the two disagree. Returning 0 for a matrix whose determinant is −1 is a different category of failure: the answer is not imprecise, it has lost the fact that the matrix is invertible at all. 3 ONE DIMENSION Two roundings, and one. 4 TWO DIMENSIONS · INTERACTIVE Bring the two products closer and watch naive arithmetic fail. closer ▶ further apart reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a product kept whole. AVAN’s addition (the inverse-companion): the forward reading is that fusing removes a rounding and improves accuracy. The inverse is that it makes the same expression mean two things . a*b+c now depends on whether the compiler chose to fuse it, so a program can produce different results on the same inputs with the same source on the same machine at a different optimisation level. Read backwards, the accuracy was bought with reproducibility, and every language that later added a “do not fuse” pragma was buying it back. pause spin LIT over 200,000 random triples a*b+c and the fused result differ 22,469 times - 11.23% - and for the 2x2 determinant of [1e8+1, 1e8; 1e8, 1e8-1] whose true value is -1, fused arithmetic returns -1 while naive arithmetic returns 0, which is not close to wrong but the wrong sign of nothing at all FIG FMA is in IEEE 754-2008 and in every modern instruction set; the exact product here is recovered with Dekker's splitting, which is how you get it without hardware help. AVAN chose a determinant rather than a percentage as the headline, because 11.23% only says the two disagree. Returning 0 for a matrix whose determinant is -1 is a different category of failure: the answer is not imprecise, it has lost the fact that the matrix is invertible at all. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SHORTCUT · David Lee Wise (ROOT0), with AVAN


THE DECIMAL VS BINARY

One tenth has no exact binary representation, for the same reason one third has no exact decimal one. Every currency figure you have added in a float was approximate before you touched it.

THE DECIMAL VS BINARY · WORLD II — THE FOLD ◀ THE FOLD 0ROOT.AI // WORLD II · GLITCH · HEISENBUG ◆ .dlw.fold THE FOLD / GLITCH / HEISENBUG / THE DECIMAL VS BINARY THE DECIMAL VS BINARY a translation defect, not a precision one 1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION One tenth has no exact binary representation, for the same reason one third has no exact decimal one. Every currency figure you have ever added in a float was approximate before you touched it. LIT verified live and exhaustively. All 10,000 sums of two two-decimal values were tested against the exact decimal answer. 2,106 of them disagree — 21.06% . This is not a corner case selected to embarrass the format; it is one sum in five. 0.1 + 0.2 gives 0.30000000000000004 , and 0.01 + 0.05 gives 0.060000000000000005 . 2 HOW IT WAS WEAVED · AI + HUMAN This is why financial systems use decimal types or integer cents, and why 0.1 + 0.2 is the most famous three characters in floating point. AVAN (AI) enumerated the whole space rather than quoting the famous example, because the famous example invites the response that it is a curiosity. 2,106 of 10,000 is not a curiosity. The failure rate is the finding, and the celebrated case is simply one of two thousand. 3 ONE DIMENSION The full grid. Every red cell is a sum that misses. 4 TWO DIMENSIONS · INTERACTIVE Pick a pair and see the exact bits you got. next failing pair ▶ famous one reset 5 THREE DIMENSIONS + AVAN’S INVERSE The green forward object: a grid mostly right and reliably wrong. AVAN’s addition (the inverse-companion): the forward reading is that binary floats cannot represent decimal fractions. The inverse is that they represent them perfectly and we are asking the wrong question . The stored value is the exact binary number nearest one tenth, and it is stored, added and returned with total fidelity; what fails is the assumption that a decimal string and a binary float are the same object. Read backwards, this is not a precision defect but a translation one, and it happens at the boundary where a human writes 0.1 and a machine agrees to pretend it heard that. pause spin LIT all 10,000 sums of two two-decimal values tested against the exact decimal answer give 2,106 disagreements - 21.06%, one sum in five and not a corner case selected to embarrass the format - with 0.1 + 0.2 giving 0.30000000000000004 and 0.01 + 0.05 giving 0.060000000000000005 FIG This is why financial systems use decimal types or integer cents, and why 0.1 + 0.2 is the most famous three characters in floating point. AVAN enumerated the whole space rather than quoting the famous example, because the famous example invites the response that it is a curiosity. 2,106 of 10,000 is not a curiosity - the failure rate is the finding, and the celebrated case is simply one of two thousand. ◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of HEISENBUG · David Lee Wise (ROOT0), with AVAN