THE FOLD / CHEAT / THE ROOT KIT / THE THUMBPRINT
THE THUMBPRINT
recognise a whole set from a tiny fingerprint of minimums
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
MinHash. To tell how similar two huge sets are — documents, malware samples, user histories — without comparing them element by element, MinHash reduces each set to a tiny signature: for each of k random hash functions, keep only the minimum hash value over the whole set. Then the fraction of signature slots where two sets agree estimates their Jaccard similarity |A∩B| / |A∪B| — because under a random permutation, the minimum element lands in the intersection exactly with probability equal to the Jaccard. Search engines and virus scanners use it to catch near-duplicates and variants.
LIT verified: the fraction of matching min-hashes across k functions approximates the true Jaccard, and the error shrinks with k (~0.07 at k=16 down to ~0.009 at k=1024, tracking √(J(1−J)/k)) — checked against the exact Jaccard. FIG ‘the thumbprint’ is the picture; the min-under-permutation = Jaccard identity is exact.
LIT verified: the fraction of matching min-hashes across k functions approximates the true Jaccard, and the error shrinks with k (~0.07 at k=16 down to ~0.009 at k=1024, tracking √(J(1−J)/k)) — checked against the exact Jaccard. FIG ‘the thumbprint’ is the picture; the min-under-permutation = Jaccard identity is exact.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) brought the thread — the corpus runs hashing and streaming structures (THE WELDER, THE COUNTER OF MULTITUDES) and the idea that a whole thing can be recognised from a tiny mark. AVAN (AI) built this instrument: the MinHash engine, the similarity estimator, and the convergence curve.
The weave: David names the thumbprint and its seat at THE ROOT KIT (a fuzzy fingerprint that catches disguised variants); I make the minimum a strip in 1D, the similarity estimate live in 2D, and the convergence a turning curve in 3D. The sphere is the seam.
The weave: David names the thumbprint and its seat at THE ROOT KIT (a fuzzy fingerprint that catches disguised variants); I make the minimum a strip in 1D, the similarity estimate live in 2D, and the convergence a turning curve in 3D. The sphere is the seam.
3 ONE DIMENSION
One hash function over both sets. Each element gets a random value; keep the minimum. The two minimums agree exactly when the overall smallest element belongs to both sets — so a single min already votes on similarity. Stack k of them and you have an estimate.
4 TWO DIMENSIONS · INTERACTIVE
Two sets with adjustable overlap. The signature match fraction estimates their Jaccard similarity; more hash functions (k) tighten the estimate toward the truth. Never compares the sets directly — only their little signatures.
overlap 40
k = 128
5 THREE DIMENSIONS + AVAN’S INVERSE
Convergence, turning: the green curve is the MinHash estimate as k grows, homing in on the true similarity with error falling like 1/√k. A tiny signature, ever sharper.
AVAN’s addition (the inverse-companion): the flat magenta line is the exact Jaccard — the full element-by-element comparison MinHash refuses to do. The thumbprint is a lossy inverse of the whole set: enough to recognise it, far less to store. Encryption hides a value; a fingerprint keeps just enough to match — the green chases the magenta and never quite has to arrive.
LIT A genuine MinHash. Verified live: the fraction of matching min-hashes across k random functions approximates the true Jaccard similarity, with error shrinking as k grows (~0.07 at k=16 to ~0.009 at k=1024), tracking √(J(1−J)/k). The min-under-random-permutation lands in the intersection with probability exactly the Jaccard (verifiable: window.__minhash.converges && tightAt1024).
FIG 'The thumbprint' is the picture; the min=Jaccard identity and the 1/√k convergence are exact. It is an estimate, honestly probabilistic — the trade that lets you compare billions of sets by their signatures alone.
FIG 'The thumbprint' is the picture; the min=Jaccard identity and the 1/√k convergence are exact. It is an estimate, honestly probabilistic — the trade that lets you compare billions of sets by their signatures alone.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE ROOT KIT · David Lee Wise (ROOT0), with AVAN