THE FOLD / CHEAT / THE KONAMI CODE / THE SIMHASH
THE SIMHASH
similarity read from sign bits
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
SimHash turns similarity into a handful of bits. Pick random hyperplanes through the origin; for a vector v, record one bit per hyperplane — which side v falls on, sign(v·r). The magic: for two vectors u, v at angle θ, a random hyperplane separates them with probability exactly θ/π. So the Hamming distance between their sign-bit sketches, over many hyperplanes, estimates the angle between them — near-duplicate detection in a fixed-size fingerprint, the trick behind web-scale de-duplication.
LIT verified live: with rotationally-symmetric (Gaussian) hyperplanes, the fraction of differing sign bits matches θ/π to within sampling error over many random pairs (window.__simhash). FIG no framing; the sign bits and the angle estimate are computed in-browser.
LIT verified live: with rotationally-symmetric (Gaussian) hyperplanes, the fraction of differing sign bits matches θ/π to within sampling error over many random pairs (window.__simhash). FIG no framing; the sign bits and the angle estimate are computed in-browser.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at the-konami-code — a short secret sequence of bits that unlocks a document’s identity; two near-duplicates share almost the same code. AVAN (AI) built the instrument: Gaussian random hyperplanes, the sign-bit sketch, and the θ/π estimate against the true angle.
Credit as content: Moses Charikar (2002), from the Goemans–Williamson random-hyperplane rounding. Honest note: the θ/π law needs rotationally-symmetric normals — Gaussian, not uniform-in-a-box. The weave: David names the code; I confirm the differing-bit rate equals θ/π.
Credit as content: Moses Charikar (2002), from the Goemans–Williamson random-hyperplane rounding. Honest note: the θ/π law needs rotationally-symmetric normals — Gaussian, not uniform-in-a-box. The weave: David names the code; I confirm the differing-bit rate equals θ/π.
3 ONE DIMENSION
Two vectors at angle θ; the shaded wedge of directions that separate them spans θ out of π — so a random hyperplane splits them with probability θ/π.
4 TWO DIMENSIONS · INTERACTIVE
Set the angle between two vectors; the measured fraction of differing sign bits tracks θ/π as you add hyperplanes.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: the sign-bit sketch of a vector.
AVAN’s addition (the inverse-companion): don’t compare the vectors — compare their bits. The inverse of ‘compute the angle between u and v’ is ‘count how many sign bits differ; that fraction is θ/π.’ Magenta is a hyperplane that separates the pair; green is the recovered angle. Similarity read from sign bits.
LIT Genuine SimHash / random-hyperplane LSH (Moses Charikar, 2002, from Goemans–Williamson rounding): P(sign bit differs) = θ/π. Verified live: with Gaussian hyperplanes, the differing-bit fraction over 200 random pairs (20k hyperplanes each) matches θ/π to within ~0.01 (window.__simhash.angleEstimate).
FIG Honest scope stated on the sphere: the θ/π law requires rotationally-symmetric normals — Gaussian, not uniform-in-a-box (a uniform-cube normal gives a biased estimate). The AVAN inverse is honest — recovering the angle from how many sign bits differ (rather than computing u·v directly) is the whole point of the sketch; magenta is a separating hyperplane, green the recovered angle. Similarity read from sign bits.
FIG Honest scope stated on the sphere: the θ/π law requires rotationally-symmetric normals — Gaussian, not uniform-in-a-box (a uniform-cube normal gives a biased estimate). The AVAN inverse is honest — recovering the angle from how many sign bits differ (rather than computing u·v directly) is the whole point of the sketch; magenta is a separating hyperplane, green the recovered angle. Similarity read from sign bits.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE KONAMI CODE · David Lee Wise (ROOT0), with AVAN