THE FOLD / SPAWN / THE TOOLCHAIN / THE SURROGATE PAIR
THE SURROGATE PAIR
two units that are not characters
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Sixteen bits held every character, once. Then there were more than sixty-five thousand of them, and the fix was to spend two units on the rest — two units that are not characters, and mean nothing apart.
LIT verified live. the musical clef U+1D11E occupies 2 UTF-16 code units and is 1 code point. Sampling 768 code points across the range, 512 need two units and 256 fit in one. Cutting the string
LIT verified live. the musical clef U+1D11E occupies 2 UTF-16 code units and is 1 code point. Sampling 768 code points across the range, 512 need two units and 256 fit in one. Cutting the string
a 𝄞 b 😀 c at every one of its 8 indices produces a lone high surrogate 2 times — a value that is not a character, cannot be rendered, and is still a perfectly legal string. The string reports length 7; it holds 5 characters.2 HOW IT WAS WEAVED · AI + HUMAN
UTF-16 and the surrogate range U+D800–U+DFFF are how a 16-bit encoding was extended past the basic plane without breaking existing data.
AVAN (AI) counted the lone surrogates produced by slicing rather than asserting that slicing is unsafe, because 2 of 8 is the shape of the hazard: not every cut is dangerous, so a test that slices once will usually pass. The type system is silent throughout — a lone surrogate has the same type as any other string, and there is no operation that will complain until something tries to draw it.
AVAN (AI) counted the lone surrogates produced by slicing rather than asserting that slicing is unsafe, because 2 of 8 is the shape of the hazard: not every cut is dangerous, so a test that slices once will usually pass. The type system is silent throughout — a lone surrogate has the same type as any other string, and there is no operation that will complain until something tries to draw it.
3 ONE DIMENSION
Seven long, five characters, two bad places to cut.
4 TWO DIMENSIONS · INTERACTIVE
Cut the string at each index and find the broken halves.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object.
AVAN’s addition (the inverse-companion): the forward reading is that you must not index into UTF-16 blindly. The inverse is that the encoding leaked into the type, and then the type became the interface. A
string was supposed to be a sequence of characters; it is a sequence of storage units, and every language that adopted UTF-16 in the nineties made that permanent by exposing length and [i] in those units. Read backwards, this is a twenty-year-old compatibility decision still being paid for in every truncated name and every mangled emoji, by people who never chose the encoding.LIT the musical clef U+1D11E occupies 2 UTF-16 code units and is 1 code point, and sampling 768 code points across the range, 512 need two units while 256 fit in one; cutting a five-character test string at every one of its 8 indices produces a lone high surrogate 2 times - a value that is not a character, cannot be rendered, and is still a perfectly legal string - and that string reports length 7 while holding 5 characters
FIG UTF-16 and the surrogate range U+D800 to U+DFFF are how a 16-bit encoding was extended past the basic plane without breaking existing data. AVAN counted the lone surrogates produced by slicing rather than asserting that slicing is unsafe, because 2 of 8 is the shape of the hazard: not every cut is dangerous, so a test that slices once will usually pass. The type system is silent throughout - a lone surrogate has the same type as any other string.
FIG UTF-16 and the surrogate range U+D800 to U+DFFF are how a 16-bit encoding was extended past the basic plane without breaking existing data. AVAN counted the lone surrogates produced by slicing rather than asserting that slicing is unsafe, because 2 of 8 is the shape of the hazard: not every cut is dangerous, so a test that slices once will usually pass. The type system is silent throughout - a lone surrogate has the same type as any other string.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE TOOLCHAIN · David Lee Wise (ROOT0), with AVAN