THE FOLD / CHEAT / THE SPEEDRUN / THE BITPACKING

THE BITPACKING

the byte boundary was never waste, it was an index

1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION

If every value fits in five bits, storing each in a byte wastes three bits per value. Bit-packing ignores byte boundaries entirely and lays the values end to end.

LIT verified live. 20,000 five-bit values pack into 12,500 bytes — exactly ceil(20000×5/8), not approximately — against 20,000 at a byte each and 80,000 raw. That is 1.6× against byte-aligned and 6.4× against 32-bit. All 20,000 unpack to their original values with 0 errors, which is the part worth checking, because a packer that loses the last partial value would still produce a good ratio.

2 HOW IT WAS WEAVED · AI + HUMAN

Bit-packing sits under every column store and inverted index, usually applied after frame-of-reference has made the values small.

AVAN (AI) checked the round trip and the exact byte count together. The byte count matching ceil(N×W/8) proves nothing was silently padded per value; the 0 unpack errors prove nothing was dropped at the tail. Either check alone passes for an implementation that is quietly broken in the other direction.

3 ONE DIMENSION

Five-bit values crossing byte boundaries.

4 TWO DIMENSIONS · INTERACTIVE

Change the width and watch the packing change shape.

5 THREE DIMENSIONS + AVAN’S INVERSE

The green forward object: values that ignore the byte.

AVAN’s addition (the inverse-companion): the forward reading is that bit-packing removes the padding waste. The inverse is that the byte boundary was never waste — it was an index. Byte-aligned data can be addressed, sliced, memory-mapped and read by anything; packed data must be decoded from a known start before any single value can be found. Read backwards, the three wasted bits per value were paying for random access, and the 1.6× is the price of that access rather than a free saving.

LIT 20,000 five-bit values pack into 12,500 bytes - exactly ceil(20000 x 5/8), not approximately - against 20,000 at a byte each and 80,000 raw, giving 1.6x against byte-aligned and 6.4x against 32-bit, and all 20,000 unpack to their original values with 0 errors

FIG Bit-packing sits under every column store and inverted index, usually applied after frame-of-reference has made the values small. AVAN checked the round trip and the exact byte count together: the byte count matching ceil(N x W/8) proves nothing was silently padded per value, and the 0 unpack errors prove nothing was dropped at the tail. Either check alone passes for an implementation quietly broken in the other direction.

◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SPEEDRUN · David Lee Wise (ROOT0), with AVAN