THE FOLD / GRIND / THE EPOCH / THE DELTA OF DELTA
THE DELTA OF DELTA
an operational health metric wearing a storage costume
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Timestamps arriving at a regular interval have constant first differences, so their second differences are almost all zero. Store those, and the common case costs nothing at all.
LIT verified live. 100,000 timestamps at a nominal ten-second cadence with occasional jitter. 93.42% of second differences are exactly 0. The deltas need 5 bits; the delta-of-deltas need 4 — against 32 raw, an 8× reduction. Every value reconstructs exactly: 0 errors across all 100,000, because the transform is subtraction twice and nothing is approximated.
LIT verified live. 100,000 timestamps at a nominal ten-second cadence with occasional jitter. 93.42% of second differences are exactly 0. The deltas need 5 bits; the delta-of-deltas need 4 — against 32 raw, an 8× reduction. Every value reconstructs exactly: 0 errors across all 100,000, because the transform is subtraction twice and nothing is approximated.
2 HOW IT WAS WEAVED · AI + HUMAN
Delta-of-delta is the timestamp half of Facebook’s Gorilla paper (2015) and is now standard in time-series stores.
AVAN (AI) reports the 93.42% rather than the ratio, because the ratio is a consequence and the zero-fraction is the mechanism. A coder that spends a single bit on “same as last time” converts a regular cadence into almost nothing — and the moment the cadence stops being regular, the whole advantage disappears with it.
AVAN (AI) reports the 93.42% rather than the ratio, because the ratio is a consequence and the zero-fraction is the mechanism. A coder that spends a single bit on “same as last time” converts a regular cadence into almost nothing — and the moment the cadence stops being regular, the whole advantage disappears with it.
3 ONE DIMENSION
Values, deltas, and delta-of-deltas.
4 TWO DIMENSIONS · INTERACTIVE
Add jitter and watch the zeros disappear.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: a difference of a difference.
AVAN’s addition (the inverse-companion): the forward reading is that delta-of-delta exploits regular cadence. The inverse is that it is a bet on a machine behaving well, stored in your data format. The 93.42% is a measurement of how reliable somebody’s scheduler was, and a garbage collection pause or a clock correction converts your cheapest column into your most expensive one. Read backwards, the compression ratio of a time series is an operational health metric wearing a storage costume.
LIT 100,000 timestamps at a nominal ten-second cadence with occasional jitter give 93.42% of second differences exactly 0, needing 4 bits where the deltas need 5 and the raw values need 32 - an 8x reduction - with every value reconstructing exactly and 0 errors across all 100,000
FIG Delta-of-delta is the timestamp half of Facebook's Gorilla paper (2015) and is now standard in time-series stores. AVAN reports the 93.42% rather than the ratio, because the ratio is a consequence and the zero-fraction is the mechanism. A coder spending a single bit on same-as-last-time converts a regular cadence into almost nothing - and the moment the cadence stops being regular the whole advantage goes with it.
FIG Delta-of-delta is the timestamp half of Facebook's Gorilla paper (2015) and is now standard in time-series stores. AVAN reports the 93.42% rather than the ratio, because the ratio is a consequence and the zero-fraction is the mechanism. A coder spending a single bit on same-as-last-time converts a regular cadence into almost nothing - and the moment the cadence stops being regular the whole advantage goes with it.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE EPOCH · David Lee Wise (ROOT0), with AVAN