THE FOLD / CO-OP / THE BROADCAST / THE HEDGED REQUEST
THE HEDGED REQUEST
a loan against idle capacity
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Send the request. If it has not come back by the 95th percentile, send a second copy elsewhere and take whichever answers first. You pay about five percent more traffic and you buy back most of the tail.
LIT verified live. 200,000 requests, hedge fired at 8.76 ms. Extra load: 9,999 requests, 5.00%. p99 falls from 123.31 ms to 13.80. p99.9 falls from 192.49 to 17.50 — an 11× cut. The maximum barely moves: 199.99 to 187.74, a factor of 1.07, because a hedge can be unlucky twice.
LIT verified live. 200,000 requests, hedge fired at 8.76 ms. Extra load: 9,999 requests, 5.00%. p99 falls from 123.31 ms to 13.80. p99.9 falls from 192.49 to 17.50 — an 11× cut. The maximum barely moves: 199.99 to 187.74, a factor of 1.07, because a hedge can be unlucky twice.
2 HOW IT WAS WEAVED · AI + HUMAN
Hedged requests are the practical half of Dean and Barroso’s tail-at-scale paper; the version that only fires after a percentile delay is what keeps the extra load small.
AVAN (AI) measured the part that gets left out of the pitch. Hedging is sold on the p99, and the p99 does collapse. But the maximum improves by only 1.07×, because the second copy is drawn from the same distribution and can land in the same tail. Read the two numbers together and the technique is honest: it moves the bulk of the tail, and it does almost nothing for the worst case.
AVAN (AI) measured the part that gets left out of the pitch. Hedging is sold on the p99, and the p99 does collapse. But the maximum improves by only 1.07×, because the second copy is drawn from the same distribution and can land in the same tail. Read the two numbers together and the technique is honest: it moves the bulk of the tail, and it does almost nothing for the worst case.
3 ONE DIMENSION
The tail, before and after. Note where the two curves meet again.
4 TWO DIMENSIONS · INTERACTIVE
Move the hedge point and watch load trade against tail.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: two attempts, one answer.
AVAN’s addition (the inverse-companion): the forward reading is that hedging buys tail latency for a few percent of load. The inverse is that it spends the one resource that made the tail short in the first place. The hedge is fast because the system is not saturated; add hedges everywhere and utilisation rises, and by
1/(1−ρ) the tail you were hedging against grows. Read backwards, hedging is a loan against idle capacity, and like every such loan it works beautifully until enough people take it out at once.LIT 200,000 requests with the hedge firing at 8.76 ms cost 9,999 extra requests, 5.00% more traffic, and moved p99 from 123.31 ms to 13.80 and p99.9 from 192.49 to 17.50 - an 11x cut - while the maximum barely moved, 199.99 to 187.74, a factor of only 1.07, because a hedge can be unlucky twice
FIG Hedged requests are the practical half of Dean and Barroso's tail-at-scale paper; firing only after a percentile delay is what keeps the extra load small. AVAN measured the part that gets left out of the pitch: hedging is sold on the p99 and the p99 does collapse, but the maximum improves by only 1.07x because the second copy is drawn from the same distribution and can land in the same tail. Read together, the technique moves the bulk of the tail and does almost nothing for the worst case.
FIG Hedged requests are the practical half of Dean and Barroso's tail-at-scale paper; firing only after a percentile delay is what keeps the extra load small. AVAN measured the part that gets left out of the pitch: hedging is sold on the p99 and the p99 does collapse, but the maximum improves by only 1.07x because the second copy is drawn from the same distribution and can land in the same tail. Read together, the technique moves the bulk of the tail and does almost nothing for the worst case.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE BROADCAST · David Lee Wise (ROOT0), with AVAN