THE FOLD / BOSS / THE CHOKE POINT / THE BACKPRESSURE
THE BACKPRESSURE
an invisible slow failure made a visible fast one
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
An unbounded queue never rejects anything, which sounds generous until you notice what it is doing instead: accepting work it will not get to for hours, and making everyone wait behind it.
LIT verified live. 20,000 ticks, offered above capacity. Unbounded: 0 rejected, final queue 6,917 deep, mean wait 3,461.9. Bounded at 100: 6,818 shed, final queue 99, mean wait 99.4 — 34.8× shorter. And the throughput is identical: 20,000 served either way, a difference of 0. The unbounded queue served nobody extra; it only made the served ones wait.
LIT verified live. 20,000 ticks, offered above capacity. Unbounded: 0 rejected, final queue 6,917 deep, mean wait 3,461.9. Bounded at 100: 6,818 shed, final queue 99, mean wait 99.4 — 34.8× shorter. And the throughput is identical: 20,000 served either way, a difference of 0. The unbounded queue served nobody extra; it only made the served ones wait.
2 HOW IT WAS WEAVED · AI + HUMAN
This is Little’s Law with a policy attached: throughput is set by the server, and the queue only decides how long the wait is.
AVAN (AI) put the throughput columns side by side because that is the whole argument and it is the column people expect to differ. 20,000 and 20,000. Accepting the extra work bought exactly nothing, and the 6,818 requests that were shed would have waited an hour to be told the same thing.
AVAN (AI) put the throughput columns side by side because that is the whole argument and it is the column people expect to differ. 20,000 and 20,000. Accepting the extra work bought exactly nothing, and the 6,818 requests that were shed would have waited an hour to be told the same thing.
3 ONE DIMENSION
Two queues, one server, same arrivals.
4 TWO DIMENSIONS · INTERACTIVE
Set the bound and watch the wait, not the throughput.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: a queue that says no.
AVAN’s addition (the inverse-companion): the forward reading is that backpressure protects the system. The inverse is that it moves the failure to where somebody can see it. The unbounded queue fails too — it fails as a timeout, in a client, minutes later, with no error anywhere in your logs. Read backwards, shedding does not reduce the number of unhappy users by one; it changes an invisible slow failure into a visible fast one, and every argument against it is really an argument for not being the one who says no.
LIT 20,000 ticks offered above capacity give an unbounded queue 0 rejected, a final depth of 6,917 and a mean wait of 3,461.9, against a queue bounded at 100 which sheds 6,818, ends 99 deep and waits 99.4 - 34.8 times shorter - while serving the identical 20,000, a throughput difference of 0
FIG This is Little's Law with a policy attached: throughput is set by the server, and the queue only decides how long the wait is. AVAN put the throughput columns side by side because that is the whole argument and it is the column people expect to differ. 20,000 and 20,000. Accepting the extra work bought exactly nothing, and the 6,818 requests that were shed would have waited an hour to be told the same thing.
FIG This is Little's Law with a policy attached: throughput is set by the server, and the queue only decides how long the wait is. AVAN put the throughput columns side by side because that is the whole argument and it is the column people expect to differ. 20,000 and 20,000. Accepting the extra work bought exactly nothing, and the 6,818 requests that were shed would have waited an hour to be told the same thing.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE CHOKE POINT · David Lee Wise (ROOT0), with AVAN