THE FOLD / CHEAT / THE SHORTCUT / THE FMA
THE FMA
accuracy bought with reproducibility
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Multiply then add, and the machine rounds twice: once to store the product, once to store the sum. A fused multiply-add keeps the exact product and rounds only at the end.
LIT verified live. Over 200,000 random triples,
LIT verified live. Over 200,000 random triples,
a*b+c and the fused result differ 22,469 times — 11.23%. The clearest case is a 2×2 determinant: for [1e8+1, 1e8; 1e8, 1e8-1] the true answer is −1. Fused arithmetic returns −1. Naive arithmetic returns 0 — not close to wrong, but the wrong sign of nothing at all.2 HOW IT WAS WEAVED · AI + HUMAN
FMA is in IEEE 754-2008 and in every modern instruction set; the exact product here is recovered with Dekker’s splitting, which is how you get it without hardware help.
AVAN (AI) chose a determinant rather than a percentage as the headline, because 11.23% only says the two disagree. Returning 0 for a matrix whose determinant is −1 is a different category of failure: the answer is not imprecise, it has lost the fact that the matrix is invertible at all.
AVAN (AI) chose a determinant rather than a percentage as the headline, because 11.23% only says the two disagree. Returning 0 for a matrix whose determinant is −1 is a different category of failure: the answer is not imprecise, it has lost the fact that the matrix is invertible at all.
3 ONE DIMENSION
Two roundings, and one.
4 TWO DIMENSIONS · INTERACTIVE
Bring the two products closer and watch naive arithmetic fail.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: a product kept whole.
AVAN’s addition (the inverse-companion): the forward reading is that fusing removes a rounding and improves accuracy. The inverse is that it makes the same expression mean two things.
a*b+c now depends on whether the compiler chose to fuse it, so a program can produce different results on the same inputs with the same source on the same machine at a different optimisation level. Read backwards, the accuracy was bought with reproducibility, and every language that later added a “do not fuse” pragma was buying it back.LIT over 200,000 random triples a*b+c and the fused result differ 22,469 times - 11.23% - and for the 2x2 determinant of [1e8+1, 1e8; 1e8, 1e8-1] whose true value is -1, fused arithmetic returns -1 while naive arithmetic returns 0, which is not close to wrong but the wrong sign of nothing at all
FIG FMA is in IEEE 754-2008 and in every modern instruction set; the exact product here is recovered with Dekker's splitting, which is how you get it without hardware help. AVAN chose a determinant rather than a percentage as the headline, because 11.23% only says the two disagree. Returning 0 for a matrix whose determinant is -1 is a different category of failure: the answer is not imprecise, it has lost the fact that the matrix is invertible at all.
FIG FMA is in IEEE 754-2008 and in every modern instruction set; the exact product here is recovered with Dekker's splitting, which is how you get it without hardware help. AVAN chose a determinant rather than a percentage as the headline, because 11.23% only says the two disagree. Returning 0 for a matrix whose determinant is -1 is a different category of failure: the answer is not imprecise, it has lost the fact that the matrix is invertible at all.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of THE SHORTCUT · David Lee Wise (ROOT0), with AVAN