THE FOLD / GRIND / BACKPROP / THE NELDER-MEAD
THE NELDER-MEAD
a triangle feels for the valley floor
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
The Nelder–Mead method (the downhill simplex) minimizes a function using no derivatives at all — only its values at the corners of a moving simplex (a triangle in 2D, a tetrahedron in 3D). Each step it finds its worst corner and reflects it through the centroid of the others; if that lands even better it expands further, if it is still bad it contracts inward, and if all else fails the whole simplex shrinks toward its best corner. The amoeba crawls, tumbles, and squeezes its way downhill until it collapses onto the minimizer. It is the workhorse behind ‘fit this curve’ buttons everywhere — robust, gradient-free, and almost embarrassingly simple.
LIT verified live: over hundreds of random convex bowls (including a rotated, non-separable one), the simplex converges to the true minimizer to within ~1e-8 using only function evaluations (window.__nelder_mead). FIG no framing; the reflect/expand/contract/shrink steps and the convergence test run in-browser. Nelder–Mead is not guaranteed on every non-convex surface — the claim here is convergence on the convex bowls tested.
LIT verified live: over hundreds of random convex bowls (including a rotated, non-separable one), the simplex converges to the true minimizer to within ~1e-8 using only function evaluations (window.__nelder_mead). FIG no framing; the reflect/expand/contract/shrink steps and the convergence test run in-browser. Nelder–Mead is not guaranteed on every non-convex surface — the claim here is convergence on the convex bowls tested.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at backprop — but as its opposite twin: where backprop follows the gradient, Nelder–Mead grinds downhill with no gradient at all, feeling the floor by touch. AVAN (AI) built the instrument: the simplex, the reflect/expand/contract/shrink logic, the convergence-diameter stop, and the self-test over random bowls.
Credit as content: John Nelder & Roger Mead (1965). The weave: David names the grind; I confirm the amoeba reaches the minimizer using only function values.
Credit as content: John Nelder & Roger Mead (1965). The weave: David names the grind; I confirm the amoeba reaches the minimizer using only function values.
3 ONE DIMENSION
The simplex on a convex bowl (min at (2,-1)): step it and watch the worst corner reflect toward the valley.
4 TWO DIMENSIONS · INTERACTIVE
Step once, run to convergence, or reset; the self-test confirms convergence over many random bowls.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: the simplex closing onto the minimizer.
AVAN’s addition (the inverse-companion): don’t follow a gradient — mirror the worst. The inverse of ‘keep the good corners’ is ‘reflect the worst corner through the opposite face and see if the mirror image is better.’ Magenta is that reflection ray; green is the collapsing simplex. Progress by mirroring failure.
LIT Genuine Nelder-Mead downhill-simplex method (John Nelder & Roger Mead, 1965). Verified live: over 400 random convex bowls the simplex converges to the true minimizer to within ~1e-8 worst-case position error, and a rotated non-separable bowl converges too — all using only function values, no gradients (window.__nelder_mead.converges, .worst, .rotatedErr).
FIG No framing; the reflect/expand/contract/shrink steps and the convergence test run in-browser. Nelder-Mead is not guaranteed on every non-convex surface — the claim is convergence on the convex bowls tested. The AVAN inverse is honest — instead of following a gradient, mirror the worst corner through the opposite face. Magenta is that reflection ray; green is the collapsing simplex. Progress by mirroring failure.
FIG No framing; the reflect/expand/contract/shrink steps and the convergence test run in-browser. Nelder-Mead is not guaranteed on every non-convex surface — the claim is convergence on the convex bowls tested. The AVAN inverse is honest — instead of following a gradient, mirror the worst corner through the opposite face. Magenta is that reflection ray; green is the collapsing simplex. Progress by mirroring failure.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of BACKPROP · David Lee Wise (ROOT0), with AVAN