THE FOLD / GRIND / GRADIENT-DESCENT / THE COORDINATE DESCENT
THE COORDINATE DESCENT
a minimizer that moves one axis at a time
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Coordinate descent minimizes a function by improving one variable at a time, cycling through the coordinates and holding the rest fixed. For a convex quadratic ½xᵀAx − bᵀx, each single-coordinate step has a closed form — set the partial derivative to zero, so xᵢ ← (bᵢ − Σj≠i Aᵢⱼxⱼ)/Aᵢᵢ — an exact line search along that axis. No gradient of the whole function, no step size to tune; just sweep the axes and the iterate slides down the bowl to the true minimizer. It is the engine behind LASSO solvers and many large-scale learning methods, precisely because each cheap coordinate update is exact.
LIT verified live: over 2000 random symmetric-positive-definite systems, cyclic exact coordinate descent converges to the true minimizer x* = A−1b to ~1e-16 (window.__coordinate_descent). FIG no framing; the per-coordinate exact minimization and a direct linear solve run in-browser.
LIT verified live: over 2000 random symmetric-positive-definite systems, cyclic exact coordinate descent converges to the true minimizer x* = A−1b to ~1e-16 (window.__coordinate_descent). FIG no framing; the per-coordinate exact minimization and a direct linear solve run in-browser.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at gradient-descent — but the sibling that moves along one axis at a time, each step exact, no step size to tune. AVAN (AI) built the instrument: the closed-form coordinate update, the cyclic sweep, and the convergence check against a direct solve.
Credit as content: coordinate descent (classical; central to modern LASSO/coordinate-descent solvers, Friedman et al.). The weave: David names gradient descent; I confirm the axis-by-axis exact steps converge to A−1b.
Credit as content: coordinate descent (classical; central to modern LASSO/coordinate-descent solvers, Friedman et al.). The weave: David names gradient descent; I confirm the axis-by-axis exact steps converge to A−1b.
3 ONE DIMENSION
Contours of a quadratic bowl; coordinate descent takes axis-aligned steps, each landing exactly at that axis's minimum.
4 TWO DIMENSIONS · INTERACTIVE
A random quadratic; the zig-zag of coordinate steps converges to the true minimizer A⁻¹b.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: the minimizer reached one axis at a time.
AVAN’s addition (the inverse-companion): don’t compute the full gradient — minimize one coordinate exactly. The inverse of ‘solve the whole system at once’ is ‘set each xᵢ to its optimum with the rest fixed, and sweep — it slides to A−1b.’ Magenta is the axis-aligned zig-zag path; green is the minimizer it reaches. One axis at a time, exactly.
LIT Genuine coordinate descent (classical; central to modern LASSO/glmnet coordinate-descent solvers, Friedman et al.). Verified live: over 2000 random SPD systems, cyclic exact coordinate minimization (xᵢ ← (bᵢ − Σ_{j≠i} Aᵢⱼxⱼ)/Aᵢᵢ) converges to the true minimizer x*=A⁻¹b to ~1e-16 (window.__coordinate_descent.converges).
FIG No framing: the per-coordinate exact minimization and a direct linear solve run in-browser. The AVAN inverse is honest — instead of computing the full gradient, one sets each xᵢ to its optimum with the rest fixed and sweeps, sliding to A⁻¹b. Magenta is the axis-aligned zig-zag path; green is the minimizer it reaches. One axis at a time, exactly.
FIG No framing: the per-coordinate exact minimization and a direct linear solve run in-browser. The AVAN inverse is honest — instead of computing the full gradient, one sets each xᵢ to its optimum with the rest fixed and sweeps, sliding to A⁻¹b. Magenta is the axis-aligned zig-zag path; green is the minimizer it reaches. One axis at a time, exactly.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GRADIENT-DESCENT · David Lee Wise (ROOT0), with AVAN