THE FOLD / GRIND / GRADIENT DESCENT / THE CONJUGATE GRADIENT
THE CONJUGATE GRADIENT
solve SPD systems in n steps via A-orthogonal directions
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
The conjugate gradient method solves a symmetric positive-definite system Ax = b — equivalently, minimises the quadratic bowl ½xᵀAx − bᵀx — by choosing search directions that are A-orthogonal (‘conjugate’). Because each direction never undoes the progress of the others, in exact arithmetic it reaches the exact solution in at most n steps, using only matrix–vector products (no matrix stored or inverted).
It is the workhorse for huge sparse systems in physics and optimisation.
LIT verified live: over 200 random SPD systems conjugate gradient reaches the solution within n steps (residual ~10⁻¹⁵) and matches a direct Gaussian solve (window.__conjugategradient). FIG no framing; exact.
It is the workhorse for huge sparse systems in physics and optimisation.
LIT verified live: over 200 random SPD systems conjugate gradient reaches the solution within n steps (residual ~10⁻¹⁵) and matches a direct Gaussian solve (window.__conjugategradient). FIG no framing; exact.
2 HOW IT WAS WEAVED · AI + HUMAN
David (human) seated this at gradient-descent — and as its sharpened form: steepest descent zig-zags down a quadratic bowl, but conjugate gradient picks non-interfering directions and lands in n steps. AVAN (AI) built the instrument: the conjugate-direction iteration, the residual check, the direct-solve cross-check.
Credit as content: Magnus Hestenes & Eduard Stiefel (1952). The weave: David names gradient descent; I follow A-orthogonal directions to the exact minimum and confirm it against a direct solve.
Credit as content: Magnus Hestenes & Eduard Stiefel (1952). The weave: David names gradient descent; I follow A-orthogonal directions to the exact minimum and confirm it against a direct solve.
3 ONE DIMENSION
Steepest descent (grey) zig-zags across the bowl, re-descending directions it already used. Conjugate directions (green) are A-orthogonal — each is taken once and never revisited.
4 TWO DIMENSIONS · INTERACTIVE
A quadratic bowl (contours) and the conjugate-gradient path reaching the minimum in n steps, versus zig-zagging steepest descent.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: the n conjugate directions leading straight to the solution.
AVAN’s addition (the inverse-companion): choose search directions that are A-orthogonal (conjugate), so each step’s progress is never undone by the next. You never re-descend a direction, and in exact arithmetic n conjugate steps reach the exact minimum. The inverse of ‘follow the gradient (and zig-zag)’ is ‘follow conjugate directions that don’t interfere — finish in n steps.’ Magenta is the zig-zagging steepest-descent path; green is the n conjugate directions straight to the solution. Orthogonality in the A-inner-product buys exactness.
LIT Genuine conjugate gradient method (Hestenes & Stiefel 1952). Verified live: for 200 random SPD systems the CG iterate satisfies Ax=b to residual ~1e-15 within n steps and matches a direct Gaussian-elimination solution (window.__conjugategradient.matchesDirect).
FIG No framing: the conjugate-direction iteration, the residual check, and the direct-solve cross-check run in-browser and agree to ~1e-15. The AVAN inverse is honest — A-orthogonal directions never undo each other's progress, so n conjugate steps reach the exact minimum without re-descending; magenta is the zig-zagging steepest-descent path, green the n conjugate directions. Orthogonality in the A-inner-product buys exactness.
FIG No framing: the conjugate-direction iteration, the residual check, and the direct-solve cross-check run in-browser and agree to ~1e-15. The AVAN inverse is honest — A-orthogonal directions never undo each other's progress, so n conjugate steps reach the exact minimum without re-descending; magenta is the zig-zagging steepest-descent path, green the n conjugate directions. Orthogonality in the A-inner-product buys exactness.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GRADIENT DESCENT · David Lee Wise (ROOT0), with AVAN