You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Re-measure parity and sweep docs for epic #324 #351
Final measurement phase of epic #324: re-measure every user-facing parity figure under the finished code, and update the documents that quote them, including the paper.
Epic #324 changed default trajectories on every backend:
Every figure measured before these changes is now a claim about code that no longer ships.
Scope
Per-backend harness rows. Re-measure the rows in docs/guides/validation.md with validate_implementations.py --backend all, along with the bundled amicaout fixture comparisons.
The paper's Table 1, both tiers of benchmarks/reproduce_table1.py, same protocol as before.
The external tier (ds002718 sub-002, 5 seeds at 2000 iterations) runs on the CUDA workstation.
The bundled tier gives the Amari, score-function and multi-model rows.
Update paper.md's table and the prose around it, README.md, AGENTS.md and validation.md wherever they quote these figures.
The ensemble log-likelihood row is the one metric on which the old ensembles differed significantly. Re-test the explanation that it is convergence speed ("reaches Fortran's mean by 200 iterations, passes it by 300"), or replace it with what the new fits show.
The paper's other measured claims:
the Newton-enabled independent-seed statement (one seed of three with ten of seventy components in a different basin; a matched initialization restores ~0.997);
"all backends converge to the same single-model log-likelihood (maximum pairwise difference ~0.003)";
"component-level float32 agreement is not yet characterized at a matched iteration budget", which the per-backend harness rows now characterize.
ADR 0003's keep_best figures (the log-likelihood spread relative to Fortran's, and how often a restore fires). Re-measure them with a seeded multi-model ensemble against the pinned binary. Phase 7 (Phase 7: normalize component rows in doscaling #340) found that the old overshoots came mostly from the column-rule doscaling bug.
The share_comps merge counts quoted in docs/guides/amica-differences.md, which Phase 11's order change made stale.
Every re-measured figure is recorded with its run record (command, commit, host, raw output) under .context/issue-351/.
The documentation audit and the dead-code cleanup that were first listed here moved to #352.
Final measurement phase of epic #324: re-measure every user-facing parity figure under the finished code, and update the documents that quote them, including the paper.
Epic #324 changed default trajectories on every backend:
Every figure measured before these changes is now a claim about code that no longer ships.
Scope
docs/guides/validation.mdwithvalidate_implementations.py --backend all, along with the bundledamicaoutfixture comparisons.benchmarks/reproduce_table1.py, same protocol as before.paper.md's table and the prose around it,README.md,AGENTS.mdandvalidation.mdwherever they quote these figures.docs/assets/figures/multimodel-ensemble.pngfrom the new fits.docs/guides/amica-differences.md, which Phase 11's order change made stale.Every re-measured figure is recorded with its run record (command, commit, host, raw output) under
.context/issue-351/.The documentation audit and the dead-code cleanup that were first listed here moved to #352.
Part of epic #324.