Repository navigation
Add per-model bias c update for multi-model AMICA (#27) - #49
neuromechanist merged 3 commits into
Conversation
Port Fortran's update_c (amica17.f90:1423-1429/1899-1901) into AMICATorchNG and the NumPy oracle: c[i,h] = sum_t v_h*x / sum_t v_h, the per-model responsibility-weighted data-space mean. The E-step now centers each model's data before unmixing (b = W(x - c)); transform() does the same. Replaces the old gradient-style dc = sum(g) accumulator, which was accumulated but never applied (c was frozen at 0). Guarded to a no-op for n_models=1: with v==1 the update collapses to the (zero) mean of mean-removed data, and skipping it keeps single-model parity bit-exact (issue #24). Tests: single-model c stays exactly zero after fit; multi-model c equals the responsibility-weighted data mean and the two models center differently; existing NG<->NumPy sufficient-stat parity holds with dc renamed to dc_numer. Controlled 2-model A/B vs the Fortran binary (same config/seed, c toggled): cross-corr 0.631 -> 0.642 (+0.011), LL unchanged. The c omission was a minor contributor; the dominant multi-model gap is intrinsic partition ambiguity (see .context/issue-27/multimodel_c_update.md). Issue #27.
Review findings (pr-review-toolkit, Sonnet): - silent-failure: the new c = dc_numer/dgm division could be 0/0 = NaN for a dead model (dgm[h]==0). Unlike log(gm[h])=-inf (which softmax tolerates), a NaN c poisons the next iteration's cross-model softmax for every model. Added a containment guard in both backends: a zero-responsibility model keeps its prior c, mirroring the existing mu/beta/rho non-finite guards. This also resolves the NumPy restart-preserves-NaN-c concern. - tests: added multi-model coverage the change opened up but the first commit left unexercised -- NumPy backend c update on real data, NG<->NumPy finalized c parity, transform() with nonzero c (verified vs W(x-c) by hand), the dead-model containment guard, multi-model dc_numer blocking invariance, and do_reject + multi-model c finiteness. Strengthened the Newton multi-model test to assert finite c and full iteration count. - comments: fixed two pre-existing docstrings this change made stale (transform()/get_weights() said "X^T @ W", now "(X-c)^T @ W"); corrected the Fortran citation for wc = W@c (:2178 get_unmixing_matrices); clarified update_c is a flag not a routine; AGENTS.md "LL unchanged" -> "LL comparable". All fast suites green (48 passed). Single-model paths unchanged (guard is n_models>1). Issue #27.
PR review (pr-review-toolkit, reviewers on Sonnet)Ran four reviewers on the diff: code-reviewer, pr-test-analyzer, silent-failure-hunter, comment-analyzer. Addressed
Not changed (with reason)
Fast suites green: 48 passed (32 NG + 16 NumPy). Single-model Fortran parity unaffected (guard is |
Records the parity confirmation for multi-model AMICA (issue #27). Because mixture-of-ICA is not partition-identifiable, exact partition parity with Fortran is the wrong acceptance bar; the right test is whether the two implementations sample the same distribution over solutions. On an N=20-each ensemble (real sample EEG, n_models=2, 100 iters), the NG-vs-Fortran partition cross-corr distribution is statistically equivalent to Fortran's own run-to-run distribution (Mann-Whitney p=0.97; TOST equivalent within +/-0.05; within-Fortran/within-NG/between all ~0.63-0.64). The single-run ~0.64 cross-corr earlier read as a shortfall is just intrinsic estimator spread: Fortran agrees with itself at 0.63. Adds: - .context/issue-27/multimodel_distributional_equivalence.md (method, results, acceptance criteria) - multimodel_ensemble.py (reproduction harness) + the figure (PNG/PDF) - research.md / AGENTS.md pointers; AGENTS.md #27 now reads VALIDATED Open residual tracked as #51: NG's LL distribution is ~0.02 lower and more variable than Fortran's (optimizer quality, not a correctness bug -- one M-step is bit-exact). Issue #27.
Multi-model validation added (the confirmation that makes this mergeable)Since mixture-of-ICA is not partition-identifiable, exact partition parity with Fortran is the wrong acceptance bar (the Ran N=20 fits per implementation on the real sample EEG (
So NG's multi-model ensemble is statistically equivalent to Fortran's. The single-run ~0.64 cross-corr was never a defect — Fortran agrees with itself at 0.63. Full method / figure / acceptance criteria: One residual, tracked as #51 (not a blocker): NG's LL distribution is ~0.02 lower and more variable than Fortran's — optimizer quality, not a correctness bug (one M-step is bit-exact vs Fortran). This is the validation #27 needed. Ready to merge. |
Closes the concrete, fixable slice of #27: the omitted per-model bias
cupdate.What changed
Ported Fortran's
update_c(amica17.f90:1423-1429 / 1899-1901) into bothAMICATorchNGand the NumPy oraclepyAMICA.py:c[i,h] = sum_t v_h(t)*x(i,t) / sum_t v_h(t)— the per-model, per-channel responsibility-weighted data-space mean.b = W(x - c));transform()in both backends does the same.dc = sum(g)accumulator, which was accumulated but never applied (cwas frozen at 0). Renamed the update-dict keydc->dc_numer.n_models=1: withv == 1the update collapses to the (zero) mean of mean-removed data, and skipping it keeps single-model parity bit-exact (issue Verify NG init-basin vs Newton bit-parity via Fortran load_* init-matching #24).Why the gap persists
A controlled 2-model A/B against the Fortran binary (same config/seed, only
ctoggled):The
comission was a genuine but minor contributor (+0.011 cross-corr, LL unchanged). The dominant residual gap is intrinsic partition ambiguity (mixture-of-ICA has many near-degenerate partitions; NG is self-consistent, cross-corr 1.0 across block sizes), so>0.95is not reachable viacalone. Details:.context/issue-27/multimodel_c_update.md.Tests
cstays exactly zero after fit (issue Verify NG init-basin vs Newton bit-parity via Fortran load_* init-matching #24 regression guard); multi-modelcequals the responsibility-weighted data mean and the two models center differently.dc->dc_numer).