Skip to content

bench: NG vs Fortran runtime harness (#59) - #62

Merged
neuromechanist merged 1 commit into
mainfrom
59-benchmark-amicatorchng-runtime-vs-fortran-binary
Jul 7, 2026
Merged

neuromechanist merged 1 commit into
mainfrom
59-benchmark-amicatorchng-runtime-vs-fortran-binary

Conversation

@neuromechanist

Copy link
Copy Markdown
Member

Part of #59. Adds benchmarks/benchmark_runtime.py, a repeatable runtime benchmark for AMICATorchNG vs the Fortran binary on the real sample EEG.

What it does

  • Matched config (same iters, lrate, Newton schedule, num_mix, pdftype) pulled from sample_params.json / input.param, so both backends do equivalent work (verified by near-identical final LL).
  • Sweeps an iteration x thread-count grid; reports total wall-clock (mean/std) and a fixed-overhead-free marginal per-iteration slope (cancels data load / process spawn / I/O).
  • Environment banner flags that the bundled amica15mac is x86_64 and runs under Rosetta 2 on arm64, so local ratios flatter NG.
  • Writes JSON + a markdown report (dir gitignored). No silent failures: a nonzero Fortran exit raises.

Measured result (this host: M4 Pro, Fortran emulated)

NG misses the 2-3x criterion on the CPU float64 parity path: ~3.2x single-thread, ~7.3x at 8 threads because NG barely parallelizes on CPU (219->204 ms/it) while Fortran's OpenMP scales ~2.4x. Full numbers + profile posted to #59; hot-spot follow-up filed separately.

Docs-adjacent tooling; no library code touched.

Times AMICATorchNG vs the amica15mac binary on real sample EEG with a matched config, over an iteration x thread-count grid. Derives total wall-clock and a fixed-overhead-free marginal per-iteration slope, prints an environment banner (flags Rosetta emulation of the x86_64 binary on arm64), and writes JSON + a markdown report. Results dir gitignored. Refs #59.
@neuromechanist neuromechanist linked an issue Jul 7, 2026 that may be closed by this pull request
5 tasks
@neuromechanist
neuromechanist merged commit 59ad4ef into main Jul 7, 2026
5 checks passed
@neuromechanist
neuromechanist deleted the 59-benchmark-amicatorchng-runtime-vs-fortran-binary branch July 7, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark AMICATorchNG runtime vs Fortran binary

1 participant