Repository navigation
Preallocate Magnus NC6/Gauss4 stage and commutator matrices - #4793
Conversation
MagnusNC6 and MagnusGauss4 rebuilt every stage snapshot, intermediate B/commutator matrix, and the final generator from scratch each step via deepcopy, matrix products with `*`, and repeated subexpressions, allocating ~1.4 MB/step (N=50, time-dependent) and ~22.7 MB/step (N=200, constant) for NC6, and ~223 KB/~3.5 MB/step for Gauss4. The caches now preallocate every stage/commutator buffer and the perform_step! bodies use copyto!/mul!/@. to fill them in place, cutting step allocation to ~61 KB (N=50) and ~962 KB (N=200) for both algorithms. The remaining bytes come from ExponentialUtilities.exponential!'s internal Pade scratch matrices, which the `ExpMethodGeneric` cache argument does not reuse; that is a separate, upstream-scoped allocation source. Every rewritten expression keeps the original's literal operator grouping (e.g. `3 * B0 / 2`, not `1.5 * B0`) so the floating-point rounding is unchanged; sol.u hashes are bitwise identical to master on a time-dependent N=50 and a constant N=200 problem at fixed dt. Adds a runtime allocation-bound regression test that fails on unfixed master (1,431,464 and 222,664 bytes/step against a 150,000 bound) and passes after the fix. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Agent-Harness: Claude Code Agent-Model: claude-sonnet-5-5 Agent-Session: https://claude.ai/code/session_01LPHREnnonfLg1VcE1EJovv
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Devin <noreply@cognition.ai> Agent-Harness: Devin CLI 3000.11.3 (conflict resolution), Claude Code 2.1.285 (orchestration) Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium Agent-Session: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master); Devin log /Users/chrisrackauckas/sandbox/master/fusion-fixes/SciML_OrdinaryDiffEq.jl-4793/log.txt
Merged the default branch into this PR (clean merge, no conflicts; head 602e7e8) so CI runs on current master. Its earlier failures in SciMLSensitivity Core1 and the downgrade jobs were fixed on master after this PR's last CI run. 🤖 Posted by an AI agent — harness: Devin CLI 3000.11.3 (merge), Claude Code 2.1.285 (posting) · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium |
The preallocated stage buffers were `similar(W)`, where `W` is an outer product of `rate_prototype`. That gave them the state's units squared (DimensionError for a unitful state with a dimensionless operator), the state's element type (complex or Float32 states changed rounding), and a dense layout for sparse operators (17 / 4 dense N×N matrices per cache on the Krylov path). The buffers are now `zero(A)` of the operator's own matrix, and only when that matrix is a `StridedMatrix` whose element type is unchanged by the stage arithmetic (`promote_type(T, Float64, typeof(dt)) === T`). Every other operator (sparse, structured, mixed precision, unitful time) forms its stages out of place exactly as on master. The final update is again `u .= exp(Ω) * uprev`: `mul!` into `u` accumulated in `eltype(u)` and changed rounding for Float32 states. Adds Core tests for a unitful state and for sparse/Tridiagonal operators (cache stays below two dense N×N matrices, solution matches the dense operator), with Unitful and SparseArrays as test-only dependencies. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.301 Agent-Model: claude-opus-5-5 Agent-Session: local session ID 152619c5-78a8-4ae5-a06c-12cc1a58a70b (headless claude -p, no URL)
Independent review (Codex CLI 0.166.0 / gpt-6.1-sol), session 01a1977f-3b3a-75c2-936e-938ee574c9a3, reviewed head 8d42b71
Review written by the reviewer named above in a session other than the PR author's, and copied here by the posting agent without change to its findings. The posting agent did not re-run the reviewer's commands. Summary: Preallocates dense MagnusNC6 and MagnusGauss4 stage matrices while retaining out-of-place arithmetic for other operator types. The allocation improvement is verified without an observed new numerical defect; a reproduced, inherited Gauss4 commutator-sign issue merits a separate fix. Blast radius: In-place MagnusNC6 and MagnusGauss4, including their Krylov branches. Dense operators with arithmetic-preserving element types retain 17 and 4 additional matrices respectively; sparse, structured and other scalar combinations use the original formulas. Default algorithm selection and the listed OrdinaryDiffEq main lanes are unchanged. SparseArrays and MIT-licensed Unitful are test-only additions. Evidence: Blind risk recorded and hashed before author context. Head linear-method tests pass 48/48 and new allocation bounds pass 4/4; the same bounds fail 4/4 on the merge-base. Added operator-type tests pass 10/10 at both revisions. An independent exact-solution probe yields the same forward error arrays at both revisions, including an inherited Gauss4 order defect. ForwardDiff checks pass 2/2 at both revisions. CI has 128 successful checks, three skipped checks and one failed downstream check whose exact assertion also fails on master. Blocking findingsNone. Non-blocking findings
What I ranGit commands ran in the source checkout: git status --short
git rev-parse HEAD
git rev-parse origin/master
git diff --stat origin/master...HEAD
git diff origin/master...HEAD
git fetch origin '+refs/pull/4793/head:refs/remotes/pr/4793' master
git checkout --detach pr/4793
git fetch origin '+refs/heads/master:refs/remotes/origin/master'
git merge-base origin/master HEAD
git log -1 --no-merges --format=%B origin/master..HEAD
git worktree add --detach ../base 59ba40e4b5bbae713d8df7b792f6ea62f033ff78These git operations exited 0. The checkout and live PR head matched the target. The fetched origin/master was d035621; the reviewed base above is the live PR metadata's baseRefOid and the merge-base used for the before comparison. Repository instruction-file searches found no AGENTS.md or CLAUDE.md. For the following exact Julia command structure, REVIEW_ROOT and REVIEW_DEPOT denote the review workspace and shared writable depot, respectively; absolute machine-specific arguments are replaced with these variables: JULIA_NUM_THREADS=4 OPENBLAS_NUM_THREADS=1 JULIA_NUM_PRECOMPILE_TASKS=4 JULIA_PKG_PRECOMPILE_AUTO=0 TMPDIR="$REVIEW_ROOT/tmp" JULIA_DEPOT_PATH="$REVIEW_DEPOT:" timeout 3600 julia +1.13.2 --startup-file=no run_head.jl
JULIA_NUM_THREADS=4 OPENBLAS_NUM_THREADS=1 JULIA_NUM_PRECOMPILE_TASKS=4 JULIA_PKG_PRECOMPILE_AUTO=0 TMPDIR="$REVIEW_ROOT/tmp" JULIA_DEPOT_PATH="$REVIEW_DEPOT:" timeout 3600 julia +1.13.2 --startup-file=no run_head_validated.jl
JULIA_NUM_THREADS=4 OPENBLAS_NUM_THREADS=1 JULIA_NUM_PRECOMPILE_TASKS=4 JULIA_PKG_PRECOMPILE_AUTO=0 TMPDIR="$REVIEW_ROOT/tmp" JULIA_DEPOT_PATH="$REVIEW_DEPOT:" timeout 3600 julia +1.13.2 --startup-file=no run_base.jlJulia version was 1.13.2, with one process at a time and three invocations total. Temporary environments used local package development, without changing checkout Project files or updating registries. Resolved numerical dependencies included SciMLOperators 1.30.4 and ExponentialUtilities 1.37.1.
The added allocation regression therefore fails before and passes after in my own runs. Base allocations were 1,430,888 and 222,632 bytes for the N=50 time-dependent NC6/Gauss4 cases, versus the 150,000-byte bound; and 22,730,984 and 3,524,648 bytes for N=200 constant cases, versus the 2,000,000-byte bound. All four bounds pass at head. The new unitful/sparse/Tridiagonal tests also pass on the original base; they guard compatibility of the optimization rather than demonstrate an original-base functional bug. The independent oracle uses R(t) = rotation(omegat), D = diag(d1,d2), A(t) = omegaJ + R(t)DR(t)' and exact u(t) = R(t)exp(Dt)*u0, with J the planar rotation generator. Parameters were omega=0.7, u0=[1,2], D=diag(-0.2,-0.8), and a ComplexF64 variant D=diag(-0.2+0.3im,-0.8-0.1im). Both algorithms and both Krylov settings ran at dt=1/4, 1/8 and 1/16, followed by reinit! with changed parameters/state and backward integration. NC6 observed orders were 5.99797–5.99867; its finest-step errors were about 1.23e-11 real and 2.15e-11 complex. Gauss4 observed orders were 2.00036–2.00138. Every forward error array was identical at head and base. The 16 failures at each revision were Gauss4 accuracy, order, reinit accuracy and backward accuracy assertions; no exceptions were hidden or tolerances relaxed. ForwardDiff differentiated the solved scalar equation u'=-p*u, u(0)=1 at p=0.3. Output at both revisions: These match the independent derivative -exp(-0.3) at rtol=1e-10. Read-only GitHub checks: gh pr view 4793 --repo SciML/OrdinaryDiffEq.jl --json number,url,state,isDraft,headRefOid,baseRefName,baseRefOid
gh pr view 4793 --repo SciML/OrdinaryDiffEq.jl --json title,body,commits,statusCheckRollup
gh pr checks 4793 --repo SciML/OrdinaryDiffEq.jl
gh run view 38919896421 --repo SciML/OrdinaryDiffEq.jl --log-failed
gh run list --repo SciML/OrdinaryDiffEq.jl --branch master --event push --limit 5 --json databaseId,conclusion,status,url
gh run view 41249816432 --repo SciML/OrdinaryDiffEq.jl --json name,jobs
gh run view 41249816166 --repo SciML/OrdinaryDiffEq.jl --json name,jobs
gh run view 41249816054 --repo SciML/OrdinaryDiffEq.jl --json name,jobs
gh run view 41249816054 --repo SciML/OrdinaryDiffEq.jl --log-failedThe initial log-download attempt hit a read-only CLI cache; using a writable XDG_CACHE_HOME retrieved both failure logs. PR checks reported a failing check. The final log reads and metadata queries exited 0. Sublibrary CI passed Linear Core on stable/LTS/prerelease plus QA and GPU groups; main CI passed its reported tests, including AD and Downstream. PR IntegrationTest failed in SciMLSensitivity Core4, What I did not verifyNo local full package suite, full QA static analysis, downstream suite, GPU execution, MPI, Reactant, Enzyme, Mooncake or reverse-mode AD runs were performed. The existing GPU test file covers LinearExponential rather than these Magnus algorithms; no new GPU kernel or AD rule is introduced here. The local AD probe covers ForwardDiff and non-Krylov scalar parameter propagation only. I did not benchmark wall-clock speed or prove bitwise equality for every supported scalar, operator type or problem dimension. NC6 exact-reference accuracy and cache reuse passed; fourth-order accuracy for Gauss4 was explicitly disproved at both revisions. The proposed separate Gauss4 sign repair was not applied or tested. The three-run limit is exhausted. No linked issue relevant to this PR was identified, and no prior-review file was read. 🤖 Posted by an AI agent — harness: Claude Code · model: claude-sonnet-5-5 |
Please ignore until reviewed by @ChrisRackauckas.
What changed and why
MagnusNC6andMagnusGauss4inlib/OrdinaryDiffEqLinear/src/linear_perform_step.jlrebuilt every stage-matrix snapshot, every intermediate
B/commutator matrix, and thefinal generator from scratch each step (
deepcopy,*for matrix products, and somesubexpressions computed twice). This was identified as a measured allocation hotspot in
a prior performance audit (
<job>/REPORT.mdon the audit machine): ~1.4 MB/step (N=50, time-dependent) and ~22.7 MB/step (N=200,
constant) for
MagnusNC6, and ~223 KB/~3.5 MB/step forMagnusGauss4.MagnusNC6CacheandMagnusGauss4Cachenow preallocate every stage and commutatorbuffer, and
perform_step!fills them in place withcopyto!,mul!, and@.broadcasting instead of allocating new matrices. A shared
_magnus_commutator!(dest, X, Y, tmp1, tmp2)helper computes[X,Y] = X*Y - Y*Xinto a preallocated buffer for bothalgorithms.
Every rewritten expression preserves the exact literal operator grouping of the
original (e.g.
3 * B0 / 2 - 6 * B2, not the mathematically-equal but differently-rounded1.5 * B0 - 6 * B2), so no floating-point rounding changes. Repeated subexpressions(e.g.
3*B0/2 - 6*B2used twice in the originalΩ2formula) are computed once into anamed buffer and reused — this does not change the bits, since it is the same
deterministic expression evaluated from the same inputs, just not recomputed.
The residual allocation (~61 KB/step at N=50, ~962 KB/step at N=200, for both
algorithms) comes from
ExponentialUtilities.exponential!'s internal Padé scratchmatrices (
y1,y2,y3inexp_generic_mutable) — thecacheargument accepted byexponential!(x, ::ExpMethodGeneric, cache)is unused by that method and alwaysallocates fresh scratch space. Eliminating that would require either reaching into
ExponentialUtilities internals or an upstream change to give
ExpMethodGenericareusable cache path; both are out of scope here and are called out as a separate,
upstream-scoped follow-up rather than silently left unaddressed.
Before / after (bytes/step,
taskset -c 0, Julia 1.12.4)solveminimum times (20-step fixed-dtBenchmarkToolsminimum, sametaskset -c 0pin) were unchanged or slightly faster after the fix (e.g. MagnusNC6 N=200: 720.5ms →
653.9ms; MagnusGauss4 N=50: 13.96ms → 11.03ms; MagnusNC6 N=50 and MagnusGauss4 N=200
were within run-to-run noise of each other).
Bitwise-identity check
sol.uon a time-dependent N=50 problem and a constant-matrix N=200 problem (fixeddt=0.01,adaptive=false) gives bitwise-identicalhash(sol.u[end])before andafter the fix, for both algorithms:
Same four hashes, same four
u[1]/norm(u)values, on unmodified master and on thisbranch.
New test: failing before, passing after
Added
"Magnus dense stage-matrix allocation regression"tolib/OrdinaryDiffEqLinear/test/qa/allocation_tests.jl, bounding per-step allocation(
@allocated step!(integrator), two warm-up steps first) well below master's numbers:150,000 bytes at N=50, 2,000,000 bytes at N=200, for both
MagnusNC6andMagnusGauss4.Before the fix (test present,
src/reverted toorigin/master,GROUP=QA):After the fix (this branch,
GROUP=QA):The 10 pre-existing
brokenentries are the unrelatedAllocCheck-based static checksin
"Linear perform_step! Static Analysis"(already markedbroken=trueon master;untouched by this PR).
GROUP=Corealso passes on this branch:Formatting / spelling
What was not verified
test/gpu/linear_exp.jl) were not run — not touched by this change, and notexercised in this environment.
GROUP=Everything/full test suite was not run (machine-load / time constraints; perlocal policy,
GROUP=Allwas intentionally not run either). OnlyCoreandQAwererun, as those cover the changed algorithms and the new regression test.
alg.krylov == true) of both algorithms share the newA/B/commutator buffers but were not independently re-benchmarked for allocation (they were
already low-allocation per the prior audit); correctness for
MagnusGauss4's Krylovbranch is exercised by the existing
linear_method_tests.jlconvergence tests, whichpass.
MagnusGL6,MagnusGL8,LieRK4,RKMK4, etc.) have the same deepcopy/*-product allocation pattern and arenot addressed here — out of scope for this PR, left as further opportunities.
Risk assessment
lib/OrdinaryDiffEqLinear/src/linear_caches.jlandlinear_perform_step.jlonly; changes are confined toMagnusNC6Cache/MagnusGauss4Cacheand theirperform_step!methods. No public API signature change(same algorithm structs, same
solve/initcall surface). Internal cache structfields gain new (non-exported) fields;
@cache'sfull_cachegeneration only tracksthe pre-existing
uType/rateType/etc.-typed fields, so resizing behavior isunaffected.
sol.uhashes before/after on two representativeproblems; new allocation-bound test demonstrated failing on unmodified master and
passing after the fix;
GROUP=Core(38/38) andGROUP=QA(Allocation/JET/Aqua allpassing) both green on this branch.
integrators), even though bitwise identity was verified by hash comparison rather than
a from-scratch proof.
🤖 Generated with Claude Code (model: claude-sonnet-5-5) https://claude.ai/code/session_01LPHREnnonfLg1VcE1EJovv
Review findings addressed
An independent review (Codex CLI, gpt-6-astra) returned CHANGES on
9719d1d2with two findings. Both are fixed in8d42b711.DimensionError(fixed). The stage buffers weresimilar(W), andWis an outer product ofrate_prototype, so a state in metres gave buffers in m². They are nowzero(A)of the operator's own matrix. Before the fix,solve(ODEProblem(MatrixOperator([-0.1 0; 0 -0.2]), [1.0, 2.0]u"m", (0.0, 1.0)), alg; dt = 0.1, adaptive = false)threw for both algorithms. After the fix it returns[0.90483741803596 m, 1.6374615061559628 m], the same as master. Added as a Core regression test.convert(AbstractMatrix, L)is aStridedMatrixwhose eltypeTsatisfiespromote_type(T, Float64, typeof(dt)) === T. Every other operator (sparse,Tridiagonal, Float32 operators with Float64 coefficients, unitful time) forms its stages out of place with master's formulas. For a 300×300 sparse operator withkrylov = true, the cache is back to 732,304 bytes for both algorithms (it was 12,973,256 for NC6 and 3,612,528 for Gauss4 on9719d1d2). Added as a Core regression test: the cache must stay below two dense N×N matrices, and the sparse andTridiagonalsolutions must match the dense-operator solution (rtol = 1e-12).A correction to the bitwise-identity section above. That section is only true for dense Float64. On
9719d1d2,hash(sol.u[end])differed from master for Float32 states, Float32 operators, ComplexF64 states, and sparse operators. Two causes: the buffers took the state's eltype, andmul!(u, exp(Ω), uprev)accumulates ineltype(u). The final update is againu .= exp(Ω) * uprev. With this commit, all 24 cases match master bit for bit: 6 problems (dense Float64 N=50 time-dependent and N=200 constant, Float32 state, Float32 operator, ComplexF64 state, sparse tridiagonal) × {NC6, Gauss4} × {krylov off, on}. Restoring that update costs one length-N vector per step. Dense per-step allocation is now 61,320 B at N=50 and 963,592 B at N=200 for both algorithms. On master it is 1,431,016 / 222,696 / 22,730,984 / 3,524,648 B.Unchanged pre-existing behavior, not addressed here: a unitful state with
krylov = truefails on master and here withzero(Quantity{Float64}) not defined, andkrylov = falsewith a sparse operator fails on master and here with ExponentialUtilities' "exp(A) on a sparse matrix is generally dense" error. On9719d1d2the second case ran, because that version densified the stages.Verification (Julia 1.13.0)
MagnusNC6 and MagnusGauss4 operator matrix types, run alone: fails on9719d1d2(Pass 4, Fail 2, Error 2:DimensionError: m^2 and -0.1 are not dimensionally compatible×2,Evaluated: 12973256 < 1440000,Evaluated: 3612528 < 1440000). It passes 10/10 on master and 10/10 with this commit.GROUP=Core, Julia 1.13.0:Linear Methods Tests | 48 48(38 existing + 10 new). Julia 1.10.12:Linear Methods Tests | 48 48.GROUP=QA, Julia 1.13.0:Allocation Tests | 4 pass, 10 broken(the 10 broken are the pre-existing AllocCheck entries),JET Tests | 1 1,Aqua | 21 21.--checkandtyposare clean on the changed files.Not verified: GPU (
test/gpu/linear_exp.jl), Julia pre-release, the full monorepo suite, and downstream packages.Risk assessment (this commit)
MagnusNC6/MagnusGauss4in-place caches only. Dense operators that satisfy the eltype check run the preallocated path, and every other operator runs master's code. No public API change. Unitful and SparseArrays are added as test-only dependencies (both MIT).🤖 Posted by an AI agent — harness: Claude Code 2.1.301 · model: claude-opus-5-5
Conversation: local session ID 152619c5-78a8-4ae5-a06c-12cc1a58a70b (headless, no URL)
Risk assessment (independent review, codex-cli gpt-6.1-sol, reviewed head 8d42b71)
JULIA_NUM_THREADS=4 JULIA_NUM_PRECOMPILE_TASKS=4 GROUP=Core julia +release --project=repo/lib/OrdinaryDiffEqLinear -e 'using Pkg; Pkg.test()'on Julia 1.13.2: Linear Methods Tests 48/48 passed, including both earlier regression cases and existing convergence assertions. I ran the new dense allocation testset extracted verbatim: 4/4 passed (allocation.log). A seeded direct comparison of preallocated and fallback generators for Float64, ComplexF64, and BigFloat at sizes 2 and 7, including both Gauss4 Krylov scalings, passed 18/18 at rtol=1e-14 (generator.log). All 22 PR checks were pending at both observations; no failing jobs were available for log inspection. PR CI: https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/38919897229 and https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/38919897619. Latest master Sublibrary CI and CI were also pending: https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/38931067549 and https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/38931067408. There is no observed CI regression, but pending CI does not establish a passing result.Reviewed head:
8d42b7114660184a78580e69b79b5b9d5d8f2f21, recorded at the start inOrdinaryDiffEq.jl-4793.headand rechecked unchanged at the end. Initial independent risk was recorded as medium before reading the author's risk claims.🤖 Posted by an AI agent — harness: codex-cli · model: gpt-6.1-sol (independent reviewer; appended by Claude Code 2.1.301 · claude-opus-5-5)
Conversation: https://claude.ai/code/session_01VZjwf7Ux9YtTzwHGHMqnsq