Repository navigation
Make AN5 and JVODE caches resizable after resize! - #4759
Conversation
…order 1. AN5Cache and JVODECache had no full_cache, so resize!(integrator, n) threw MethodError. Nested z/Tsit5 arrays also need resizing, and the history is invalid after a size change. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent 2026.10.01-14929f9 Agent-Model: unknown (Cursor auto) Agent-Session: local session, transcript /home/crackauc/sandbox/goals/performance/jobs/pr-nordsieck-resize/log.txt on amdci2
Setting reeval_fsal in the resize hook skipped Core's discontinuity sync at iter==0, so startup consumed uninitialized history. Restrict the extra FSAL eval to a resize restart so same-size callbacks are unchanged. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent 2026.10.01-14929f9 Agent-Model: unknown (Cursor auto) Agent-Session: local session, transcript /home/crackauc/sandbox/goals/performance/jobs/pr-nordsieck-resize/log.txt on amdci2
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Devin <noreply@cognition.ai> Agent-Harness: Devin CLI 3000.11.3 (conflict resolution), Claude Code 2.1.285 (orchestration) Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium Agent-Session: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master); Devin log /Users/chrisrackauckas/sandbox/master/fusion-fixes/SciML_OrdinaryDiffEq.jl-4759/log.txt
Merged the default branch into this PR (clean merge, no conflicts; head 9cac8fa) so CI runs on current master. Its earlier failures in SciMLSensitivity Core1 and the downgrade jobs were fixed on master after this PR's last CI run. 🤖 Posted by an AI agent — harness: Devin CLI 3000.11.3 (merge), Claude Code 2.1.285 (posting) · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium |
Include the JVODE controller recovery and BDF formula fixes needed by the resize regression tests. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.163.0 Agent-Model: gpt-6.1-sol Agent-Session: local session, transcript /home/crackauc/sandbox/goals/performance/jobs/pr-nordsieck-resize/log-codex.txt on amdci2
Cover both modification flags and JVODE_BDF in the analytic resize regression, and tighten JVODE solve and reference tolerances. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.163.0 Agent-Model: gpt-6.1-sol Agent-Session: local session, transcript /home/crackauc/sandbox/goals/performance/jobs/pr-nordsieck-resize/log-codex.txt on amdci2
🛡️ Tently review✅ No design-intent regressions or decision drift found. Posted by Tently |
Independent review (Codex CLI 0.166.0 / gpt-6.1-sol), session 01a19834-a9a7-7d80-9de9-55255a433487, reviewed head eb80c56
Review written by the reviewer named above in a session other than the PR author's, and copied here by the posting agent without change to its findings. The posting agent did not re-run the reviewer's commands. Summary: Adds resizing and history restart support for in-place AN5 and both JVODE variants, with a Core compatibility floor and analytic regression tests. The added regressions fail on the merge-base and pass on the exact head; no blocking defect was found. Blast radius: In-place AN5, JVODE_Adams and JVODE_BDF state resizing, including their history and nested Tsit5 buffers. No shared Core implementation or default solver changes. Evidence: Head targeted tests pass 83/83; the identical regression file on merge-base b94e13a gives 50 passes and seven errors from missing full_cache methods. Nordsieck CI Core, QA and downgrade checks pass. The sole failing PR check matches an exact master Enzyme failure. The additional adversarial probe failed during reviewer harness setup, before executing assertions; its coverage is not claimed. Blocking findingsNone. Non-blocking findingsNone. What I ranRead the complete rules and diff before author context; recorded and hashed INITIAL.md. Inspected the generic resize/delete interfaces, cache macro, controllers, restart code and tests. Git commands exited 0: Julia 1.13.2, one process at a time, with JULIA_NUM_THREADS=4, OPENBLAS_NUM_THREADS=1, JULIA_NUM_PRECOMPILE_TASKS=4, JULIA_PKG_PRECOMPILE_AUTO=0, an isolated tmp directory, and the assigned shared depot followed by default depots. Absolute environment paths are omitted. Scratch packages retain identical Julia source; only their Project.toml sources tables are removed. The head environment develops local Nordsieck, Core, Tsit5, DiffEqBase and DiffEqDevTools, and adds Test, LinearAlgebra and ODEProblemLibrary using offline resolution. The base environment reuses exactly that manifest, changing only the Nordsieck source path to the merge-base snapshot.
Read live checks with Inspected CI URLs:
G1 is satisfied by the added piecewise matrix-exponential accuracy reference. G3 is satisfied by growth followed by shrinkage under both modification flags, plus NaN-poisoned initialization. G2 has no changed GPU kernel, AD rule, Reactant or MPI implementation to exercise. What I did not verifyThe planned extra analytic nonautonomous manual-resize, middle-deletion, continuous-callback and bitwise base/head controls did not execute because the third-run harness failed. Runtime public-name assertions in that probe likewise did not execute. No whole-package or downstream suites were run locally. GPU/non-Vector resizing, out-of-place resizing, AD through a resize, Reactant and MPI were not exercised locally; these implementations are not changed by this PR. Exact registered Core 4.7 resolution, Julia 1.10, QA and documentation builds were not independently rerun. Long post-resize AN5 trajectories beyond the regression interval were not tested. This is a code merge recommendation; the PR remains a draft. The PR description returned embedded historical reviews and current-head review summaries after the blind assessment was hashed. The separate prior review file remained unread until the complete review was frozen; this limits the later phase's independence from review text embedded in author context. 🤖 Posted by an AI agent — harness: Claude Code · model: claude-sonnet-5-5 |
Summary
Enable in-place AN5, JVODE_Adams and JVODE_BDF integrators to resize their user buffers, Nordsieck history and Tsit5 stage buffers, then restart at order 1 with the caller-filled state. Require OrdinaryDiffEqCore 4.7, whose registered
@cacheextendsSciMLBase.full_cache.The resize regressions cover growth and shrinkage against a piecewise matrix-exponential reference, both
u_modified!(integrator, true)andu_modified!(integrator, false), and NaN-poisoned resizing before the first step. All three algorithms are covered. Master is merged through b94e13a to include the JVODE controller recovery and BDF formula fixes.Review fixes and local verification
Revision: eb80c56
All Julia runs use
timeout 3600, one top-level run at a time,TMPDIR=<job tmpdir>, andJULIA_DEPOT_PATH=<job depot>:$HOME/.julia.<site tool dir>is removed from PATH. Commands below run from the checkout; probe scripts, scratch environments and logs are retained beside it in the job directory.Registered Core compatibility
Raised the Core floor from 4.4 to 4.7. General's
OrdinaryDiffEqCore/Versions.tomlrecords 4.6.0 as treec4035f06d7749c49347d8a200d3ba0a17085492eand 4.7.0 asc6acc3ee837cae1abc382569d9fb3254a7974e11; the resolved registered packages match those trees. Core 4.6.0's macro generates a caller-localfull_cache, while 4.7.0 usesGlobalRef(SciMLBase, :full_cache).The same Nordsieck source and resize/step probe were run against exact registered Core versions, with
[sources]removed from scratch copies. The 4.6.0 copy retains the old Core compat so the resolver can reproduce the failure. Each probe tests AN5, JVODE_Adams and JVODE_BDF:Logs:
floor-4.6.0.log,floor-4.7.0.log.Restart-branch regression discrimination
The callback test is parameterized over both modification flags. The exact committed test file was copied into each scratch package. Each scratch package changes only one restart branch in
nordsieck_perform_step.jl; all other solver source matches the checkout.Failing before (AN5 restart branch removed):
Failing before (JVODE restart branch removed): both Adams and BDF false-flag cases fail
successful_retcode, stop before shrink, and their grow interpolation checks throwDimensionMismatch: a has size (2,), b has size (3,).Passing after (both branches intact): the complete Core group passes, including all six algorithm/flag combinations; final output is below. Logs:
mutation-an5.log,mutation-jvode.log.JVODE coverage and tolerance
Added JVODE_BDF to both the growth/shrink and initial-resize cases. Both JVODE wrappers use
reltol=1e-8,abstol=1e-10, and a separate analytic-reference threshold ofrtol=1e-6. The threshold bounds global solve and interpolation error separately from the local step tolerance. With the merged controller and BDF fixes, the measured relative errors are:Both modification flags produce the same errors.
timeout 3600 julia +1.12 --project=lib/OrdinaryDiffEqNordsieck ../scratch/codex-accuracy.jlpasses all 27 assertions; output is inaccuracy-1.12.log.Final test groups and static checks
Julia 1.10.12 uses a separate local environment developing this checkout's package and sibling libraries; Julia 1.12.7 uses the sublibrary environment. Both test the same source.
The allocation test's broken assertion is present in the starting head; this revision adds no skips or broken assertions. Logs:
final-core-1.12.log,final-core-1.10.log,final-qa-1.12.log.All three exit 0 without findings. Logs:
final-runic.log,final-typos.log.Not verified
Out-of-place resize, GPU/non-Vector state, downstream packages, and long post-resize AN5 trajectories beyond the regression horizon
t=0.8were not tested. A docs build was not run; this revision changes a compatibility bound and tests, with no new public names or documentation changes. CI and independent review of this revision are pending. Historical reviews below apply to their stated heads and are preserved verbatim.Please ignore until reviewed by @ChrisRackauckas.
Original implementation: Cursor Agent 2026.10.01-14929f9 (model: unknown, Cursor Auto), orchestrated by Claude Code (claude-opus-5-5[1m]), https://claude.ai/code/session_01LPHREnnonfLg1VcE1EJovv.
Risk assessment
AN5andJVODE_Adams/JVODE_BDFusers in OrdinaryDiffEqNordsieck. The new code runs only afterresize!(integrator, n), which used to throwMethodErrorfor these solvers. The rest of the diff is theresize_restartflag (false unless a resize happened, so non-resize steps take the same path as before),@cacheonAN5Cache/JVODECache, a renamed type parameter (dType→uNoUnitsType) and one new struct field. Those are internal cache types, not public API, so SemVer-wise this is a new capability (minor/patch), not a breaking change. No other sublibrary is touched.139d471a: 127 checks pass, 4 fail, 1 skipped (matrix placeholder). All 4 failures are pre-existing. Each also fails on master HEAD4a5c0e71(4a5c0e7). For the first three I compared the failure signatures; for the fourth I compared the error on head and the job status on master:tests / GPU (julia 1) / Tests - GPU: "170 passed, 3 failed, 12 broken" on both head (https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37113625117/job/111178272181) and master (https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37122715839/job/111212873505).downgrade-sublibraries / test (lib/DiffEqBase): "29 passed, 1 failed, 3 errored" on both (https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37113624882/job/111178652790 vs https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37122715590/job/111216280986).downgrade-sublibraries / test (lib/OrdinaryDiffEqDifferentiation): the sameMethodError: no method matching diag(::SciMLOperators.ScalarOperator…)on both (https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37113624882/job/111178652800 vs https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37122715590/job/111216281181).SciMLSensitivity.jl/Core1 / Downstream Tests - Core1: on head this is a MooncakeMutableTangent"no constructors have been defined for Any" error inconcrete_solve_derivatives.jl:451on a Tsit5 cache, unrelated to Nordsieck (https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37113624892/job/111176134239). The same job also fails on master (https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/37122715578/job/111205760924); I did not compare the master error text.@cache,ODEIntegrator,reinit_controller!) arepublicin OrdinaryDiffEqCore on master. SciMLBaseresize_non_user_cache!is in SciMLBase's public list at v3.39.0. No tests were weakened; the PR only adds tests.tspan=(0,0.8)to avoid this, so the tests do not cover the known weakness and the fix may be only partly correct.OrdinaryDiffEqCore = "4.4". AtOrdinaryDiffEqCore-v4.4.0,@cache,ODEIntegratorandreinit_controller!exist but are not in Core'spubliclist. Against the floor they are non-public imports, and the ExplicitImports QA check only passes because it runs against the monorepo's own Core.Δ,atmp,fsalfirstandratetmpare resized twice: once by the@cachefull_cacheand again manually. This is harmless but redundant.(#4722)reference.reltol=1e-4because of a recursion bug in the separate PR 4758; there is a merge overlap with 4758 innordsieck_perform_step.jlandnordsieck_tests.jl; and out-of-place and GPU resize are not tested.🤖 Risk assessment posted by an AI agent (fleet master) — harness: Devin CLI / fusion-claude-opus-5-5-high-sidekick-swe-2-medium; dispatched by Devin CLI 3000.11.3 fleet-master caretaker, model Fusion (claude-opus-5-5 medium + swe-2 medium)
Conversation: local transcript (no session URL)
Risk assessment (independent review, Codex gpt-6.1-sol, head 1be71a8)
git diff origin/HEAD...HEAD;git diff --check origin/HEAD...HEADpassed. All Julia commands used Julia 1.13.2,JULIA_NUM_THREADS=4,JULIA_NUM_PRECOMPILE_TASKS=4, the requested run-local depot followed by$HOME/.julia, offline package resolution, andtimeout 3600. Commands below use paths relative to the review run directory, with--compiled-modules=existing:GROUP=Core julia --project=scratch/head/review-env --compiled-modules=existing scratch/head/lib/OrdinaryDiffEqNordsieck/test/runtests.jl: Nordsieck Tests 34/34 and AN5 Startup Tests 6/6 passed. This includes every new regression assertion.julia --project=scratch/base/review-env --compiled-modules=existing scratch/regression.jl: the unchanged new regression failed on origin/HEAD (0139bdc23709266d93ebadf377d63484886d1478) withMethodError: no method matching full_cache(::AN5Cache{...}).julia --project=scratch/unfixed/review-env --compiled-modules=existing scratch/diagnostic.jl scratch/regression-all.jl: using Nordsieck source from the merge base (59ba40e4b5bbae713d8df7b792f6ea62f033ff78), all three new testsets errored with missing full_cache methods, covering AN5, JVODE_Adams, and initial resizing. The scratch wrapper groups the original testsets so failures do not stop after AN5.julia --project=scratch/head/review-env --compiled-modules=existing scratch/diagnostic.jl scratch/probes-refined.jl: 24/24 additional assertions passed across AN5, JVODE_Adams, and JVODE_BDF. Checks cover nonautonomous analytic accuracy after manual growth, deletion of a middle component, continuous-callback growth, and poisoned initial resizing without calling u_modified!.dt=0.001failed eight JVODE accuracy assertions. No-resize controls reproduced approximately1.05e-4relative error on head and default; reducing dt decreased that error proportionally. All nine no-resize diagnostic rows matched head versus merge-base source. The refined probes reduce dt to1e-5(continuous callbacks:3e-5) and retain the original assertion tolerances; no repository tests were changed.Base.ispublicchecks passed for all newly imported dependency names: @cache, ODEIntegrator, reinit_controller!, and resize_non_user_cache!. No new public names require added docstrings/docs entries, and no dependency or breaking API change was found.🤖 Posted by an AI agent — harness: Codex CLI 0.162.1 · model: gpt-6.1-sol
Conversation: local headless review run (no public session URL); transcript kept by the fleet master.
Risk assessment (independent review, Devin CLI 3000.11.92 / fusion-claude-opus-5-5-high-sidekick-swe-2-medium, head 1be71a8)
MethodError.)AN5,JVODE_AdamsandJVODE_BDFintegrators in OrdinaryDiffEqNordsieck, and only afterresize!/deleteat!/addat!. Those calls used to throwMethodError. These are peripheral solvers, not an OrdinaryDiffEq main lane. Non-resize code takes the same instructions:resize_restartstays false. I confirmed 24/24 bitwise-identical solutions against master, including same-size callbacks and out-of-place solves. No dependency, export, public name or tolerance changes.AN5Cache/JVODECachegain a field and a renamed type parameter; both are internal types.JULIA_NUM_THREADS=4, run-local depot, one process at a time, all in scratch copies (checkout untouched).98fa0512esource plus the PR's test file,GROUP=Core Pkg.test(). Result: 50 pass, 3 errored, allMethodError: no method matching full_cache(::AN5Cache…)/(::JVODECache…).98fa0512e.git merge-treeis clean; I built the tree withgit merge-file, because master has since landed the JVODE_BDF rewrite (Use BDF formulas in JVODE_BDF #4786).GROUP=Core Pkg.test(): Nordsieck Tests 60/60, AN5 Startup Tests 6/6.scratch/bitwise.jl) on master and merged: AN5, JVODE_Adams and JVODE_BDF × {in-place linear, out-of-place linear, Lorenz, same-sizeu .*= 1.1callback} × reltol {1e-4, 1e-8}. All 24 hashes of(t, u)and everynfare identical.exp(A t)(scratch/probes.jl, merged tree):step!calls,deleteat!, and a NaN-poisoned resize before the first step withoutu_modified!are all accurate or finite on all three solvers.scratch/floor_probe.jl): the PR's Nordsieck source with[sources]stripped resolves against registered OrdinaryDiffEqCore 4.6.0, OrdinaryDiffEqTsit5 2.1.0, SciMLBase 3.39.1 and DiffEqBase 7.13.1. Result:OrdinaryDiffEqNordsieck.full_cache === SciMLBase.full_cacheisfalse(a local function with 2 methods), andresize!throwsMethodErrorfor both AN5 and JVODE.scratch/mut): with head'snordsieck_perform_step.jlreplaced by master's, all 10 new assertions still pass. Only resize plusu_modified!(integrator, false)exposes the change. Without the hunks, JVODE aborts withdt_epsilon/DimensionMismatchand AN5's error goes from 6.7e-9 to 1.5e-5.resize_restartcache flag (finding 2).@cacheit relies on, and the restart-flag code is untested.OrdinaryDiffEqCorecompat floor to4.7; add a test that exercisesresize_restart(resize +u_modified!(integrator, false)); also still draft, and the PR body says "Please ignore until reviewed by @ChrisRackauckas")🤖 Posted by an AI agent — harness: Devin CLI 3000.11.92 · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Conversation: local headless Devin review run (no public session URL); transcript kept by the fleet master.
Risk assessment (independent review, Claude Opus 5.5 (claude-opus-5-5), head 1be71a8)
AN5,JVODE_AdamsandJVODE_BDFin OrdinaryDiffEqNordsieck. These are peripheral, non-main-lane solvers. The new branches run only afterresize!/deleteat!/addat!, which used to throwMethodError.resize_restartisfalseotherwise, and I showed the resulting trajectories are bit-identical to base. The internal cache structs gain a field and rename a type parameter; neither struct is public. No exports, defaults, dependencies or tolerances change, and no other sublibrary is touched.JULIA_DEPOT_PATH=<run>/depot:<local path>andtimeout≥ 2400 s, on trees extracted withgit archiveintoscratch/.GROUP=Core Pkg.test()inlib/OrdinaryDiffEqNordsieckpassed: Nordsieck Tests 34/34, AN5 Startup 6/6.MethodError: no method matching full_cache(::AN5Cache{...}), 24 pass. Fails before, passes after.GROUP=QApassed. AllocCheck: 2 pass plus 1 pre-existing broken. JET 1/1, Aqua 21/21.u-modifying DiscreteCallback, at reltol 1e-7 and 1e-4. Thehash((sol.t, sol.u)), step counts andnfall match exactly in every case.deleteat!at t=0.6, out to t=3. It is checked against a Vern9 solve at reltol 1e-14 with the same callback.resize!/deleteat!betweenstep!calls: 4.1e-8 at t=3.uprev[3]=NaNand 1000 fixed steps againstexp(A t): AN5 relerr 4.7e-15, JVODE_Adams/BDF 1.7e-5.copyto!, because Core already syncsuprev.🤖 Posted by an AI agent — harness: Claude Code · model: claude-opus-5-5
Conversation: local headless review run (no public session URL); transcript kept by the fleet master.
Risk assessment (review-fix revision)
1be71a8; the Devin CLI / fusion-claude-opus-5-5-high-sidekick-swe-2-medium review requested the three changes addressed here. Independent review of the new head is pending. Review history: Make AN5 and JVODE caches resizable after resize! #4759.🤖 Updated by an AI agent using Codex CLI 0.163.0 (model: gpt-6.1-sol). Session: local session, transcript
<local path>on a CI worker host.Independent current-head review of eb80c56
CHANGES for required original/continuation session attribution. Strong numerical independent evidence: Nordsieck77/77/startup6, base with newtests50pass7errors; removing AN5restart76pass1fail and JVODErestart69pass6fail2errors. QA unchanged with polling resolves EMFILE; Core4.7floor6/6, independent matrix exponential controls. Separate clean default AN5 adaptive/restart defect gives5pass1fail/error0.126 despite tight tolerances, plus no-callback control; queue separate bisection, not a new resize regression. Do not claim original unavailable author scripts or full monorepo coverage.
Risk assessment
Links:
🤖 Posted by an AI agent — harness: Codex CLI 0.163.0 · model: gpt-6.1-sol
Conversation: local session 01a154f0-8986-78e1-8af2-1119e8cdea4a (no session URL)
Assigned correction of eb80c56
This correction is queued to a fresh Codex CLI worker, model
gpt-6-luna, under the SciML submaster’s pipeline ownership. Scope: Correct original/continuation session attribution and footer from verified runtime records only; preserve source and human history, stop if signed ancestry or missing original identity prevents a safe correction. Existing ML2 numerical assessment and separate AN5 baseline investigation remain.This assignment permits metadata work only, with no Julia/test execution or code changes. Historical attribution corrections require verified original identities, complete source/author/date/message preservation proofs and a guarded expected-head update. Missing records must be reported honestly; no identity or review is fabricated.
The worker does not merge, approve or close PRs. Any pushed revision needs a fresh independent counted review; existing source reviews are not silently repinned. This assignment does not change the PR’s risk level or resolve any finding.
Links:
🤖 Posted by an AI agent — harness: Codex CLI 0.163.0 · model: gpt-6.1-sol
Conversation: local session 01a154f0-8986-78e1-8af2-1119e8cdea4a (no session URL)