Skip to content

Compare the remade chain_dae solve at the init tolerance - #5413

Open
piyushraj44 wants to merge 1 commit into
SciML:masterfrom
piyushraj44:remade-init-tolerance
Open

piyushraj44 wants to merge 1 commit into
SciML:masterfrom
piyushraj44:remade-init-tolerance

Conversation

@piyushraj44

Copy link
Copy Markdown

model_independent_initialization.jl remakes each problem from sol.u[1] and checks remadesol.u[end] ≈ sol.u[end] at the default √eps. The remade solve runs initialization again, starting from sol.u[1], and that point is only converged to the init solve's tolerance (abstol = 1e-6). So whenever the nonlinear solver takes one more step there, the start moves by up to that much and the end states can differ by more than √eps, even though both inits are fine.

That's what happens with SciML/OrdinaryDiffEq.jl#5010, which makes OrdinaryDiffEq's default init solve start SCC blocks at Newton instead of Broyden. It's a follow-up to SciML/OrdinaryDiffEq.jl#5008. For chain_dae with FBDF:

init error vs exact root remade init moves u(t0) by max abs diff at t_end ≈ (√eps) rtol = 1e-6
current OrdinaryDiffEq, n=6 2.5e-9 0 0 passes passes
with the OrdinaryDiffEq change, n=6 3.3e-7 3.3e-7 1.5e-8 (rel 1.83e-8) fails, limit 1.49e-8 passes
current OrdinaryDiffEq, n=9 1.2e-7 2.3e-7 6.9e-10 passes passes
with the OrdinaryDiffEq change, n=9 3.3e-7 3.3e-7 5.9e-10 passes passes

Both solver orders land on the same root, inside the requested tolerance. The n=9 case already moves by 2.3e-7 at init today; it just stays under √eps at the end. So this compares the two end states at rtol = 1e-6, with a comment saying why. The other checks in the testset are unchanged. The new tolerance is looser, so the test passes wherever it passed before.

I ran the Initialization group locally on Julia 1.13.4, with this branch and the OrdinaryDiffEq change dev'd: 1015 pass, 12 broken, 0 failures. Without this change, the same setup fails only this check, for chain_dae(6).

The remade problem in model_independent_initialization.jl runs
initialization again, starting from sol.u[1]. That point is only
converged to the init solve's abstol (1e-6), so when the nonlinear solver
takes one more step there, the start moves by up to that much and the
end states can differ by more than the default sqrt(eps).

That happens once OrdinaryDiffEq's default init solve starts SCC blocks
at Newton: for chain_dae(6) with FBDF the init lands 3.3e-7 from the
root, the remade init moves it by 3.3e-7, and the end states differ by
rel 1.83e-8 (the limit is 1.49e-8). Compare them at rtol = 1e-6 instead.
@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

PR: #5413

Reviewed head: 0c9118d
Raw SHA-256: dce573609931cc4110c34ee1b7dad8fd154679ce9be49947231b6fb83d9f58ee
Source SHA-256: fb83b595bfb1b55f41600ad2b26e3553330a0f6a93812e9f7c1e29632d551dd4
Source: standard-review-v1 (harness-log)
Reviewer session: 01a1eda7-57b6-7982-bea4-da83f203b01c
Slot: 1
Confidence: unknown
Own risk fixed before author rating/discussion: yes
Redaction: ran (substitutions=0)

Risk assessment

  • Risk: medium-low (ML5)
  • Verdict: MERGE
  • Reviewed head: 0c9118d

Review report

VERDICT: MERGE
Round: 1
Head: 0c9118d
Slot: 1
CI: CI unreadable; review only; merge gates unchanged
Session: 01a1eda7-57b6-7982-bea4-da83f203b01c

Risk

medium-low (ML5: explained test-tolerance relaxation; explicit Newton initialization reproduces the old failure, and tighter initialization restores agreement).

Blocking findings

None.

  • Blocking priorities: none

Non-blocking findings

  • test/model_independent_initialization.jl:58: Consider retaining the original tolerance for chain_ode. Both ODE sizes produced bitwise-identical original/remade endpoints in the probes; the nonlinear-initialization explanation concerns the DAE cases. Restricting the relaxation would preserve the stronger ODE check.

What I ran

  • Inspected git diff origin/master...HEAD, root instructions, the official test dispatcher and Initialization group. The diff changes only one assertion and its explanation; no production code or dependencies change. git diff --check origin/master...HEAD passed, and the checkout remained clean.
  • Julia 1.13.2. Created a scratch environment from the repository's dependency, test-extra and compatibility declarations, with local package sources and the supplied layered JULIA_DEPOT_PATH preserved. Resolved versions included OrdinaryDiffEq 7.8.1, OrdinaryDiffEqNonlinearSolve 2.9.10 and SciMLBase 3.58.0. The repository supplies no Julia manifest.
  • GROUP=Initialization JULIA_NUM_THREADS=4 JULIA_NUM_PRECOMPILE_TASKS=4 julia --startup-file=no --project=../scratch/env test/runtests.jl: exit 0; 1015 passed, 12 broken, 1027 total, 19m55.9s. The broken markers were not introduced by this diff.
  • julia --startup-file=no --project=../scratch/env ../scratch/probes.jl "$PWD": exit 0, PROBES_DONE. Exercised both chain sizes with default initialization, initialization tolerances 1e-10/1e-12, and explicit OverrideInit(nlsolve=NewtonRaphson()).
    • With Newton, DAE size 6 failed the old comparison and passed the new one: relative endpoint difference 1.8286714986280824e-8, initial movement 3.2724134890749212e-7. The algebraic residual was 7.661489705990121e-7, below the initialization absolute tolerance.
    • Tightening Newton initialization to 1e-12 reduced that endpoint difference to 7.877888412828606e-14. This supports benign initialization accuracy as the cause.
    • Independently eliminated each cubic using its analytic real root and integrated the resulting explicit ODE. Tight-reference solutions agreed within 2.40e-14 and 2.04e-14 for sizes 6 and 9. This was an accuracy cross-check, not a fitted expected value.
  • uv run --locked python --version: Python 3.15.0, with warning that --locked has no effect outside a project. No uv.lock or Python project exists here; this does not establish a locked project Python interpreter.

What I did not verify

  • CI, other Julia versions, the full test matrix, QA or documentation builds. The resolved default initializer did not reproduce the old failure; explicit Newton initialization did.
  • Requested Devin provenance was unavailable: devin list --format json returned [] in both the checkout and review directory. The Session field therefore records the actual runtime CODEX_THREAD_ID; no Devin session name was invented.
  • No PR body, comments, author risk rating or earlier review was read. No checkout edits, commits or GitHub writes were made. All started commands finished.

AUTHOR_GRADE: {"task":"test","score":4,"difficulty":2,"rounds":1,"author_harness":"unknown","author_model":"unknown","note":"Initialization group passed; explicit Newton reproduced the original assertion failure, and tighter initialization removed the discrepancy. The change is a justified test-tolerance adjustment with no production-code change."}

Head link: 0c9118d


🤖 Posted by an AI agent — harness: codex-cli 0.169.0 · model: gpt-6.1-sol
Conversation: local session 01a1eda7-57b6-7982-bea4-da83f203b01c (no session URL)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants