Skip to content

Start the default DAE init solve at Newton for small systems - #5008

Open
piyushraj44 wants to merge 2 commits into
SciML:masterfrom
piyushraj44:init-newton-start
Open

piyushraj44 wants to merge 2 commits into
SciML:masterfrom
piyushraj44:init-newton-start

Conversation

@piyushraj44

@piyushraj44 piyushraj44 commented Nov 17, 2026 •

Copy link
Copy Markdown

Fixes #5005.

default_nlsolve was building FastShortcutNonlinearPolyalg without u0_len, so the default initialization solve always started at Broyden. NonlinearSolve's own solve(prob, nothing) passes u0_len and starts at Newton for ≤ 25 unknowns. Broyden can wander off to a distant root that still reports Success, which is what happens with the two-rod linkage in the issue.

What changed

  • u0_len is now passed in the plain AbstractNonlinearProblem methods, which is where OverrideInit, BrownFullBasicInit and ShampineCollocationInit end up. The size comes from the init problem, not from u, because BrownFullBasicInit passes the full state but only solves for the algebraic vars.
  • GPU arrays keep the old Broyden-first order. If ArrayInterface.fast_scalar_indexing is false for u or for the init problem's u0, no u0_len is passed. Newton needs a dense ForwardDiff Jacobian on its very first step, and seeding one scalar-indexes the array. Broyden starts from an identity Jacobian, so those cases behave exactly like before.
  • SCCNonlinearProblem init problems are not changed here. More on that below.
  • The least-squares methods are left alone. FastShortcutNLLSPolyalg has no u0_len and already starts at Gauss-Newton.
  • I didn't touch the homotopy inner corrector, since it's meant to mirror FastShortcutHomotopyPolyalg upstream.

A second bug this exposed

Once Newton runs first, the init solve takes a ForwardDiff Jacobian right away, and the issue's MWE then crashed with No matching function wrapper was found!. The cause:

  • An AutoSpecialize init problem (the default for a hand-built NonlinearProblem passed through OverrideInitData) gets wrapped by NonlinearSolveBase for NonlinearSolveTag Duals.
  • standardize_forwarddiff_tag only swaps that tag into an untagged AutoForwardDiff, so the OrdinaryDiffEqTag we were forcing never matched.
  • On master this stayed hidden because Broyden starts from an identity Jacobian. Any OverrideInit solve that fell through to Newton would have hit it.

The OrdinaryDiffEqTag (#3067) is only needed for the FullSpecialize residuals that Brown and Shampine build, because those call the ODE's own wrappers. So in the plain methods, FullSpecialize problems keep the tag, and everything else gets an untagged backend with the same chunk size, letting NonlinearSolve choose the matching tag. SciMLBase.specialization and the untagged re-stamping both exist at the current compat floors, so no compat bumps are needed.

Why SCC problems are left out for now

My first push also sized SCC problems by their largest block. That made model_independent_initialization.jl fail in the ModelingToolkit Initialization downstream group. The test solves chain_dae(6), remakes the problem from sol.u[1], and checks remadesol.u[end] ≈ sol.u[end] at the default √eps tolerance. I reproduced it locally, with the same 1.5079e-8 difference CI printed:

init error vs exact root remade init moves u(t0) by max abs diff at t_end ≈
master (Broyden first), n=6 2.5e-9 0 0 passes
Newton first, n=6 3.3e-7 3.3e-7 1.5e-8 (rel 1.83e-8) fails, limit is 1.49e-8
master, n=9 1.2e-7 2.3e-7 6.9e-10 passes
Newton first, n=9 3.3e-7 3.3e-7 5.9e-10 passes

Both orders find the same (unique) root and both stop inside the requested abstol = 1e-6. The remade solve then starts from a point that is already converged, and Newton polishes it a little further. So it's a tolerance thing rather than a wrong answer, and master's n=9 row moves by about the same amount. Still, switching SCC problems changes MTK's init results and needs a matching tweak to that test, so I'd rather do it as a separate PR together with MTK than fold it in here. The SCC methods are byte-for-byte what master has.

Tests

default_nlsolve_u0_len_tests.jl (new) covers:

  • the start index for in-place and out-of-place problems, with and without AD
  • the 25/26 boundary
  • u longer or shorter than the problem
  • arrays without fast scalar indexing (a small stand-in for GPU arrays) keeping the old order
  • SCC and NLLS defaults unchanged
  • the tag choice for FullSpecialize and AutoSpecialize problems, including a full solve through Newton's Jacobian
  • the issue's linkage end to end through both OverrideInit and BrownFullBasicInit

On master both of those last paths return [24.0228, 5.6732]. With this PR both return [-0.7, -1.2].

I changed one existing assertion. The reinit! test in dae_initialization_tests.jl checked atol = 1e-8, but the init solve runs at the integrator's default abstol = 1e-6. Newton (starting from u = 1) stops at 2.0000000929, with a residual of 3.7e-7, inside the tolerance. Broyden happened to stop closer. The test is checking that reinit! reruns init (2 rather than the 0 guess), so I set the assertion to the solve tolerance and left a comment.

Run locally on Julia 1.13.4 with every sublibrary in this repo dev'd (NonlinearSolve 4.32.0, NonlinearSolveBase 2.55.1, SciMLBase 3.58.0, ModelingToolkit 11.46.1):

suite result
OrdinaryDiffEqNonlinearSolve Core, all 33 files (--check-bounds=yes) 1588 pass, 40 broken, plus the 1 error + 4 failures described in the next section (not from this PR)
↳ default_nlsolve_u0_len_tests.jl (new) 30/30
↳ dae_initialization_tests.jl / brown_inplace_tests.jl / sparse_dae_initialization_tests.jl / checkinit_tests.jl 80/80, 62/62, 39/39, 4/4
OrdinaryDiffEqNonlinearSolve QA (Aqua + ExplicitImports) 40/40
OrdinaryDiffEqNonlinearSolve JET 13 pass, 33 broken (same as CI)
OrdinaryDiffEqNonlinearSolve ModelingToolkit group 62 pass, 2 broken (the @test_broken in nlstep_tests.jl)
InterfaceV (incl. the JLArray GPU DAE test) 51/51
ModelingToolkit Initialization group 1015 pass, 12 broken, 0 fail (CI on the first push: 1014 pass, 1 fail)

I couldn't run test/gpu/dae_tests.jl here (no CUDA); the JLArray test exercises the same scalar-indexing path.

CI failures that aren't from this PR

These fail with identical counts on master (or on other PRs touching this sublibrary):

  • NonlinearSolve sublibrary Newton Tests: a LineSearchException from BackTracking inside the stepper's NLNewton, also failing on Resize the member caches of a CompositeCache in a DDEIntegrator #4958. Its two problems (Lorenz, Oregonator) have identity mass matrices, so the init code isn't reached; I double-checked by making the new init helpers throw and it still fails the same way.
  • SciMLSensitivity Core2 (194 pass, 4 errored), ProbNumDiffEq (1730 pass, 6 failed) and AD on lts (Zygote error in autodiff_events.jl), same numbers as master.
  • GPU autoswitch.jl (also in the OrdinaryDiffEqDefault GPU job): a GPUCompiler deprecation warning trips @test_nowarn, same on master.

One more that CI can't show yet, because Newton Tests aborts the Core run before it gets there: locally, nsa_noinit_tests.jl has 4 MaxIters cells (TRBDF2 with each Simple* inner solver on the Robertson mass-matrix problem). I get exactly the same 4 with master's init code, and the init helpers are never called there, since u0 is already consistent and Brown returns before solving anything.

One trade-off to flag

The issue mentions that Broyden and Klement stay available as fallbacks. They don't: the polyalgorithm never goes below start_index, so small systems skip them entirely. That matches what NonlinearSolve's default already does. The practical difference: if the initial guess sits exactly where the Jacobian is singular (e.g. u^2 - 4 from u = 0), the Jacobian-based methods stall where Broyden could have moved. solve(prob) in NonlinearSolve stalls on that too. A loud InitialFailure seemed like a better failure mode than silently landing on the wrong root, but I'm happy to change it if you'd rather handle it differently.

default_nlsolve built FastShortcutNonlinearPolyalg without u0_len, so the
init solve always started at Broyden. On small systems that can hop to a
far-away root and still report Success, e.g. the two-rod linkage in SciML#5005
lands at ~24 rad from a guess 0.005 rad off. NonlinearSolve's own default
passes u0_len and starts at Newton for <= 25 unknowns, so do the same.

The size is taken from the init problem rather than from `u`, since
BrownFullBasicInit passes the full state but only solves the algebraic
part. SCC problems are sized by their largest block.

Starting at Newton means the first thing the solve does is a ForwardDiff
Jacobian, which exposed a tag mismatch on the OverrideInit path: an
AutoSpecialize init problem gets wrapped by NonlinearSolveBase for its own
tag, and it only re-tags untagged backends, so forcing OrdinaryDiffEqTag
there throws NoFunctionWrapperFoundError. Keep OrdinaryDiffEqTag for the
FullSpecialize residuals Brown/Shampine build (they call the ODE's
wrappers) and hand an untagged backend to everything else.

The reinit! test in dae_initialization_tests asserted 1e-8 while the init
solve runs at abstol 1e-6. Newton now stops at 2.00000009 (inside the
tolerance) where Broyden happened to stop closer, so the assertion now
matches the solve tolerance.

Fixes SciML#5005
The first CI run turned up three things with starting the init solve at
Newton:

- On GPU arrays Newton's first ForwardDiff Jacobian scalar-indexes the
  state (test/gpu/dae_tests.jl and the JLArray DAE test in InterfaceV).
  Only pass u0_len when both `u` and the init problem's u0 have fast
  scalar indexing, so GPU states keep the Broyden-first order, which
  doesn't need a Jacobian to get going.
- SciMLBase.state_values isn't public there, which the QA explicit
  imports check flags. Read nlprob.u0 directly instead.
- Sizing SCC problems moved MTK's chain_dae init around inside abstol,
  and its remake test compares two solves at sqrt(eps), so it tipped
  over (1.83e-8 vs 1.49e-8). Put the SCC methods back the way they are
  on master for now. That change needs coordinating with MTK.
@singhharsh1708

Copy link
Copy Markdown
Member

LGTM, the #5005 MWE gives [-0.7, -1.2] here against [24.02, 5.67] on master, for OverrideInit and BrownFullBasicInit in place and out of place.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Default DAE initialization starts with Jacobian-free Broyden and can return a distant root

2 participants