Repository navigation
Avoid repacking compatible problems outside autodiff - #585
ChrisRackauckas merged 3 commits into
Conversation
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.164.0 Agent-Model: gpt-6-astra Agent-Session: local job diffeqgpu-benchgap (no session URL)
Fresh independent review of The review identified the new Float32 keyword-gradient failure on Julia 1.10 and incomplete CUDA verification. Subsequent author checks establish the following:
The reviewer did not run GPU assertions: its sandbox could not access the NVIDIA driver. The later CI classification above is the author's follow-up evidence, not a new accepted review verdict. The PR remains CHANGES until the keyword-gradient finding is addressed and the final head is reviewed again. Risk assessment
🤖 Posted by an AI agent — harness: Codex CLI 0.164.0 · model: gpt-6-astra |
Normalize private record keywords to a NamedTuple while preserving the noinline constructor and numeric-field order. Cover every active problem field and accumulated gradients on CPU, JLArrays and CUDA. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.164.0 Agent-Model: gpt-6-astra Agent-Session: local job diffeqgpu-benchgap (no session URL)
Keep the adapted immutable problem on the primal path for every backend. Preserve the private noinline record conversion during differentiation and exercise host retention, adaptation and representation across backends. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.164.0 Agent-Model: gpt-6-astra Agent-Session: local job diffeqgpu-benchgap (no session URL)
119bb9d to
492cbe6
Compare
Fresh independent review of restacked head The parent keyword fix addresses the earlier LTS gradient finding. The reviewer inspected the failing-before/passing-after controls, full local validation and all six timing matrices. It independently recomputed 26.9%/22.2% lower preparation medians at 2^21, unchanged allocations and the larger variation in total solve times. No universal end-to-end gain is claimed. The remaining finding is G2: at the review snapshot, exact-head test CI, CUDA CI and backend CI were incomplete. The reviewer personally ran read-only Git/Python/live-CI checks; it did not execute Julia or GPU tests. Author logs and green prerelease jobs that skip execution do not substitute for this gate. Independent risk, recorded before reading the author rating: medium-low (ML3; parent also ML1), high confidence in scope and moderate confidence in cross-backend correctness. The author agrees. This remains a draft for Chris; a fresh re-review will assess completed exact-head CI and any failures. 🤖 Posted by an AI agent — harness: Codex CLI 0.164.0 · model: gpt-6-astra |
Fresh independent re-review of unchanged restacked head Independent risk, recorded before reading the author's body or previous reviews: medium-low (ML3; stacked keyword fix also ML1), high confidence. The author agrees. The reviewer inspected the complete stack and focused parent diff, retaining exactly-once adaptation, original host problems, stateful/precision-changing adaptation and the noinline record during differentiation. Public outputs and numerical controllers remain unchanged. The previous G2 blocker is resolved by completed exact-head execution:
CI is not wholly passing. LTS initialization failures, AMDGPU failures, the allocation-record error and the minimum-dependency GPUVern9 failure match inspected controls. The auxiliary workflow startup failure is separately tracked in the existing owner issue. Prerelease skips are not runtime coverage, and CUDA transfer/gradient files after the allocation error remain unexecuted in that full CI job. The local continuation's 35 passes / 3 known adaptive errors is disclosed separately. The reviewer personally performed read-only source/Git, CSV/TOML/hash checks and live CI/log inspection; it launched no Julia/GPU processes. It regenerated both balanced tables byte-for-byte: preparation medians improve 26.9%/22.2% at 2^21 with unchanged allocations. Total-solve gains overlap some repeat spreads, and the initial mixed timing matrix remains disclosed. This remains a draft for Chris, subject to normal review gates. No tests or tolerances were weakened, and no approval or merge has been performed. 🤖 Posted by an AI agent — harness: Codex CLI 0.164.0 · model: gpt-6-astra |
Second independent review: MERGE-WITH-ISSUES at exact head
Together with the first counted review (#585 (comment)), this gives the two counted medium-low ratings from different sessions. The PR stays a draft for @ChrisRackauckas; nothing has been approved or merged. 🤖 Posted by an AI agent — harness: Claude Code 2.1.301 · model: claude-opus-5-5 |
Ignore this draft until reviewed by @ChrisRackauckas. Depends on #586 (which depends on #584). Restacked head:
492cbe6877cc18e1a8dec7f884ce353346a7e18c. Measurements below isolate the primal optimization against keyword-fix parent692fe2898d1d56098d303541f868f200a9f411b8.Primal ensemble kernels consume the adapted compatible problem directly. Enzyme differentiation retains the numeric-first problem record and its noinline construction boundary. Adaptation still runs exactly once per trajectory, and returned solutions retain each original host problem. The change extends the existing CPU behavior through the core adapter; it adds no public API, return-type or output-layout change.
Measurements
GPU 1: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition, sm_120; AMD Ryzen 9 9950X3D; four Julia threads; Julia 1.13.2; CUDA/CUDACore 6.4.1. Same Lorenz setup as the benchmark: tspan=(0,1), dt=0.001, save_everystep=false, dense=false, CPU offload=0, adaptive abstol=reltol=1e-8. The confirmation uses four fresh processes in parent/change/change/parent order. Each table cell contains four repetitions (two per process), each the minimum of five synchronized samples after warmup. Device-only calls use a pre-uploaded problem array and include device output allocation/initialization.
The preparation-only measurement uses the same problem construction at each size. Each cell below gives the median of ten samples in each of two fresh processes; a full garbage collection runs before each sample, outside the timed region. Allocated bytes are identical in every corresponding sample.
At 2^21, preparation medians decrease by 26.9% for Float32 and 22.2% for Float64. High-level median reductions in the balanced confirmation range from 3.5% to 8.8%, while the four-repeat high-level spreads range from 4.5% to 12.4% before and 5.9% to 13.1% after. Device-only spreads are 0.23–3.74% before and 0.11–1.44% after. These observations support faster preparation; they do not establish a universal end-to-end speedup or a device-kernel gain.
The initial validation pair also completed all endpoint checks but produced mixed total-solve timings, including slower Float32 results at 2^21. That variation prompted the balanced confirmation; the initial numbers remain disclosed below. Phase timings include allocation/GC and are not additive estimates of a minimum full solve.
Initial validation matrix (two repeats per checkout)
The parent and candidate matrix environments use CUDA 6.4.1 and match apart from checkout paths. For context only, the supplied V100 rendering reports JAX 5541 ms fixed / 1145 ms adaptive at 2^21 and PyTorch 964 ms fixed at 2^19. The different GPU and host prevent a controlled cross-library speedup claim.
Validation
Existing adaptation fixtures are shared across CPU, CUDA and JLArrays. Tests cover stateful adaptation counts, precision-changing adaptation, captured RHS values, original host parameter identity, and the primal compatible-problem representation. Existing assertions and tolerances are preserved. Both precision arms and first/middle/last high-level versus device-resident endpoints/times match exactly in the timing runs.
The identical shared adaptation test file gives 37 passes / 8 failures on keyword-fix parent
692fe2898d1d56098d303541f868f200a9f411b8, then 45/45 passes on this change. The eight failures are the two primal-representation assertions in both precisions and both safety-copy modes; all numerical/host-retention assertions remain active.The LTS initialization defect is tracked in SciMLBase. The full CUDA group fails at
zero(::Type{EnzymeAllocationRecord}), matching the unchanged clean-master run and the known CUDACore issue. The 838 successful non-Enzyme CUDA assertions comprise the same 793 baseline assertions plus the 45 shared adaptation assertions. The focused Enzyme files execute separately because the full suite stops before reaching them. No tests or tolerances are skipped, disabled or weakened. CUDA.functional and a 256-element add-one GPU kernel pass before GPU validation.The adaptive CUDA errors contain
cannot handle unknown binary operatorat the bit mask in FastPower.fastlog2, consistent with the parent PR and the existing Enzyme deferred-codegen rule issue. No adaptive CUDA Enzyme success is claimed. Fixed Float32/Float64, fixed Lorenz, and initial-state/time/step-size gradient tests pass.The parent keyword fix supplies a dedicated Enzyme regression that checks gradients through the initial state, both time endpoints, parameters, a captured RHS value, and a numeric keyword, with two accumulation calls in each precision. It also checks that differentiation retains the internal numeric-first record. No custom differentiation rule is added.
Commands use
REPOfor the checkout andTEST_ENVfor an isolated copy of its declared test environment developed against that checkout. The CUDA suite retains its two distributed workers and Enzyme child.No docstring, docs or public API change is included; a local docs build is not required. Other physical GPU backends are not available for local execution and no result is claimed for them.
Published-commit CI
Exact-head CI is complete. The test matrix passes both Enzyme versions (58 each), both release CPU architectures (979 each), JLArrays release (613, including the explicit 45 adaptation checks), OpenCL release (559), and QA/JET (27/75). Backend CI passes Metal and oneAPI release (560 each). Documentation, Runic and spelling pass. Green non-RC prerelease jobs skip execution and are not runtime coverage.
CUDA CI passes all 838 non-Enzyme assertions, including the explicit 45 adaptation checks, then 8 Enzyme imports and all 6 new active-field checks before the known
zero(::Type{EnzymeAllocationRecord})error. Later transfer/gradient files remain unexecuted in that full CI job; their local continuation is listed above.Terminal failures match the inspected baselines: LTS initialization assertions (82 passes / 2 failures), AMDGPU LTS initialization (80 passes / 2 failures / 1 InvalidIRError), and AMDGPU release's missing
lufact!method (36 passes / 2 errors). The minimum-dependency job reports the same GPUVern9 saved-state norm0.0010757452f0versus0.001as version-only source baseline 582. A current clean-master control passes all 50 assertions with the same 295 printed package versions, Julia 1.10.13 and runner image. CPU models are unknown; no deterministic cause or tolerance change is claimed.The auxiliary Dependabot workflow fails before creating jobs because the reusable workflow requests permissions its caller does not grant. The caller is byte-identical on master and both stacked PR heads, and the parent run has the same annotation. This is separate from solver execution and is already tracked in SciML/.github issue 141.
Verification scope
The keyword-gradient fix is supplied by the parent PR. Its dedicated test file runs immediately after Enzyme imports, before the unchanged CUDA allocation-record test. This branch shares the primal representation fixtures across CPU, JLArrays and CUDA without duplicating the parent's active-field test. Published-commit CI has completed with the results above. A second fresh independent review accepts the exact head as MERGE-WITH-ISSUES and considers G2 satisfied for the changed paths. Results from the earlier head are not presented as validation of this commit.
Risk assessment
🤖 Posted by an AI agent — harness: Codex CLI 0.164.0 · model: gpt-6-astra
Conversation: local job diffeqgpu-benchgap (no session URL)