Repository navigation
Build EnsembleGPUArray batch matrices in one allocation - #550
Closed
ChrisRackauckas-Claude wants to merge 1 commit into
Closed
ChrisRackauckas-Claude wants to merge 1 commit into
ChrisRackauckas-Claude wants to merge 1 commit into
Conversation
Collect state and parameter columns into a vector before reduce(hcat, ...). Base allocates that reduction once; a generator left-folds and copies Θ(n²). A one-column batch still returns that column. Non-array parameters become one row. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Grok <noreply@x.ai> Agent-Harness: Grok Build 1.0.41 Agent-Model: grok-4.7 Agent-Session: /home/crackauc/.grok/sessions/%2Fhome%2Fcrackauc%2Fsandbox%2Ftmp_20260923_091211_54765%2Fgrok-ensemblegpuarray/01a0d6f0-b9fb-79e2-9044-9bcfa0e598c9
This was referenced Sep 26, 2026
ChrisRackauckas-Claude
pushed a commit
to ChrisRackauckas-Claude/DiffEqGPU.jl
that referenced
this pull request
Sep 27, 2026
Stack on SciML#550's `_hcat_batch`: keep ordinary scalars as a 1-D vector (including n = 1) instead of a 1×n matrix or bare Number, and select `p[i]` vs `p[:, i]` via `ensemble_param` so length-1 arrays stay arrays. Apply the same packing in the lower-level array path. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu
ChrisRackauckas
added a commit
that referenced
this pull request
Sep 27, 2026
* Build EnsembleGPUArray batch matrices in one allocation Collect state and parameter columns into a vector before reduce(hcat, ...). Base allocates that reduction once; a generator left-folds and copies Θ(n²). A one-column batch still returns that column. Non-array parameters become one row. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Grok <noreply@x.ai> Agent-Harness: Grok Build 1.0.41 Agent-Model: grok-4.7 Agent-Session: /home/crackauc/.grok/sessions/%2Fhome%2Fcrackauc%2Fsandbox%2Ftmp_20260923_091211_54765%2Fgrok-ensemblegpuarray/01a0d6f0-b9fb-79e2-9044-9bcfa0e598c9 * Preserve scalar EnsembleGPUArray parameter contract through kernels. Stack on #550's `_hcat_batch`: keep ordinary scalars as a 1-D vector (including n = 1) instead of a 1×n matrix or bare Number, and select `p[i]` vs `p[:, i]` via `ensemble_param` so length-1 arrays stay arrays. Apply the same packing in the lower-level array path. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu * Test EnsembleGPUArray scalar RHS for batch_size 1, 2, and 4. Use the issue's unchanged `p * u[1]` scalar RHS plus vector and inplace cases with analytic endpoints; import Tsit5 from OrdinaryDiffEq. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu * Keep singleton array parameters as n×1 matrices for EnsembleGPUArray. `_hcat_batch` returned a bare vector for n=1, which `ensemble_param` treated as a scalar batch and truncated. Pack ordinary array `p` via `hcat`/`reduce` so columns stay matrices, and restore lower-level `u0` to `hcat` for n×1 shape. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu * Regress EnsembleGPUArray vector p with typed RHS and singleton remainder. Cover sum(p) endpoints for batch_size 1–4, length-1 AbstractVector RHS, and lower-level singleton state shape (1,1), while keeping the scalar issue RHS. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu * Support scalar vector packs in batch_solve_up reverse-mode seeding. `seed_duals` now accepts the 1-D scalar layout from `pack_ordinary_parameters`, keeping Dual parameters as scalars at the RHS, and the pullback returns a vector cotangent instead of forcing a matrix. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu * Test batch_solve_up rrule for scalar and array parameter packs. Analytic ∂u/∂p = exp(p) for scalar batches (including singleton) and exp(sum(p)) components for matrix packs, via the package ChainRules rule. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Cursor Agent <noreply@cursor.com> Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe Agent-Model: auto Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/DiffEqGPU.jl/jobs/552-cursor/log.txt on amdci2.julia.csail.mit.edu --------- Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com> Co-authored-by: Grok <noreply@x.ai> Co-authored-by: Cursor Agent <noreply@cursor.com>
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
batch_solveinsrc/solve.jlbuilt each EnsembleGPUArray batch withreduce(hcat, generator). On a generator that is a left fold, so every added trajectory copies the matrix built so far and a batch ofntrajectories allocates Θ(n²).Base.reduce(hcat, ::AbstractVector{<:AbstractVecOrMat})allocates once, so the columns are collected into aVectorfirst (src/solve.jl:324foru0, and the same helper forp). A one-column batch still returns that column. Parameters that are not arrays (scalars andParamWrapper) are placed withreshape(cols, 1, :):reduce(hcat, ::Vector{<:Number})is only the pairwise reducer and still allocated about 2.7 GiB at n = 2^20.CPU only, Julia 1.11.9,
KernelAbstractions.CPU(),EnsembleGPUArray(backend, 0.0), Float32 Lorenz,RK4,dt = 0.001,save_everystep = false. One solve.Before, DiffEqGPU v3.21.2:
u0 allocation scaled exactly ×16 when n ×4 (384 GiB at 2^18, 6144 GiB at 2^20). Log-log exponent of the u0 concatenation was 2.08.
After this change:
The full solve is linear (exponent 1.04 across 2^14…2^20). The remaining time is the ODE.
Same script, one solve at n = 2^18, separate environments. The fold is unchanged across these releases. It was introduced in
16997f1a(2020-08-22, “asynchronous mixing of CPU with GPU”). v3.10 callsprob_func(prob, i, repeat); v3.14+ passes anEnsembleContext. Allocations match.test/ensemblegpuarray_hcat.jlchecks that allocations grow by less than 8× when the trajectory count grows 4× (n = 1024 vs 4096).On master (
git show HEAD:src/solve.jlrestored, test file present):With this commit:
GROUP=CPU ~/.juliaup/bin/julia +1.11 --project=. -e 'using Pkg; Pkg.test()'on this commit:Testing DiffEqGPU tests passed.Public generic interfaces19/19,EnsembleGPUArray host concatenation6/6.Not verified: CUDA or any GPU run.
Found from SciMLBenchmarks
benchmarks/DiffEqGPU/lorenz_ensemble.jmdon a V100, whereEnsembleGPUArrayat 2^21 trajectories took about 7.2 h: https://github.com/SciML/SciMLBenchmarks.jl/actions/runs/35832799318Ignore this draft until reviewed by @ChrisRackauckas.
Risk assessment
solveonEnsembleGPUArrayare the callers.GROUP=CPUpassed. No GPU run.🤖 Generated with Grok Build (model: grok-4.7)
Session: /home/crackauc/.grok/sessions/%2Fhome%2Fcrackauc%2Fsandbox%2Ftmp_20260923_091211_54765%2Fgrok-ensemblegpuarray/01a0d6f0-b9fb-79e2-9044-9bcfa0e598c9
Risk assessment
Risk: low
Blast radius: Every
EnsembleGPUArraysolve, on all backends, runs throughbatch_solveinsrc/solve.jl. The change covers how theu0matrix is built, plus thepmatrix in both branches (theParamWrapperbranch for differing tspans and the plain-pbranch). The new_hcat_batchreturns the same results as the oldreduce(hcat, generator)fold: the same matrix shapes, a one-trajectory batch still returns its bare column or scalar, and non-arraypbecomes a 1×n row. There are two small differences, both on inputs that are already broken. A batch whosepmixes arrays and non-arrays is now routed bycols[1] isa AbstractArray. An empty batch still throws, now as aBoundsError. Public API does not change and no SemVer bump is needed. The only calls areBase.reduce(hcat, ::AbstractVector)andreshape, which are public Base, so no dependency internals are used. No existing test is weakened. One test file is added,test/ensemblegpuarray_hcat.jl, wired intoGROUP=CPU. It checks that the GPU-array and serial results agree, that abatch_size = 1batch works, and that allocation grows by less than 8× when trajectories grow 4×. The expected ratio is about 4 and master measures 13.3, so the margin against flakiness is reasonable.Evidence: On head
3d7256a, 3 jobs fail (plus the Buildkite rollup), and each fails the same way on themastercommitsb3d6a17and4b59cb4, so all are pre-existing:CUDA Tests (Julia 1)fails withInvalidIRErrorintest/enzyme/cuda_records.jl:9("Allocate records"), so the "Enzyme ensemble gradients" testset errors. The master run shows the same error, and the non-Enzyme CUDA testsets passed on head.buildkite/.../julia-oneapi-run-tests-on-julia-v1and-v1-dot-10both error at "Initialization preprocessing runs once" intest/gpu_kernel_de/conversions.jl:82(15 passed, 1 errored). Master build 1714 has the same error.buildkite/diffeqgpu-dot-jlis only the rollup of those two oneAPI jobs.Everything else passes: CPU (1, lts, pre, x86), Downgrade CPU, JLArrays, OpenCL, Enzyme, QA, Runic, typos, Documentation, benchmark, and AMDGPU and Metal on both 1 and 1.10. In the CPU Julia 1 log the new testset ran and passed:
EnsembleGPUArray host concatenation | 6 6. No regressions.Independent review: Devin CLI (Mac) / fusion-claude-opus-5-5-high-sidekick-swe-2-medium rated it low, high confidence: the change is 13 lines, a like-for-like swap of the Θ(n²)
reduce(hcat, generator)fold for a single-allocationreduce(hcat, ::Vector)/reshapethat gives the same results, and every CI failure matches master.Merge: auto-merge candidate. There are no test regressions and no code findings. Three things for whoever merges:
🤖 Risk assessment posted by an AI agent (fleet master) — harness: Devin CLI (Mac) / fusion-claude-opus-5-5-high-sidekick-swe-2-medium; dispatched by Claude Code head, model claude-opus-5-5[1m]
Conversation: local Claude Code session 3cd6500a-1f81-46b5-ac0b-c466e15b6a53 on Chris's Mac (session ID, no URL)