Skip to content

EnsembleGPUKernel solve(): reduce host-side overhead (~0.8 s at 2^20 trajectories) #557

Closed

Description

@ChrisRackauckas-Claude

At N = 2^20 trajectories, solve(ens, GPUTsit5(), EnsembleGPUKernel(backend, 0.0); ...) spends about 0.8 s on the host, while the Lorenz kernel itself takes milliseconds. This was measured on the KernelAbstractions CPU backend with 16 threads and safetycopy=false, on DiffEqGPU 3.21.2:

  • Problem preparation (_prepare_kernel_problems, src/solve.jl:207-221 at v3.21.2) takes 0.56–0.59 s and allocates 487 MB. Only about 0.07 s of that is prob_func/remake; the rest is per-element adapt / _kernel_record.
  • Building 2^20 ODESolutions (src/solve.jl:280-305) takes 0.27–0.45 s.
  • With the default safetycopy=true, the per-trajectory deepcopy pushes preparation to 1.5–2.3 s.

This dominates host-to-host timings in SciML/SciMLBenchmarks.jl#1847. The low-level vectorized_asolve avoids it but isn't what most users call.

Goal: bring host overhead for the EnsembleGPUKernel path down to a small multiple of the kernel time without changing the public API or the results.

  • Build the kernel problem records in bulk (one allocation, no per-element adapt).
  • Construct the returned EnsembleSolution cheaply, e.g. lazily backed by the (ts, us) device/host arrays, as long as sol.u[i].u / sol.u[i].t indexing and existing tests keep working.
  • Avoid the safetycopy deepcopy when the problem type makes it unnecessary, if that can be done safely.

Done means before/after host-phase timings at N = 2^14…2^20 on the CPU backend, identical endpoint results, and the full GROUP=CPU suite passing.

🤖 Filed by an AI agent: Claude Code (model: claude-opus-5-5[1m]).

https://claude.ai/code/session_01SJfy3ez7u4CqvAerKrT5np

Activity

  1. ChrisRackauckas-Claude commented on Sep 27, 2026

    @ChrisRackauckas-Claude
    MemberAuthor

    Taking this — Codex CLI 0.157.1, model gpt-6-sol (local job 557-codex on amdci2.julia.csail.mit.edu).

  2. ChrisRackauckas-Claude commented on Sep 28, 2026

    @ChrisRackauckas-Claude
    MemberAuthor

    Taking this — Cursor Agent CLI 2026.09.26-dd393fe, model auto (local job 557-cursor on amdci2.julia.csail.mit.edu).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions