Skip to content

Refactor CPU and GPU cache for faster memory allocation - #253

Merged
huiyuxie merged 29 commits into
mainfrom
opt/cache-refactoring
Nov 16, 2026
Merged

huiyuxie merged 29 commits into
mainfrom
opt/cache-refactoring

Conversation

@huiyuxie

@huiyuxie huiyuxie commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Closes #239.

Replaces the separate CPU and GPU caches with one GPU cache, semi.cache, and removes redundant allocations and copies.

  • One cache: removes cache_cpu, initial_cache_cpu and initial_cache, and renames cache_gpu to cache. The RHS functions now take cache like Trixi.jl.
  • HG indicator: smoothing runs on the GPU with preallocated alpha buffers, so the indicator no longer needs the CPU cache.
  • Containers: fills the preallocated GPU arrays with copyto! instead of allocating them twice, makes the structs immutable, keeps cell_ids on the host, and fixes the element type of the mortar arrays.
  • Semidiscretization: marks the fields that are never reassigned as const.
  • Solution layout: the initial state is allocated directly on the GPU in the DG layout, so the RHS no longer wraps its arrays. Removes the unused GPU wrap_array_native.
  • Benchmarks and examples: the RHS benchmark works with both compared revisions, and the 2D shallow-water example now applies its element-dependent initial condition on the GPU.

huiyuxie and others added 10 commits November 13, 2026 15:50
The host-side `apply_smoothing!` from Trixi.jl reads the interface and mortar
connectivity element by element, which only works with the CPU cache. Move the
smoothing to the GPU, where `neighbor_ids` already lives:

- The host indicator now only computes the raw values.
- `calc_indicator_gpu!` copies them into a preallocated device buffer and runs
  the smoothing kernels, so the HG volume integral no longer allocates `alpha`
  on every RHS call.
- The kernels use an atomic `max` and read only from the frozen `alpha_tmp`,
  so the result matches Trixi.jl's serial loop independent of thread order.

The HG volume integral now passes `cache_gpu` to the indicator, so it no longer
needs the CPU cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
…yout

The benchmark scripts of the target revision also measure the baseline
revision, so they must work whether or not the semidiscretization still has a
CPU cache:

- Time the `semi`-level `rhs_hyperbolic!`, whose signature is the same in both
  revisions, instead of unpacking the caches.
- Build the host element data for the initial input with Trixi directly. The
  node coordinates are the same as before, so the input hashes do not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
…ample

The example modified a host copy of the initial state and never wrote it back,
so `ode.u0` on the GPU kept the default initial condition. Copy the modified
state back to `ode.u0`, and read the node coordinates from the GPU cache instead
of the CPU cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
Nothing reads the CPU cache anymore: the HG indicator now works on the GPU
cache, and `digest_boundary_conditions` does not read the cache for tree
meshes. Drop it completely:

- Remove the `cache_cpu` field, the `CacheCPU` type parameter, and the
  `initial_cache_cpu` keyword from `SemidiscretizationHyperbolicGPU`, and stop
  building Trixi's full CPU cache in the constructor.
- Pass `cache_gpu` to `digest_boundary_conditions`.
- Drop the `cache_cpu` argument from `calc_volume_integral!` and
  `rhs_hyperbolic!` and update their docstrings.
- Update the tests and profiling scripts accordingly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
With the CPU cache gone there is only one cache, so follow Trixi.jl's naming:

- `semi.cache_gpu` becomes `semi.cache`, and the `initial_cache_gpu` keyword
  becomes `initial_cache`.
- The cache argument of `calc_volume_integral!`, `rhs_hyperbolic!`, and the
  other solver functions is now called `cache`, matching Trixi.jl's signatures.

Tests and profiling scripts keep their local `cache_gpu` variable, since `cache`
there already refers to Trixi.jl's CPU semidiscretization.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
@huiyuxie huiyuxie self-assigned this Nov 15, 2026
@huiyuxie huiyuxie added performance Improve performance refactoring Refactoring labels Nov 15, 2026
@huiyuxie huiyuxie changed the title Refactor CPU and GPU cache to one cache for easy memory allocation Refactor CPU and GPU cache to one cache for faster memory allocation Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

GPU RHS Benchmarks

07b24b5 vs merge base 4447f0e; 26 cases, minimum synchronized wall time.
Geometric mean speedup: 0.983x.

Largest time changes Time change Speedup
euler_ec_3d +33.02% 0.752x
euler_shock_1d +19.48% 0.837x
hypdiff_nonperiodic_1d +1.48% 0.985x

Full benchmark report
Results are informational only; differences may be measurement noise.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@huiyuxie

Copy link
Copy Markdown
Member Author

It would be better if benchmark cache initialization before / after could be given before merge.

huiyuxie and others added 11 commits November 16, 2026 10:39
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Remove the empty line at the start of the function body, which JuliaFormatter
flags in the style check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
The RHS no longer wraps its arrays, since TrixiCUDA keeps the solution in the
DG layout. Clean up what is no longer needed:

- Allocate the initial state directly on the GPU in the DG layout instead of
  allocating a flat array, wrapping it, and copying it again with `CuArray`.
- Drop the wrapping in `calc_error_norms`, which receives the integrator state
  in the DG layout.
- Remove the GPU `wrap_array_native` method. Trixi.jl's save callbacks copy GPU
  states to the host first, so only the `Array` method is reached.
- Remove the commented-out FDSBP `wrap_array` stub and outdated comments, and
  explain why the GPU `wrap_array` method is kept (Trixi.jl's step-size and
  analysis callbacks call it).
- Update the `rhs_hyperbolic!` docstring to the `du, u` names and layout.
- Pass `u` and `du` to the RHS in the benchmark instead of flat views, which the
  RHS no longer reshapes. Both compared revisions accept the DG layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The documentation cleanup in 211ce57 also removed the methods of
`calc_volume_integral!`, `calc_interface_flux!`, `calc_boundary_flux!`, and
`calc_mortar_flux!` that select `have_nonconservative_terms(equations)`
themselves. `rhs_hyperbolic!` and the tests still call these methods, so every
RHS call failed with a `MethodError`, starting at the volume integral.

Restore the four methods without their docstrings, so the documentation
cleanup stays as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhLr3TjfMCfrH8o5zgUDc1
Comment thread benchmark/cases.jl Resolved
Comment thread benchmark/cases.jl Resolved
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@huiyuxie

huiyuxie commented Nov 16, 2026 •

Copy link
Copy Markdown
Member Author

The benchmark results for before / after this PR.

We mainly benchmarked the CPU and GPU memory allocation and runtime for three processes: 1. SemidiscretizationHyperbolicGPU where the cache allocation happens, and we refactor the cache structure here (this process is marked as semi in the following report); and 2. semidiscretizeGPU where u, an very crucial variable is initialized here and we have done an small optimization here (this process is marked as semidiscretize in the following report); and 3. almost everything happens before we call rhs_hyperbolic! (this process is marked as total in the following report).

For simplicity, we only did this benchmark on basic linear advection cases (since the memory allocation is obviously improved in theory).

advection_basic_1d / semi
==========================================================================================
BASELINE
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  347.978 μs … 437.333 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     394.391 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   392.140 μs ±  20.004 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

             ▃ ▃    ▃            ▃▃ ▃    ▃▃ █                    
  ▇▁▁▁▁▁▁▁▁▁▁█▁█▁▇▇▁█▁▇▁▇▁▇▁▇▁▇▇▁██▁█▁▇▁▁██▁█▇▁▁▇▁▁▁▁▁▁▁▇▁▁▁▁▁▇ ▁
  348 μs           Histogram: frequency by time          437 μs <

 Memory estimate: 328.19 KiB, allocs estimate: 1136.

TARGET
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  209.232 μs … 242.864 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     225.306 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   225.451 μs ±   7.512 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▁▁       ▁  ▁    ▁█ ▁▁▁▁ █▁▁███  █▁ ██  ▁   ▁▁  ▁          ▁▁  
  ██▁▁▁▁▁▁▁█▁▁█▁▁▁▁██▁████▁██████▁▁██▁██▁▁█▁▁▁██▁▁█▁▁▁▁▁▁▁▁▁▁██ ▁
  209 μs           Histogram: frequency by time          243 μs <

 Memory estimate: 164.38 KiB, allocs estimate: 239.

Judgement (median): TrialJudgement(-42.87% => improvement)

==========================================================================================
advection_basic_1d / semidiscretize
==========================================================================================
BASELINE
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  74.075 μs … 107.487 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     95.518 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   94.771 μs ±   6.121 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                   ▄ ▁▁▁█▄ ▁   ▁                
  ▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▁▁▁▁▆▆▁▁▁▆▁▁▆▁▁▁█▆█████▆█▁▁▆█▁▆▆▆▁▁▁▁▆▁▁▁▁▆ ▁
  74.1 μs         Histogram: frequency by time          107 μs <

 Memory estimate: 2.09 KiB, allocs estimate: 53.

TARGET
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  51.125 μs … 72.146 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     63.875 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   63.496 μs ±  5.026 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                ▄ ▄  ██   ▁                 ▁  
  ▆▁▆▁▁▁▁▁▁▆▁▁▆▁▆▁▁▁▁▁▁▁▁▁▁▁▁▆▁▁█▆█▆▁██▆▁▆█▁▆▁▁▁▁▆▆▆▁▁▆▁▁▆▁▁█ ▁
  51.1 μs         Histogram: frequency by time        72.1 μs <

 Memory estimate: 1.34 KiB, allocs estimate: 32.

Judgement (median): TrialJudgement(-33.13% => improvement)

==========================================================================================
advection_basic_1d / total
==========================================================================================
BASELINE
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  614.690 μs … 927.261 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     651.832 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   661.732 μs ±  50.210 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

     ▁ ▃█▃▁ ▃                                                    
  ▄▄▄█▁████▇█▁▄▄▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄ ▁
  615 μs           Histogram: frequency by time          927 μs <

 Memory estimate: 486.41 KiB, allocs estimate: 2093.

TARGET
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  499.627 μs … 742.791 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     564.154 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   566.082 μs ±  49.213 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

   ▁     █  ▁    ▄▁ ▁  ▁ ▄                                       
  ▆█▆▁▆▆▆█▁▁█▆▁▁▁██▆█▆▁█▁█▁▆▆▆▁▁▁▁▁▁▆▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ ▁
  500 μs           Histogram: frequency by time          743 μs <

 Memory estimate: 321.85 KiB, allocs estimate: 1175.

Judgement (median): TrialJudgement(-13.45% => improvement)

==========================================================================================
advection_basic_2d / semi
==========================================================================================
BASELINE
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  1.495 ms …  1.696 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.585 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.587 ms ± 59.516 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

      █                                     ▃                 
  ▇▁▇▁█▁▇▁▁▇▇▁▇▇▇▁▇▁▇▇▁▁▁▇▇▁▇▇▇▁▁▁▇▇▁▁▇▁▇▇▁▇█▇▁▁▇▇▁▁▁▁▁▇▁▇▁▇ ▁
  1.5 ms         Histogram: frequency by time         1.7 ms <

 Memory estimate: 4.78 MiB, allocs estimate: 1381.

TARGET
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  938.991 μs …  1.086 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):       1.040 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):     1.033 ms ± 36.077 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                      ▃  ▃ ▃█   ▃        ▃      
  ▇▁▁▁▁▁▁▁▇▁▇▁▁▁▁▁▁▁▇▇▁▁▇▁▁▁▇▁▇▇▁▁▁▁▁▇█▇▁█▇██▁▇▁█▇▇▁▁▇▇▁▇█▁▇▁▇ ▁
  939 μs          Histogram: frequency by time         1.09 ms <

 Memory estimate: 2.42 MiB, allocs estimate: 403.

Judgement (median): TrialJudgement(-34.39% => improvement)

==========================================================================================
advection_basic_2d / semidiscretize
==========================================================================================
BASELINE
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  123.366 μs … 188.330 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     141.817 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   143.669 μs ±  11.097 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                 ▂█▂     ▂                                       
  ▅▅▅▁▁▁▁▁▁▁▁▅█▁▅███▅▅█▅██▁█▁▁▁▅▁▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅ ▁
  123 μs           Histogram: frequency by time          188 μs <

 Memory estimate: 2.17 KiB, allocs estimate: 53.

TARGET
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):   94.713 μs … 117.797 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     107.477 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   107.381 μs ±   6.483 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▄                             ▄  █                        ▁    
  █▁▁▁▆▁▁▁▁▁▆▁▆▁▁▁▁▁▆▁▁▁▁▆▁▁▁▁▆▁█▆▁█▁▁▆▆▆▆▆▆▆▆▆▁▁▆▆▁▆▁▁▁▁▁▁▁█▆▆ ▁
  94.7 μs          Histogram: frequency by time          118 μs <

 Memory estimate: 1.42 KiB, allocs estimate: 32.

Judgement (median): TrialJudgement(-24.21% => improvement)

==========================================================================================
advection_basic_2d / total
==========================================================================================
BASELINE
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  2.075 ms …  2.337 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     2.124 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.149 ms ± 65.082 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

      ▄▁ ▁█   ▄        ▁                                      
  ▆▆▆▆██▆██▆▁▁█▁▆▁▁▆▁▆▁█▆▁▆▁▁▁▁▁▁▆▁▆▆▁▁▁▆▁▆▁▁▁▁▁▁▆▁▁▁▁▁▁▁▁▁▆ ▁
  2.07 ms        Histogram: frequency by time        2.34 ms <

 Memory estimate: 5.35 MiB, allocs estimate: 2340.

TARGET
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  1.568 ms …  1.798 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.629 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.628 ms ± 43.109 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

    ▁  ▁  ▁  █   ▄ █ ▁▁                                       
  ▆▆█▆▁█▁▁█▆▆█▁▁▆█▁█▆██▆▁▁▁▁▁▆▆▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ ▁
  1.57 ms        Histogram: frequency by time         1.8 ms <

 Memory estimate: 2.99 MiB, allocs estimate: 1341.

Judgement (median): TrialJudgement(-23.28% => improvement)

==========================================================================================
advection_basic_3d / semi
==========================================================================================
BASELINE
BenchmarkTools.Trial: 32 samples with 1 evaluation per sample.
 Range (min … max):  11.400 ms …  12.221 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     11.633 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   11.621 ms ± 163.179 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

    ▃▃▃        ▃▃ ▃██▃     █                                    
  ▇▇███▁▁▇▁▁▇▁▁██▁████▇▇▇▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇ ▁
  11.4 ms         Histogram: frequency by time         12.2 ms <

 Memory estimate: 25.03 MiB, allocs estimate: 1586.

TARGET
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  3.673 ms …  3.986 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     3.865 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   3.839 ms ± 79.359 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▃                               █  ▃▃█  ▃▃ ▃                
  █▁▇▁▁▁▇▇▁▁▇▁▁▇▁▁▁▁▁▇▁▁▁▁▁▁▁▁▁▇▇▇█▇▇███▁▇██▁█▇▁▇▁▁▁▁▁▁▁▁▁▁▇ ▁
  3.67 ms        Histogram: frequency by time        3.99 ms <

 Memory estimate: 12.54 MiB, allocs estimate: 536.

Judgement (median): TrialJudgement(-66.78% => improvement)

==========================================================================================
advection_basic_3d / semidiscretize
==========================================================================================
BASELINE
BenchmarkTools.Trial: 34 samples with 1 evaluation per sample.
 Range (min … max):  182.778 μs … 213.572 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     201.107 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   199.975 μs ±   5.641 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                   ▁  █   ▁                      
  ▄▁▁▁▁▄▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▄▄▁▁▁▁▄▄▄▄▄█▄▇█▄▇▇█▄▁▁▁▁▄▁▁▄▁▁▁▁▁▁▁▁▁▁▄ ▁
  183 μs           Histogram: frequency by time          214 μs <

 Memory estimate: 2.28 KiB, allocs estimate: 53.

TARGET
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  160.060 μs … 306.701 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     241.114 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   230.345 μs ±  56.954 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

     ▅  █                                               ▂        
  ▅▁▅█▅▁█▁█▁▅▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅▅▁▅▁▁▁▁▁▁▁▁▁▁▁█▁▁█▅██▁▁▁█▅▅ ▁
  160 μs           Histogram: frequency by time          307 μs <

 Memory estimate: 1.44 KiB, allocs estimate: 29.

Judgement (median): TrialJudgement(+19.89% => regression)

==========================================================================================
advection_basic_3d / total
==========================================================================================
BASELINE
BenchmarkTools.Trial: 32 samples with 1 evaluation per sample.
 Range (min … max):  11.982 ms …  13.116 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     12.345 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   12.355 ms ± 234.281 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

      ▁  ▁▁         ▁   ▄ ▄   █                                 
  ▆▁▁▁█▆▁██▆▁▆▆▁▁▆▆▆█▆▁▁█▁█▁▁▆█▁▆▁▆▁▁▁▁▁▁▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ ▁
  12 ms           Histogram: frequency by time         13.1 ms <

 Memory estimate: 25.77 MiB, allocs estimate: 2558.

TARGET
BenchmarkTools.Trial: 33 samples with 1 evaluation per sample.
 Range (min … max):  4.479 ms …   5.565 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     4.658 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   4.695 ms ± 197.808 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▃ ▃▃█▃  ▃█     ██ █  ▃                                       
  █▇████▇▁██▇▁▁▁▇██▇█▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇ ▁
  4.48 ms         Histogram: frequency by time        5.56 ms <

 Memory estimate: 13.28 MiB, allocs estimate: 1487.

Judgement (median): TrialJudgement(-62.27% => improvement)

==========================================================================================
SUMMARY (median time; CPU memory from BenchmarkTools; GPU memory from CUDA.@allocated)
==========================================================================================
case                  stage              time base    time new   ratio    CPU base     CPU new    GPU base     GPU new
advection_basic_1d    semi              394.391 μs  225.306 μs    0.57  328.19 KiB  164.38 KiB 208.031 KiB  96.016 KiB
advection_basic_1d    semidiscretize     95.518 μs   63.875 μs    0.67    2.09 KiB    1.34 KiB  64.000 KiB  32.000 KiB
advection_basic_1d    total             651.832 μs  564.154 μs    0.87  486.41 KiB  321.85 KiB 273.062 KiB 129.047 KiB
advection_basic_2d    semi                1.585 ms    1.040 ms    0.66    4.78 MiB    2.42 MiB   4.500 MiB   2.219 MiB
advection_basic_2d    semidiscretize    141.817 μs  107.477 μs    0.76    2.17 KiB    1.42 KiB1024.000 KiB 512.000 KiB
advection_basic_2d    total               2.124 ms    1.629 ms    0.77    5.35 MiB    2.99 MiB   5.501 MiB   2.720 MiB
advection_basic_3d    semi               11.633 ms    3.865 ms    0.33   25.03 MiB   12.54 MiB  24.688 MiB  12.313 MiB
advection_basic_3d    semidiscretize    201.107 μs  241.114 μs     1.2    2.28 KiB    1.44 KiB   4.000 MiB   2.000 MiB
advection_basic_3d    total              12.345 ms    4.658 ms    0.38   25.77 MiB   13.28 MiB  28.689 MiB  14.314 MiB

ratio < 1 means the target is faster.

@huiyuxie

huiyuxie commented Nov 16, 2026 •

Copy link
Copy Markdown
Member Author

The benchmark check in CI failed because the new benchmark suite does not match the original source code, and it is normal. Since we have already given out the benchmark results below, we could safely merge this PR by passing [skip benchmarks].

@huiyuxie huiyuxie changed the title Refactor CPU and GPU cache to one cache for faster memory allocation Refactor CPU and GPU cache for faster memory allocation Nov 16, 2026
@huiyuxie
huiyuxie merged commit 5710818 into main Nov 16, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Improve performance refactoring Refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unify CPU and GPU caches

2 participants