Skip to content

Phase B: Dimension-sweep benchmark - Apple GPU vs CPU crossover #77

Closed

Description

@neuromechanist

Context

Measurement phase of the epic. MPS lost on 32-channel data (#63: 0.51x) because of
fixed per-op Metal dispatch overhead, which amortizes as tensors grow - 32 channels
was the wrong regime to judge MPS. This phase finds the crossover where Apple GPU actually
wins. Runs last so it can benchmark the Phase A float32 path (and Phase C MLX if landed).

Approach

  • Extend benchmarks/benchmark_gpu.py with a dimension sweep: channel count
    (32/64/128/256), block_size, and n_models.
  • Compare MPS-float32 (and MLX if Phase C landed) vs CPU float64/float32 - warmed, min of
    repeats, CUDA-synchronized.
  • Identify and document the crossover point(s).

Data note (timing, not correctness)

Real sample EEG is 32 channels; higher channel counts are produced by tiling/replicating
real channels purely to size the tensors. This is a throughput benchmark, not a
correctness test
- the NO-MOCKS rule governs correctness tests, and no correctness claim
is made from tiled data.

Acceptance

  • Crossover curve documented in .context/issue-63/perf_findings.md (or a new perf note).
  • benchmark_gpu.py runs the sweep behind a single CLI flag.

Dependencies

Blocked by Phase A (needs stable float32); benefits from Phase C (compare MLX).
Research: .context/mps_pathways.md Pathway B.

Activity

  1. neuromechanist commented on Jul 8, 2026

    @neuromechanist
    MemberAuthor

    Done via PR #80 (squash-merged into the epic branch, epic #74 Phase B). Cross-platform benchmark on real 70-ch EEG: MLX is the Apple-GPU win (~15-25 ms/it, ~7x over torch-CPU, faster than an RTX 4090 at EEG scale); PyTorch-MPS never wins (>= CPU); results agree across cpu/mps/cuda/mlx to ~3 digits. Multi-model has no GPU path yet -> follow-up #81. See .context/issue-77/benchmark_findings.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or enhancement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions