Skip to content

Grouping-apply at 50k regressed from ~400ms to ~5-7s (#321); target-scale group benches report partial #500

Description

@blove

Applying row grouping to 50k rows (S2 scale=target, group by col_5, 4 groups) takes ~5–7 seconds from setQuery to the first painted [data-pretable-group-row] on current main (a29298a). Before #321 it settled in ~400 ms on the same machine in the same session. The result: the group-expand and group bench scripts can no longer complete at target scale — waitForGroupedRowModel's 120-frame budget expires and the run reports partial (and group's measured window is cut off by the #326 row-count honesty check: "result row count settled at 50000, not the 50004 rows the plan handed the surface").

This was found while verifying #483 in a real browser. #483 is behaving exactly as designed — it refuses to measure and reports partial instead of emitting a number contaminated by the grouping render. The pre-#483 model-only gate fails at target too, so this is not the paint gate over-correcting.

Bisect

git bisect run over 33fd60f..5f59f86, driving the real measurement (bench:e2e, external server, S2/target/group-expand, judged on the summary's status):

Scale dependence (current main)

scale rows status interaction latency
dev 750 completed 8.3 / 8.5 / 7.5 ms
hypothesis 3,000 completed 16.2 / 8.2 / 8.3 ms
target 50,000 partial — (first group row paints ~5.5 s after the grouping is applied)

Dev-scale latency matches the committed Aug-10 baseline median (8.4 ms), so the toggle itself hasn't drifted — what regressed is the grouping-apply settle, which #321's cooperative slicing appears to stretch from ~0.4 s to seconds at 50k. Not load: 1-minute load average during these runs was ~30–60, far below the 118–165 the Aug-11 numbers were recorded under.

Impact

Repro (from a repo checkout, port isolated from other worktrees):

pnpm --filter @pretable/app-bench build
pnpm --filter @pretable/app-bench exec vite preview --host 127.0.0.1 --port 4519 &
PRETABLE_BENCH_EXTERNAL_SERVER=1 PRETABLE_BENCH_BASE_URL=http://127.0.0.1:4519 \
PRETABLE_BENCH_ADAPTER=pretable PRETABLE_BENCH_SCENARIO=S2 \
PRETABLE_BENCH_SCALE=target PRETABLE_BENCH_SCRIPT=group-expand pnpm bench:e2e

Activity

  1. blove commented on Aug 29, 2026

    @blove
    ContributorAuthor

    Root-caused (independent bisect landed on the same first-bad commit, 72e7d47 / #321). Mechanism, measured in headed Chromium with instrumentation:

    • The cooperative grouping candidate charges one seal unit per (row × aggregated column × {all,filtered} aggregate root) — group-index.ts sealStep/sealActiveAggregate (~1740–1860), counted into totalRows at cooperative-transition.ts:662-664. S2 target = 50k rows × 10 avg columns × 2 roots = 1,050,008 units.
    • Slices run at DEFAULT_BUDGET_MS = 0.25 / 256-unit cap with a now() call per unit (runCooperativeTransitionSlice, ~line 280). Measured: 9,540 slices, Σ 2.60s, p50 slice 0.30ms — real sliced work (~6.5× the pre-feat: complete incremental row-model migration #321 synchronous build), not scheduler starvation. Nothing hangs; the bench's 96-frame budget just expires.
    • The engine-only cost at node level is ~647ms for 50k — the overhead is unit granularity + per-unit clock reads, not the data structures.
    • Separate second course: after the engine commits, renderer-dom runs three cooperative height-index replacements over 50,004 rows (~0.6s) before READY.

    Also, for the record: grouped streaming is fine on current main (group-updates 20k p95 10.0ms, overruns ~1 — the #487 structures fixed the 2026-08-10 baseline's 34.7ms/59). This issue is apply-latency only.

    Fix in progress on blove/grouping-apply-cooperative-cost: coarsen the seal unit to per-row (all aggregated columns + both roots per unit) and amortize the clock check, keeping the budget semantics — targeting apply back to the old ballpark while leaving streaming slice sizes (and its 10.0ms p95) untouched.

  2. added a commit that references this issue on Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions