bench(results): re-measure the in-process and redis legs on mains, add batch ceilings to the README

[zannis]

Re-measures the two published legs that predated harness fixes, regenerates every chart, and adds the batch consume ceilings to the README. Two commits, one per concern.

Why these two legs

• in-process was measured before the 32 GiB drain byte cap. Its 64 KiB drain corpus was 49 152, so all eleven 64 KiB drain rows were setup_bound and the charts withheld every one: the fastest backend in the set published no 64 KiB consume rate. benches/README.md already described the 32 GiB cap as if the published leg carried it.
• redis was measured 19 hours before the 1 s acked-entry trim cadence (feat(redis): make the acked-entry trim cadence configurable #192) and before the direct consumer honoured concurrent_processing (fix(redis): honour concurrent_processing on a direct consumer's run #187). Its nine failures were all 64 KiB offered-load rungs refused with XADD ... OOM, and its consume_parallel drains ran one message at a time.

Kafka is deliberately not re-measured: its seven withheld 64 KiB rows sit at corpus 49 152 under the containerised 3 GiB cap, so a rerun reproduces them. RabbitMQ, NATS and SQS are untouched; their rows are byte-identical apart from last-digit float re-serialisation by the merge (worst relative difference 2e-16, no key or value changed).

Provenance

Both passes ran on 2026-09-11 on the document's own host and toolchain (Apple M4 Max 16c / 64 GB, macOS 26.6.2, rustc 1.91.1, shove 0.14.0), through scripts/bench.sh <backend> with no flags, one at a time, on mains with Low Power Mode off (the script's own gate). Logs: target/bench-logs/inmemory-20260911T072238Z.log and target/bench-logs/redis-20260911T075353Z.log.

What changed in the document

leg	rows	failures	64 KiB drain rows published	64 KiB offered-load rows
in-process	167 → 167	1 → 0	0 of 11 → 12 of 12	unchanged
redis	143 → 156	9 → 0	12 → 11	15 → 24

in-process. 64 KiB drain corpus 49 152 → 524 288, windows 3.4 to 9.1 s, all twelve rows framework. The 1 KiB leg moves from the byte-bound 3 145 728 to the pinned 6 000 000. The "consumed before assembly" failure (consume_batch 64 KiB 8c, 37 000 of 49 152) is gone: 30 500 of 524 288 is well under half.

redis. No OOM anywhere in the log. consume_parallel drains rise 10 to 25x at 64 B and 1 KiB (for example 11.7k → 290k msg/s at one consumer) and now sit level with consumer_group on the same primitive, which is the #187 fix. Every other cell, the six publish cells included, rises a near-uniform 26 to 45 percent, and the 64 B fill drops from 89 s to 64 s. A fill is the runbook's pure host-speed probe and that factor is the Low Power Mode signature it records; the previous leg was measured on the evening of 2026-09-09, before the power gate existed. The consume_batch 64 KiB 8c drain is newly setup_bound: a 0.9 s window at 43.7k msg/s over the byte-capped 49 152 corpus, which a rerun cannot change.

Reading a run, per the runbook checklist

• Failed scenarios: absent from both logs. failures[] for both legs is empty; the document carries 3 failures (1 NATS, 1 RabbitMQ, 1 in-process → 0) down from 12.
• unique_at_start under half the corpus on every drain row of both legs (worst in-process 30 500 / 524 288, worst redis 4 000 / 49 152). duplicates 0 on every drain row.
• One sitting on both passes: the twelve 64 B fills in execution order are 9.8 to 10.7 s (in-process) and 63.8 to 68.9 s (redis), flat.
• setup_bound rows remaining: in-process 24 (broadcast, FIFO, DLQ drain, five 64 KiB rungs, one 64 B batch drain), redis 17 (broadcast, FIFO, DLQ drain, the 64 KiB batch 8c drain above). Same classes as before.

README

The ## Performance section gains one paragraph and one table: the batch flow's best published drain cell per backend and payload, its consumer count, and the ratio to the parallel consumer at that same cell, every figure a row in this document. Ratios are stated to one decimal of the measured cell, and the "at most 1.35x at 64 KiB" claim covers the largest measured ratio (1.341, RabbitMQ 4c).

Not in this PR

docs/pages/ops/performance.mdx still quotes the in-process 64 KiB figures of the previous leg (the 356k parallel row and the 3-10x / 2-7x batch ratios), and benches/README.md's 32 GiB deviation section describes the cap in terms this leg now makes true. Both need re-deriving from this document in a follow-up; nothing in them is made more wrong by this change, and the previous prose fixes (#196, #197) already scoped those sentences.

Verification

• scripts/bench.sh charts: every SVG under docs/public/bench/ regenerated, then cargo nextest run --no-default-features -E 'binary(chartgen)': 131 run, 131 passed, including the_committed_results_document_renders_every_family.
• git diff on the other four legs: no key or value change beyond float re-serialisation.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	shove-docs	efeaf43	Commit Preview URL Branch Preview URL	Sep 11 2026, 11:45 AM

[zannis]

Reviewed against the document: the table reproduces, one prose bound does not

I recomputed all eighteen cells of the README table from benches/results/bench-results.json on efeaf43 — best published (framework, method: drain) consume_batch cell per backend and payload, its consumer count, and the ratio to the same-cell comparator. Every cell matches, including SQS. No objection to the table.

The 1.35x sentence is falsified by a published row

at 64 KiB the batch flow is at most 1.35x the parallel one on every backend

RabbitMQ, 64 KiB, one consumer: consume_batch 27 818 msg/s over consume_parallel 18 997 msg/s = 1.464x. Both rows are framework, both are drains, both publish.

The 1.3x in the table is the ratio at the best-throughput cell (4c, 1.341) — which is what the table says it is and is correct. The prose bound then reads that column as if it were a maximum, and it isn't. Per-cell 64 KiB ratios, published cells only:

backend	1c	2c	4c	8c	max
in-process	0.542	0.689	0.891	1.135	1.135
Kafka	—	1.030	—	—	1.030
NATS	1.109	1.131	1.121	1.186	1.186
RabbitMQ	1.464	1.260	1.341	—	1.464
Redis	1.169	0.755	1.013	—	1.169
SQS	1.152	1.054	1.030	1.147	1.152

Rounded outward, the true bound is at most 1.5x. Suggested edit: at 64 KiB the batch flow is at most 1.5x the parallel one on every backend. (Dashes are cells where one side of the pair is setup_bound or absent, so the ratio isn't published.)

Two smaller things

• SQS's comparator isn't consume_parallel. SQS has no such rows — its flows are consume_batch, consume_fifo, dlq_drain, publish_* and supervisor. The 1.1x / 2.5x / 1.1x figures are batch over supervisor, which is the right comparator (ConsumerSupervisor is SQS's parallel-consume analogue) but not "the plain parallel consumer". One clause would fix it.
• Failure count. The body says the document carries 3 failures; it carries 2 — NATS broadcast 64 KiB 8c and RabbitMQ consume_parallel 64 KiB 8c. The in-process one is gone, which I think is what the arrow meant.

What I confirmed independently

• In-process: 167 rows, failures[] empty, all twelve 64 KiB drains framework at corpus 524 288, windows 3.39-9.09 s. The batch 64 KiB 8c cell assembles at 30 500 of 524 288.
• Redis: 156 rows, failures[] empty, no XADD ... OOM anywhere.
• The other four legs: no key or value changed; the only diff is float re-serialisation, as you said.
• The in-process fills support the corpus explanation rather than a host-speed one: 64 B fills are 9.7-13.9 s before and 9.8-10.7 s after, so that leg's host state is comparable and the 64 KiB rate drop is the larger corpus. (Your Low Power Mode reading of the redis leg is a separate argument and I'm not disputing it.)

The prose re-derivation you left me is in #200, which stacks on this branch. It supersedes #197 — I couldn't rebase #197 in place because the runner blocks force-push, so #200 carries its two commits unchanged and #197 should be closed rather than merged.