docs(bench): describe the in-process leg that is published, not the one the cap would produce

[zannis]

Depends-on: none - prose only; every PR it re-derives from is already merged on main.

Closes the prose half of the six-backend benchmark rerun. Prose only — no results-document, chart or code change, so the committed SVGs still byte-match.

Why now

The rerun itself is finished: all six legs were re-measured on the M4 Max and merged in #175–#195, and the last outstanding recovery (a fresh in-process leg under the 32 GiB cap, and a fresh Redis leg post-#192) has been called off — no further passes for now. That makes the published document final as it stands, and the issue's last acceptance criterion applies to it: no claim the document cannot support. Three claims did not survive that check.

What the document holds (origin/main @b8336f7)

	inmemory	kafka	nats	rabbitmq	redis	sqs
rows	167	156	155	154	143	60
failures[]	1	0	1	1	9	0
representative	true	true	true	true	true	false
consume_batch drain, 64 B / 1 KiB / 64 KiB	4/4/4	4/4/4	4/4/4	4/4/4	4/4/4	4/4/4

One provenance block: Apple M4 Max (16c / 64 GB), macOS 26.6.2, shove 0.14.0, rustc 1.91.1 (ed61e7d7e 2025-11-07). sqs is the only non-representative run (LocalStack) and the only run with entries in unsupported[], both capability gates (HasCoordinatedGroups, HasBroadcast) with reasons recorded.

The three corrections

1. The 32 GiB in-process byte cap is not in the published leg. #179 gave in-process its own drain byte cap; the committed in-process leg was measured before it and a leg changes only by being re-measured. The runbook described the cap's effects in the present tense ("the leg publishes instead of being withheld whole"; "32 GiB puts it back on the pinned count of 6 000 000"), which reads as a description of what is plotted. What is plotted is the 3 GiB shape — drain.corpus 49 152 at 64 KiB with all eleven rows setup_bound and withheld, 3 145 728 at 1 KiB — and that is what the chart captions name. Now stated as what the next in-process pass produces, with the published shape named beside it.

2. A caption example that exists in no chart. The section illustrated the corpus disclosure with "corpus differs by backend: inmemory 524k; …". The committed charts carry "corpus differs by backend: inmemory, kafka, nats, rabbitmq, redis 49k / 3.1M / 6.0M; sqs 49k / 60k". Quoted as it is.

3. A stale failure count, in two places. The "consumed before assembly" note counted four such cells, three of them RabbitMQ. The document carries two: in-process consume_batch (37 000 of 49 152) and RabbitMQ consume_parallel (46 467 of 49 152), both 64 KiB / 8 consumers. Fixed in benches/README.md and in the scripts/bench.sh comment it is derived from.

4. docs/pages/ops/performance.mdx mixed a probe with a row. The batching paragraph put a 500/50/10-message flush sweep (55k / 80k / 79k msg/s) next to "about 356k" for the parallel consumer as one measurement. The sweep is an off-matrix probe — the pinned matrix has no batch-size axis — and the 356k is a document row (consume_parallel, 64 KiB, 1 consumer, 355 921) that is itself setup_bound and withheld from the charts. Each side is now attributed, and the unsupported "below about 1 KiB the batch path leads" is replaced with the document's own payload axis: in-process batch ahead 3–10× at 64 B and 2–7× at 1 KiB, behind 2–4× at 64 KiB.

Failure triage — all 12, against the runbook's "Reading a run" checklist

Every failure in the document is at 64 KiB, tier moderate. No other payload or tier failed in any leg.

class	legs	n	method	verdict
A — consumed before assembly	inmemory, rabbitmq	2	drain	wants a larger corpus, not a rerun
B — XADD … OOM at maxmemory	redis	9	offered_load	the defect #192 fixed, 19 h after the leg was measured
C — publish ack timeout during the broadcast fill	nats	1	fill	host/broker transient

• A / in-process — the remedy exists (32 GiB, correction 1) and is not in the published leg.
• A / RabbitMQ — the remedy does not exist: --drain-messages is already the pinned 6 000 000 and the corpus is byte-bound (49 152 × 65 536 = exactly 3 GiB, the 8 GB Docker VM's limit). A structural cap at 64 KiB × 8 consumers, correctly recorded as a failure rather than a bad row.
• B / Redis — nine offered_load rungs refused. The leg (bench(results): replace the redis run with the re-measured pass #186, 2026-09-10 00:21) is 19 h older than the trim-cadence fix (feat(redis): make the acked-entry trim cadence configurable #192, 19:32) that examples/redis/stress.rs now connects with a 1 s interval. Cost is dispatch-latency coverage only: the 64 KiB ladder carries 15 rows against 34 at the other payloads. All twelve 64 KiB drain rows are intact, so no published ceiling is affected.
• C / NATS — broadcast, 64 KiB, 8 consumers, 864 of 1000 records acked. Producer-side, and the only failure in the cleanest 64 KiB leg in the set (0/12 setup_bound). Its chart consequence is larger than the entry: the 1-, 2- and 4-consumer rows of that series all exist and are all setup_bound, so the NATS 64 KiB broadcast series publishes nothing. The charts account for this on their face; the dispatch-latency panel already prints 12 recorded failure(s) in the dataset — see failures[].

handler_cost, checked across the whole document. setup_bound rows fall in two places. The flows measured without a drain (broadcast, consume_fifo, dlq_drain — the tier's 5 000 messages, short by construction) account for 88 of them. Of the drain rows the throughput charts publish from: NATS 0/36, Redis 0/36, SQS 0/24, RabbitMQ 1/35, Kafka 7/36 (all 64 KiB, 0.50–0.91 s, at a byte-bound corpus a rerun cannot change), in-process 12/35 (the eleven 64 KiB cells at 0.09–0.46 s, plus one 1 KiB cell at 0.95 s). The two legs carrying withheld 64 KiB rates are the two measured before #179, and only in-process has an available remedy.

The rest of the checklist. drain.duplicates is 0 on all 202 drain rows, on every backend. No zero-rate row exists in any leg (#194's defect class). drain.unique_at_start is under half the corpus on every published drain row — worst case 0.448 (RabbitMQ consume_batch, 64 KiB, 8 consumers); the cells that crossed that line are the class-A failures above, recorded as failures rather than as rows.

Verification

cargo nextest run --no-default-features
Summary [0.539s] 346 tests run: 346 passed, 0 skipped

including the_committed_results_document_renders_every_family, the byte-compare of the committed SVGs against the committed document.

[zannis]

One more overclaim, found by re-deriving the ratios rather than re-reading them — 4421f2e

Verifying this PR's own numbers against benches/results/bench-results.json
turned up a claim it introduced that the document does not support.

The batching paragraph stated the in-process batch/parallel ratio as 3-10x at
64 B and 2-7x at 1 KiB. Recomputed from the drain rows:

payload	1c	2c	4c	8c	range
64 B	3.77x	2.76x	9.83x	10.47x	2.8 - 10.5x
1 KiB	1.96x	2.11x	6.19x	6.90x	2.0 - 6.9x
64 KiB	0.27x	0.43x	0.50x	—	2.0 - 3.7x slower

The 64 B 2-consumer cell measures 2.76x, and all four 64 B rows are
handler_cost: framework — plotted, not withheld. So a range floor of "3x" is
contradicted by a bar a reader can see. The 1 KiB floor had the same rounding
(1.96x written as 2x). Both floors rounded inward, which is the direction that
makes a claim stronger than its data.

Now stated to one decimal, so each bound is a row: 2.8-10.5x, 2.0-6.9x,
2.0-3.7x slower. The 64 KiB figure was already rounded outward and is
unchanged in substance.

Everything else in the diff re-derived and confirmed

Checked against the committed document rather than against the previous prose:

• In-process 64 KiB: drain.corpus 49 152, exactly 11 drain rows, all
  eleven setup_bound. 1 KiB corpus 3 145 728. ✓
• "about 356k": consume_parallel / 1c / 64 KiB = 355 921. ✓
• Two "consumed before assembly" failures, not four: in-process
  consume_batch 8c 64 KiB at 37 000 / 49 152, RabbitMQ consume_parallel
  8c 64 KiB at 46 467 / 49 152. No third or fourth exists in the document. ✓
• Chart caption string inmemory, kafka, nats, rabbitmq, redis 49k / 3.1M / 6.0M; sqs 49k / 60k is present in both throughput-vs-payload.svg and its
  dark variant; SQS's corpora are 60 000 / 60 000 / 49 152, so 49k / 60k is
  right. The replaced inmemory 524k example appears nowhere in the repo. ✓

Whole-document invariants, unchanged by this PR but re-run: 202 drain rows,
0 with duplicates, 0 zero-rate rows anywhere, 12 recorded failures
(1 in-process / 1 NATS / 1 RabbitMQ / 9 Redis).

One clarification on a number in my issue comment: the worst unique_at_start
fraction 0.448 is the worst over all drain rows (RabbitMQ consume_batch
8c 64 KiB, 22 000 / 49 152) — and that row is setup_bound, so it is withheld.
Over published rows the worst is 0.083. Under half either way, which is
the property that matters, but the two are not the same statistic.

Gates

cargo nextest run --no-default-features -E 'binary(chartgen)' — 131 tests
run, 131 passed, 0 skipped, including
the_committed_results_document_renders_every_family. Still prose-only: no
results-document, chart or code change, so the committed SVGs byte-match.

Analyze (rust), which was still running when I last commented, finished
success on 6aad841. Checks are re-running on 4421f2e.