Skip to content

bench(results): add the rabbitmq and redis runs to the results document - #166

Merged
zannis merged 2 commits into
mainfrom
zannis/bench/rabbitmq-redis-results
Sep 8, 2026
Merged

bench(results): add the rabbitmq and redis runs to the results document#166
zannis merged 2 commits into
mainfrom
zannis/bench/rabbitmq-redis-results

Conversation

@zannis

@zannis zannis commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Summary

Measure RabbitMQ and Redis on the pinned drain matrix, on the same host as the published in-process and Kafka runs, and regenerate every chart from the merged document. Both backends now carry a consume_batch series, and the README and performance page name the four measured backends.

  • RabbitMQ: 145 rows, 3 failures. The batch drain ceiling sits 1.35 to 1.5x above consume_parallel at every payload. The failures are the 64 KiB drain at eight consumers, where the byte-capped 49 152-message corpus is mostly gone before the last worker is assigned; the in-process run already carries the same failure.
  • Redis: 144 rows, 5 failures. The batch drain reaches 926k msg/s at 64 B against 453k for consumer_group. Three failures are the consume_fifo cells, which the matrix's --concurrent flag makes register_fifo reject on this backend; one is the purge of a six-million-entry stream timing out on DEL; one is a producer XADD timeout on the 64 KiB load ladder.

Known outliers left as measured

RabbitMQ consume_fifo at 64 B and Redis broadcast at four consumers came in well below their neighbours. Both were re-run in isolation and came back in line, but the harness merges whole runs, so they stay until the backend is measured again.

Not in the document

  • NATS: the unpaced drain fill deadlocks. NatsPublisher::publish_batch fires every publish before awaiting any ack, and async-nats 0.49 defaults to blocking backpressure at 5 000 in-flight acks, so eight fill producers on one context wedge with no timeout. Reproduced deterministically at a 1M-message fill. Needs a publisher fix before the run.
  • SQS: LocalStack was OOM-killed under the pinned six-million-message corpus. fix(bench): give sqs its own drain corpus instead of a 51-hour pass #163 has since given SQS its own corpus; the consume_fifo cells still run at about 2.6 msg/s on LocalStack.

Verification

scripts/bench.sh charts regenerated the SVGs and the chartgen byte-compare suite passes (119 tests).

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 8, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
shove-docs 2f235fe Commit Preview URL

Branch Preview URL
Sep 08 2026, 11:13 AM

@zannis
zannis force-pushed the zannis/bench/rabbitmq-redis-results branch from e711473 to 5719619 Compare September 8, 2026 10:39
@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Measure RabbitMQ and Redis on the pinned drain matrix, on the same host as
the published in-process and Kafka runs, and regenerate every chart from the
merged document. Both backends now carry a consume_batch series.

RabbitMQ: 145 rows, 3 failures. The batch drain ceiling sits 1.35 to 1.5x
above consume_parallel at every payload. The three failures are the 64 KiB
drain at eight consumers, where the byte-capped 49 152-message corpus is
mostly gone before the last worker is assigned; the in-process run already
carries the same failure.

Redis: 144 rows, 5 failures. The batch drain reaches 926k msg/s at 64 B
against 453k for consumer_group. Three failures are the consume_fifo cells,
which the matrix's --concurrent flag makes register_fifo reject on this
backend; one is the purge of a six-million-entry stream timing out on DEL;
one is a producer XADD timeout on the 64 KiB load ladder.

Two cells in the document are known low outliers that did not reproduce on
a targeted re-run: RabbitMQ consume_fifo at 64 B and Redis broadcast at
four consumers. They stay as measured until the backend is re-run whole.

NATS and SQS are not in the document. The NATS drain fill deadlocks in the
publisher's in-flight ack budget, and LocalStack was OOM-killed under the
pinned SQS corpus before the per-backend corpus deviation landed.
@zannis
zannis force-pushed the zannis/bench/rabbitmq-redis-results branch from 4d7b47c to 2f235fe Compare September 8, 2026 11:10
@zannis
zannis merged commit cbd4aed into main Sep 8, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant