bench(results): add the rabbitmq and redis runs to the results document - #166
Merged
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
shove-docs | 2f235fe | Commit Preview URL Branch Preview URL |
Sep 08 2026, 11:13 AM |
zannis
force-pushed
the
zannis/bench/rabbitmq-redis-results
branch
from
September 8, 2026 10:39
e711473 to
5719619
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Measure RabbitMQ and Redis on the pinned drain matrix, on the same host as the published in-process and Kafka runs, and regenerate every chart from the merged document. Both backends now carry a consume_batch series. RabbitMQ: 145 rows, 3 failures. The batch drain ceiling sits 1.35 to 1.5x above consume_parallel at every payload. The three failures are the 64 KiB drain at eight consumers, where the byte-capped 49 152-message corpus is mostly gone before the last worker is assigned; the in-process run already carries the same failure. Redis: 144 rows, 5 failures. The batch drain reaches 926k msg/s at 64 B against 453k for consumer_group. Three failures are the consume_fifo cells, which the matrix's --concurrent flag makes register_fifo reject on this backend; one is the purge of a six-million-entry stream timing out on DEL; one is a producer XADD timeout on the 64 KiB load ladder. Two cells in the document are known low outliers that did not reproduce on a targeted re-run: RabbitMQ consume_fifo at 64 B and Redis broadcast at four consumers. They stay as measured until the backend is re-run whole. NATS and SQS are not in the document. The NATS drain fill deadlocks in the publisher's in-flight ack budget, and LocalStack was OOM-killed under the pinned SQS corpus before the per-backend corpus deviation landed.
zannis
force-pushed
the
zannis/bench/rabbitmq-redis-results
branch
from
September 8, 2026 11:10
4d7b47c to
2f235fe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Measure RabbitMQ and Redis on the pinned drain matrix, on the same host as the published in-process and Kafka runs, and regenerate every chart from the merged document. Both backends now carry a
consume_batchseries, and the README and performance page name the four measured backends.consume_parallelat every payload. The failures are the 64 KiB drain at eight consumers, where the byte-capped 49 152-message corpus is mostly gone before the last worker is assigned; the in-process run already carries the same failure.consumer_group. Three failures are theconsume_fifocells, which the matrix's--concurrentflag makesregister_fiforeject on this backend; one is the purge of a six-million-entry stream timing out onDEL; one is a producerXADDtimeout on the 64 KiB load ladder.Known outliers left as measured
RabbitMQ
consume_fifoat 64 B and Redis broadcast at four consumers came in well below their neighbours. Both were re-run in isolation and came back in line, but the harness merges whole runs, so they stay until the backend is measured again.Not in the document
NatsPublisher::publish_batchfires every publish before awaiting any ack, and async-nats 0.49 defaults to blocking backpressure at 5 000 in-flight acks, so eight fill producers on one context wedge with no timeout. Reproduced deterministically at a 1M-message fill. Needs a publisher fix before the run.consume_fifocells still run at about 2.6 msg/s on LocalStack.Verification
scripts/bench.sh chartsregenerated the SVGs and the chartgen byte-compare suite passes (119 tests).