Skip to content

bench(results): add the sqs run to the results document - #175

Merged
zannis merged 2 commits into
mainfrom
zannis/bench/sqs-results
Sep 8, 2026
Merged

bench(results): add the sqs run to the results document#175
zannis merged 2 commits into
mainfrom
zannis/bench/sqs-results

Conversation

@zannis

@zannis zannis commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Summary

Measure SQS against LocalStack on the same host as the other published runs, with both SQS deviations in place (the 60,000-message drain corpus from #163 and the 100-per-worker FIFO corpus from #173) and the SNS byte-limit fix from #171, so the 64 KiB batch publishes go through. 60 rows, no failures, consumer_group and broadcast recorded as unsupported per the capability traits. The document now holds every backend, and the README and performance page say what the SQS rows measure: LocalStack at a recorded corpus, not the AWS service.

The charts are regenerated with the chartgen from #176: with six backends the parallel-vs-sequenced 64 KiB pair's caption runs two lines over the base budget, so those two SVGs grow to 672 px; the other twenty render at the base height.

Also carries the codex follow-up to #176 that missed its merge: the canvas-growth test now asserts the plot minimum from the y-axis polyline rather than from tick labels at 0.8 of the minimum.

What the run shows

  • publish_batch at 64 KiB now succeeds (352 msg/s) where it failed the service's 256 KiB call limit before.
  • consume_batch with the 10-message cap runs 2.3k to 3.3k msg/s on the drain, above supervisor's 1.9k to 2.8k at the same payload and consumer count.
  • The FIFO cells ran in seconds at 100 per worker. At 40,000 messages the same cells crawled at 2.6 msg/s for hours, so LocalStack's FIFO path degrades sharply with backlog depth.
  • Every offered-load rung was "not sustained": LocalStack tops out around a few hundred to a few thousand msg/s, well under the 5k rung, so the SQS dispatch-latency figures are queue residency and the charts mark them accordingly.

Verification

scripts/bench.sh charts regenerated the SVGs and the chartgen byte-compare suite passes (128 tests). Every non-SQS run in the document is the same data as main's, re-serialised by the harness. Codex review skipped for the results and generated SVGs; the test commit was reviewed in #176's round.

@zannis
zannis marked this pull request as draft September 8, 2026 16:13
@zannis

zannis commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Draft until the chartgen caption budget is fixed: with six backends and two SQS deviation notes the parallel-vs-sequenced family's caption overflows its budget, so the SVGs on this branch are not yet regenerated from the document.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 8, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
shove-docs f09ee40 Commit Preview URL

Branch Preview URL
Sep 08 2026, 04:36 PM

Measure SQS against LocalStack on the same host as the other published runs,
with both SQS deviations in place: the 60 000-message drain corpus from #163
and the 100-per-worker FIFO corpus from #173, plus the SNS byte-limit fix
from #171 so the 64 KiB batch publishes go through. 60 rows, no failures;
consumer_group and broadcast are recorded as unsupported, as the capability
traits say.

The document now holds every backend. The README and performance page say
so, and say what the SQS rows measure: LocalStack, at a recorded corpus, not
the AWS service.
@zannis
zannis force-pushed the zannis/bench/sqs-results branch from 4e4cb70 to 1cf8e4a Compare September 8, 2026 16:33
@zannis
zannis marked this pull request as ready for review September 8, 2026 16:33
…s, not the ticks

Tick labels stop short of the plot boundaries, so a tick span at 0.8 of the
minimum could pass an undersized plot. The y axis is a two-point vertical
polyline and bounds the plot body exactly; assert the full minimum from it.
@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@zannis
zannis merged commit 845ca00 into main Sep 8, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant