Skip to content

perf: Use std::to_chars and thread-local buffers for faster stateless meter send #134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ecbadeaux merged 4 commits into from
Apr 7, 2026
Merged

perf: Use std::to_chars and thread-local buffers for faster stateless meter send #134

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0918cc5
perf: Use std::to_chars and thread-local buffers for faster stateless…
jasonk000 Mar 26, 2026
3f48e09
fix: allow handling of super-large length floating points
jasonk000 Mar 26, 2026
9bf2f84
fix: include headers to allow build inside envoy project
jasonk000 Mar 26, 2026
17c2bef
move thread_local to shared, handle double denormals
jasonk000 Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions spectator/stateless_meters.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,19 +77,28 @@ class StatelessMeter {
if (value_prefix_.empty()) {
value_prefix_ = detail::create_prefix(*id_, Type());
}
auto msg = absl::StrFormat("%s%f", value_prefix_, value);
// remove trailing zeros and decimal points
msg.erase(msg.find_last_not_of('0') + 1, std::string::npos);
msg.erase(msg.find_last_not_of('.') + 1, std::string::npos);
publisher_->send(msg);
// std::to_chars with fixed format: no trailing zeros, no scientific notation,
// ~5-10x faster than absl::StrFormat("%s%f",...) + erase.
char num_buf[327]; // fixed-format double worst case: DBL_MAX ~309 digits
Copy link
Copy Markdown
Member

@mathetake mathetake Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this method accessed concurrently? if not, I wonder if having this buf embedded as a class member is better. Allocated once on a heap together with the class vs having this hundreds of bytes on stack for each function call

Copy link
Copy Markdown
Member

@mathetake mathetake Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh sorry the PR title says "thread local".

Copy link
Copy Markdown
Member Author

@jasonk000 jasonk000 Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes there was a tradeoff/decision. The alternative path is to write directly to tl_msg as:

tl_msg.reserve(length_of_value_prefix + 327)
// then write to_chars directly into tl_msg

The reserve is required to guarantee sufficient capacity. However, the reserve() call always zeros the results so we end up writing 327 \0s to the buffer, which is not needed majority of the time. Easy enough to just put it on the stack and only pick up what we use.

I have a fix to put through as 327 is not enough in some cases.

auto [ptr, ec] = std::to_chars(num_buf, num_buf + sizeof(num_buf), value,
std::chars_format::fixed);
// thread_local retains capacity after warmup — zero allocation per send.
thread_local std::string tl_msg;
tl_msg.assign(value_prefix_);
tl_msg.append(num_buf, ptr);
publisher_->send(tl_msg);
}

void send_uint(uint64_t value) {
if (value_prefix_.empty()) {
value_prefix_ = detail::create_prefix(*id_, Type());
}
auto msg = absl::StrFormat("%s%u", value_prefix_, value);
publisher_->send(msg);
char num_buf[24];
auto [ptr, ec] = std::to_chars(num_buf, num_buf + sizeof(num_buf), value);
thread_local std::string tl_msg;
tl_msg.assign(value_prefix_);
tl_msg.append(num_buf, ptr);
publisher_->send(tl_msg);
}

private:
Expand Down
Loading