Skip to content

Serialize non-native BEVE map/pair keys via a generic-array fallback - #2669

Open
stephenberry wants to merge 1 commit into
mainfrom
beve-non-native-key-diagnostics
Open

Serialize non-native BEVE map/pair keys via a generic-array fallback#2669
stephenberry wants to merge 1 commit into
mainfrom
beve-non-native-key-diagnostics

Conversation

@stephenberry

@stephenberry stephenberry commented Jun 22, 2026

Copy link
Copy Markdown
Owner

Summary

A BEVE object (map) header encodes a single shared key type, so only numeric or string-like keys fit the compact map encoding. Keys that reduce to neither (e.g. a multi-field struct, as in std::unordered_map<user_defined_key, int>) previously failed deep inside the serializers with a cryptic template error.

Rather than requiring users to restructure such keys by hand, the map/pair serializers now fall back to a generic array of [key, value] entries when the key isn't native. Each key is written as a full BEVE value, so the container round-trips with no code changes. Each entry is encoded identically to std::tuple<Key, Value> (a non-native std::pair and the matching std::tuple produce the same bytes). Native (numeric/string) keys are unaffected and keep the compact shared-header encoding.

Covers std::map, std::unordered_map, std::vector<std::pair<Key, Value>>, and a standalone std::pair<Key, Value>. Documented in docs/binary.md.

Hardening

Bound the element count against the remaining buffer before allocating, on both the native vector-of-pairs reader and the new entry-array readers, mirroring the guard the std::map reader already has. Previously a hostile count drove unbounded emplace_back (OOM/DoS).

Testing

  • beve_test builds clean under -Wall -Wextra -pedantic with ASan + UBSan.
  • 389 tests / 9665 asserts pass.
  • New round-trip tests: the issue's std::unordered_map<CompositeKey, int> case, std::map, std::vector<std::pair>, a standalone std::pair, an explicit entry struct, std::tuple, and a pair == tuple byte-identity check.
  • DoS-protection test for the vector-of-pairs reader (passes under a 4 GB memory cap).

Closes #2666.

…2666)

A BEVE object (map) header encodes a single shared key type, so only numeric or
string-like keys fit the compact map encoding. Keys that reduce to neither (e.g. a
multi-field struct, as in std::unordered_map<user_defined_key, int>) previously failed
deep inside the serializers with a cryptic template error.

Rather than requiring such keys to be restructured by hand, the map/pair serializers
(read/write/size) now fall back to a generic array of [key, value] entries, where each
key is written as a full BEVE value. The container round-trips with no user changes.
Each entry is encoded identically to std::tuple<Key, Value>, so a non-native std::pair
and the matching std::tuple produce the same bytes. Native keys are unaffected and keep
the compact shared-header encoding.

Covers std::map, std::unordered_map, std::vector<std::pair<Key, Value>>, and a standalone
std::pair<Key, Value>.

Also bound the element count against the remaining buffer before allocating, on both the
native vector-of-pairs reader and the new entry-array readers, mirroring the std::map
reader (prevents unbounded emplace_back / OOM on a hostile count).

Adds round-trip tests (incl. the issue's unordered_map case and a pair==tuple byte check)
and a DoS-protection test for the vector-of-pairs reader.

Closes #2666.
@stephenberry
stephenberry force-pushed the beve-non-native-key-diagnostics branch from f8ad800 to df93f7b Compare June 22, 2026 18:20
@stephenberry stephenberry changed the title Reject non-native BEVE map/pair keys at compile time; bound vector-of-pairs reads Serialize non-native BEVE map/pair keys via a generic-array fallback Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BEVE std::unordered_map<user_defined_key, int> compilation error

1 participant