glaze 8.0.0

[BrewTestBot]

Created by brew bump

---

Created with brew bump-formula-pr.

Details

release notes

Glaze v8.0.0 locks in full UTF-8 validation for strings by default. This release also adds robust security from DoS attack with bounded recursion, out-of-range integers are rejected instead of wrapped, and truncated input reports truncation. BEVE variants move to the v2 wire format, variant tagging is decided per variant rather than per alternative, and streaming grows real boundaries. Buffers that used to parse will now correctly fail.
Breaking Changes
Full UTF-8 validation on every read
glz::read_json accepted some forms of invalid UTF-8 inside strings. RFC 8259 §8.1 requires JSON text to be UTF-8, and the documentation already claimed validation happened, but it wasn't complete. Now UTF-8  is complete.
std::string s{};
// 0xC0 0x80 is an overlong encoding of NUL, illegal UTF-8
auto ec = glz::read_json(s, "\"\xC0\x80\"");
// v7: ec == none, s holds the invalid bytes
// v8: ec == error_code::invalid_utf8
Validation covers materialized strings, values you do not model, values reached through glz::skip or glz::raw_json, map keys, and keys matching no member — so a malformed byte in a field you never look at fails the whole parse. A key that matches a member name is never materialized and is not checked. JSONC comment bytes are exempt.
Writing is deliberately unchanged. A value obtained by reading is already validated, so a round trip needs no second pass. This does make round-tripping asymmetric: a std::string holding non-UTF-8 bytes writes successfully and then fails to read back.
If an earlier stage already guarantees the encoding, opt out:
struct unchecked_opts : glz::opts {
   bool validate_utf8 = false;
};
auto ec = glz::read<unchecked_opts{}>(v, buffer);
The disabled branch compiles away entirely, so opting out costs nothing. On 6000-record documents, turning it off recovers 65% on a 60% non-ASCII document and is within noise on pure ASCII. With it on, the cost is about 5–6% end to end on twitter.json (22% non-ASCII, string heavy) and within noise of unvalidated parsing on a pure-ASCII document into structs.
BEVE variants are ordinary objects (BEVE v2)
The type-tag extension (extension id 1, header byte 0x0E) is deprecated. A variant now writes as an ordinary self-describing value rather than a positional index, per beve-org/beve discussion #30. The positional index was the one BEVE construct that could not be interpreted without an external schema; beve_to_json of a variant now equals write_json of the same value.



Case
Wire form




Tagged, object alternative
merged object { tag : id, ...members }


Tagged, positional (structs_as_arrays / write_beve_untagged)
adjacent array [ id, value ]


Untagged / scalar / array alternative
the bare active value



No 0x0E byte is ever emitted.
Glaze v8 reads v1 data. The legacy 0x0E path is retained verbatim, and skip and beve_to_json keep their v1 branches. Glaze v7 cannot read v8 variant data — a v2 variant is a structurally valid v1 BEVE value, so a generic v1 decoder parses it, but a pre-v2 Glaze reading it back into a std::variant fails because its reader requires the 0x0E byte. If you have BEVE data at rest containing variants, or a mixed-version deployment, upgrade readers before writers.
Untagged variants whose alternatives are genuinely indistinguishable on the wire collapse to the first alternative: two structs with identical field sets, two empty structs, std::vector<int> versus std::deque<int>, and under structs_as_arrays any two alternatives sharing a positional shape. Declare a tag/ids discriminator — that now works in positional mode too, via the adjacent form. A map whose keys are all field names of some struct alternative resolves to the struct. Adding a map or pair alternative changes how struct data carrying an unknown key decodes: the foreign key is taken as evidence the map wrote it, a deliberate divergence from the JSON reader.
Variant tagging is chosen per variant
Representation used to be picked inside the std::visit, so an alternative that could not carry a merged discriminator fell back to no discriminator at all. One variant emitted several unrelated shapes, and two tag-less alternatives sharing a shape read back as the wrong one with no error:
using AV = std::variant<std::vector<double>, std::deque<double>>;   // tag = "type"
AV v = std::deque<double>{1,2,3};
glz::read_json<AV>(glz::write_json(v).value())->index();   // v7: 0 — wrong, silently
Representation is now decided once per variant type and every alternative obeys it:



glz::meta declares
Representation
Shape




nothing
none
the alternative's own value


tag
internal
{"type":"circle","radius":5}


tag + content
adjacent
{"type":"vec","value":[1,2,3]}



Three breaks follow:

tag alone with a non-object alternative is a static_assert, naming the variant and the offending alternative and pointing at content. It fires on exactly the code that is silently wrong today.
std::monostate under internal tagging writes as the discriminator alone — {"type":"NONE"}, was null. Both readers still accept a bare null, so the break is write-only and existing data keeps parsing. This is the one change no compile error announces.
A glaze_object_t alternative declaring a member named like the tag no longer emits the key twice. It supplies the discriminator itself, as reflectable alternatives already did; the member must hold a declared id for the round trip to work.

YAML and JSONB implement internal tagging only and reject content at compile time. MsgPack and CBOR already encode every variant as a two-element array and are unaffected. TOML has no discriminator support either way.
Out-of-range integers are rejected instead of wrapped
Range checks ran after the operation that destroyed the value they were checking. This reaches ordinary JSON integers with no exponent involved:
glz::read_json(std::vector<uint8_t>{},  "[300]")    // was: 44           now: error
glz::read_json(std::vector<uint16_t>{}, "[75495]")  // was: 9959         now: error
glz::read_json(std::vector<int8_t>{},   "[13e2]")   // was: 20           now: error
glz::read_json(std::vector<int32_t>{},  "[5e9]")    // was: 705032704    now: error
699 of the 744 values in [256, 999] were accepted as uint8_t. Also fixed: 1e256 decoding to 1 through the unsigned atoi and glz::stoui paths, negative zero accepted or rejected depending on integer width, and a zero mantissa such as 0e19 wrongly rejected — that one is now accepted, since zero stays zero however far it is scaled.
Measured against a __int128 reference over 103,135 inputs: accepted-with-a-wrong-value went from 17,714 to 0, and wrongly-rejected from 150 to 0. If you relied on lenient truncation, widen the destination type.
Reader recursion is bounded at 256 levels
The BEVE, CBOR, MessagePack and JSON readers enforced no depth limit, so a small hostile buffer overflowed the stack and crashed the process. Two bytes of input bought a BEVE nesting level; one byte bought a CBOR or MessagePack level.



Format
Before
After




BEVE, std::variant<A,B>
150k levels (≈300 KB) SIGSEGV
exceeded_max_recursive_depth


BEVE, struct, error_on_unknown_keys = false
500k levels (≈1 MB) SIGSEGV
exceeded_max_recursive_depth


CBOR, struct, error_on_unknown_keys = false
200k levels (400 KB) SIGSEGV
exceeded_max_recursive_depth


MessagePack, recursive struct
100k levels (200 KB) SIGSEGV
exceeded_max_recursive_depth


JSON, recursive struct
100k levels (900 KB) SIGSEGV
exceeded_max_recursive_depth



Legitimate documents nested deeper than max_recursive_depth_limit (256) now error. Bounding the recursion turned the overflow into a hang, so variant speculation is bounded too: each rejected alternative charges what it parsed against a per-read budget of max(8 × input, 1 MB). BEVE went from 189 bytes → 55 s to a constant ~8 ms at any depth; JSON from 339 bytes → 40 s to the same.
Truncated non-null-terminated input reports unexpected_end
error_code::end_reached is documented as a non-error code and was never meant to escape a read. It did:
static constexpr glz::opts options{.null_terminated = false};
glz::generic j{};
glz::read<options>(j, "[1,2");  // v7: end_reached (a documented non-error). v8: unexpected_end
Every registry read now runs non-null-terminated, so this reached the wire: a truncated REPE body answered its client with end_reached, telling it the request parsed and merely stopped early. end_reached no longer escapes a read at all. json_stream_reader, which raises it itself to signal end of stream, is unaffected.
Input holding no value reports no_read_input
glz::read<{.null_terminated = false}>(v, "   ") and glz::read<{.null_terminated = false, .comments = true}>(v, "// hi\n") returned success with the destination untouched. Both now report no_read_input, the code an empty buffer already reported. Through the registry this had been answering a malformed request with a success response, and would have turned a REST PUT with a blank body from 400 into 204.
repe::read_params returns bool, not size_t
This one is quiet — the old form still compiles and inverts:
if (glz::repe::read_params<Opts>(params, state) == 0) { return; }  // now discards every successful read
The byte count was wrong to begin with. Without a terminator, a variant alternative that resolves at the end of the buffer rewinds its iterator, so a completed read can report zero bytes consumed, and the registry took that for an error and returned without writing any response at all. Audit any custom REPE call handler for this.
Streaming into a non-owning view is a compile error
A refill moves the streaming window, so a std::string_view produced before one addresses bytes that have since been overwritten:
std::vector<std::string_view> views{};
glz::read_json(views, buffer);   // 512 byte window
// v7: ec == none, and 28 of 40 elements hold the wrong text
There is no runtime signal to check — the views are valid pointers into a live buffer that address the wrong bytes, and ASan does not fire because the stale bytes are still inside the buffer's own allocation. A bigger buffer does not remove the problem; it just changes which elements are wrong.
The guard sits next to each assignment that hands out a pointer into the window — string_view_t, basic_raw_json<T>/basic_text<T> over a view, and the zero-copy std::span<const T> BEVE reader — so it fires wherever the view sits, behind a tuple, a map key, a glz::custom setter, ten structs down, and never on a type that merely looks like it holds one. NDJSON and json_stream_reader are covered for free.
Buffered reads are untouched. Zero-copy reads into views from a buffer holding the whole document remain fully supported.
Streaming buffers no longer bind to the buffered read path
A streaming buffer satisfies contiguous, so it bound to the buffered glz::read and was parsed as a flat span with null_terminated left on, over memory that carries no terminator. ASan reported a heap-buffer-overflow read. read_json had is_input_streaming overloads and was safe; read_jsonc, validate_json, validate_jsonc, read_beve and direct glz::read all overran. Fixed at read rather than per format, so a format added later inherits the behavior. The buffered overloads changed from contiguous auto&& buffer to a named contiguous Buf so the constraint can name it; the template parameter order is unchanged.
Lazy APIs take template <auto Opts>
glz::lazy_json and glz::lazy_beve declared their options NTTP as template <opts Opts>, so a user options struct deriving from glz::opts — the pattern glz::opts itself documents — was sliced, silently dropping every derived field:
struct my_opts : glz::opts { bool bools_as_numbers = true; };
auto doc = glz::lazy_json<my_opts{}>(buffer);  // v7: compiles, bools_as_numbers is gone
Ordinary call sites are unaffected and mangling is byte-identical for glz::opts-valued instantiations, so there is no ODR/ABI hazard across mixed-version TUs. Two patterns break: a user forward declaration written as template <glz::opts Opts> struct lazy_document; no longer redeclares (Glaze's own headers used exactly that form), and glz::lazy_document<glz::opts{}> d = *glz::lazy_json<derived_opts>(buf); no longer converts — that is the slicing going away.
Smaller source-level breaks

skip_string_opts and skip_until_closed_opts direct constructors require every argument. validate_utf8_ lands before the pre-existing null_terminated_; a caller that omitted either would silently get behavior it did not ask for.
A registry options struct that fixes null_terminated = true (static constexpr bool null_terminated = true, the shape opts_csv uses) is now a static_assert. That bound is what keeps the registry inside the caller's buffer, so it must not be opted out of quietly.
detail::handle_slice and detail::seek_array_index no longer default Opts. Internal to jmespath.hpp; a call site that forgot <Opts> used to compile and silently reset the whole option set.
Failed reads of REPE notifications are no longer answered. A notification is a request the sender said it will not read a reply to; read_params was writing a full error response for a malformed one.
Ranges of std::chrono::duration and count-based time points encode differently in BEVE: now a numeric typed array of the rep, byte-for-byte identical to a range of the rep itself, rather than a generic array with a type tag per element. Duration and time-point map and pair keys encode as numeric keys.

Improvements

Represent BEVE variants as ordinary objects (BEVE v2) by @stephenberry in Represent BEVE variants as ordinary objects (BEVE v2) stephenberry/glaze#2707
Validate UTF-8 on read by @stephenberry in Validate UTF-8 on read stephenberry/glaze#2733
Add a compile-time validate_utf8 option to disable UTF-8 validation by @stephenberry in Add a compile-time option to disable UTF-8 validation stephenberry/glaze#2756
Choose variant tagging per variant, not per active alternative by @stephenberry in Choose variant tagging per variant, not per active alternative stephenberry/glaze#2728
Bound reader recursion depth so hostile input cannot overflow the stack by @stephenberry in Bound reader recursion depth so hostile input cannot overflow the stack stephenberry/glaze#2731
Bound every registry read to its buffer rather than a null terminator by @stephenberry in Bound every registry read to its buffer rather than a null terminator stephenberry/glaze#2732
Generalize std::chrono serialization across all formats by @stephenberry in Generalize std::chrono serialization across all formats (#2671) stephenberry/glaze#2678 — durations and count-based clocks now work in BEVE, MsgPack and BSON, not just JSON/CBOR/TOML, through a single generic conversion
Add YAML support for system_clock time points and year_month_day by @stephenberry in Add YAML support for system_clock time points and year_month_day stephenberry/glaze#2715 — also fixes the libstdc++ trap where high_resolution_clock aliases system_clock
Add an HTTP headers API (glz::http_headers) by @annihilatorq in Add HTTP headers API stephenberry/glaze#2709 — a std::ranges::forward_range header store that preserves repeated fields, original case and field order, with add/set/fields/values/contains_token/serialize
Give the NDJSON reader refill points by @stephenberry in Give the NDJSON reader refill points stephenberry/glaze#2736 — format_supports_streaming<NDJSON> is now true, and a record wider than the window reports streaming_unsupported naming the buffer rather than blaming the document
Route streaming buffers through read_streaming in glz::read by @stephenberry in Route streaming buffers through read_streaming in glz::read stephenberry/glaze#2734
Add an opt-in streaming cursor to lazy JSON iteration by @stephenberry in Add an opt-in streaming cursor to lazy JSON iteration stephenberry/glaze#2745, extracted and reworked from lazy JSON: opt-in streaming cursor for read_into + sequential iteration stephenberry/glaze#2683 by @psiha — lazy_streaming_cursor records a consumed value's extent so the next ++ jumps rather than re-scanning. A 9 MB array of three-field objects with read_into per row goes 575 → 955 MB/s (+66%) on an Apple M1 with clang -O3. Two size_t on lazy_document when enabled, nothing when disabled
Add an opt-in wide number skip for lazy JSON traversal by @stephenberry in Add an opt-in wide number skip for lazy JSON traversal stephenberry/glaze#2746, also from lazy JSON: opt-in streaming cursor for read_into + sequential iteration stephenberry/glaze#2683 by @psiha — lazy_wide_number_skip, off by default: +34% on long numeric runs, −2% to −6% elsewhere, and within 0.1% of the previous skip path when off
Report which SIMD backend each subsystem compiled to by @stephenberry in Report which SIMD backend each subsystem compiled to stephenberry/glaze#2749 — glz::simd_info (detected, utf8_validation, string_escape, float_write), reflectable and constexpr. A struct rather than one name because the fields genuinely disagree: an AVX-512 build escapes strings with AVX2, and a plain SSE2 build validates UTF-8 with the scalar validator
Take lazy JSON/BEVE options as auto so derived opts are not sliced by @stephenberry in Take lazy JSON/BEVE options as auto so derived opts are not sliced stephenberry/glaze#2744
Add REPE and JSON RPC registry fuzz targets by @stephenberry in Add REPE and JSON RPC registry fuzz targets stephenberry/glaze#2737 — every buffer allocated at exactly its content size, so an over-read lands in an ASan redzone rather than allocator slack
Document repe::read_params and its bool return by @stephenberry in Document repe::read_params and its bool return stephenberry/glaze#2738

Fixes

Reject streaming reads into non-owning views by @stephenberry in Reject streaming reads into non-owning views stephenberry/glaze#2740
Report truncated non-null-terminated input as unexpected_end by @stephenberry in Report truncated non-null-terminated input as unexpected_end stephenberry/glaze#2735
Reject out-of-range integers instead of wrapping them by @stephenberry in reject out-of-range integers instead of wrapping them stephenberry/glaze#2723
Fix exponent digit handling in integer parsing by @stephenberry in fix exponent digit handling in integer parsing stephenberry/glaze#2721
Fix exponent overflow in unsigned atoi by @uwezkhan in fix exponent overflow in unsigned atoi stephenberry/glaze#2719
Fix signed overflow in the %YAML directive version parser by @stephenberry in fix signed overflow in the %YAML directive version parser stephenberry/glaze#2722
Prevent string truncation at embedded nulls across all formats by @MuhammadShahzebMalik786 in Fix #2753: Prevent string truncation at embedded nulls stephenberry/glaze#2754 — string writing now respects .size() rather than decaying to const char*
Fix dangling std::string_view when reading YAML scalars by @stephenberry in Fix dangling std::string_view when reading YAML scalars stephenberry/glaze#2716 — the reader decoded into a local std::string and assigned it to the view, producing a read of freed memory with a correct length and no error code
Guard the YAML anchor alias check against end of input by @uwezkhan in guard yaml anchor alias check against end of input stephenberry/glaze#2725
Keep the lazy view writers inside their own buffer by @stephenberry in Keep the lazy view writers inside their own buffer stephenberry/glaze#2748 — write_json of a lazy view skipped with default opts{}, so on a null_terminated = false document it scanned for a sentinel that is not there ("42" emitted "429187"). Also fixes writing a root scalar, which failed with a spurious unexpected_end in both modes
Answer a REPE request whose body is empty by @stephenberry in Answer a REPE request whose body is empty stephenberry/glaze#2739 — a non-notify request with an empty body to a parameterized function endpoint got no response at all, hanging the client
Guard shrink_to_fit on a has_shrink_to_fit concept by @stephenberry in guard shrink_to_fit on a has_shrink_to_fit concept stephenberry/glaze#2712, superseding Call shrink_to_fit after growing JSON array reads when requested stephenberry/glaze#2711 by @ays7 — resizable does not imply the member, so std::list/std::forward_list failed to compile with the option on. 17 call sites across JSON, NDJSON, BEVE, CBOR and EETF
Forward opts to handle_slice in runtime read_jmespath slices by @uwezkhan in forward opts to handle_slice in runtime read_jmespath slices stephenberry/glaze#2710 — the two runtime slice call sites bound opts{}, compiling out the end guards and running past the end of a non-null-terminated buffer
Require explicit Opts on jmespath slice/index helpers by @stephenberry in require explicit Opts on jmespath slice/index helpers stephenberry/glaze#2713
Remove the unreachable object-member branch from jmespath array access by @stephenberry in remove unreachable object-member branch from jmespath array access stephenberry/glaze#2714
Use a single msgpack string write specialization by @stephenberry in Use a single msgpack string write specialization stephenberry/glaze#2752 — fixes the clang 22 frontend crash in #2742 at the root: three overlapping string specializations forced partial ordering of disjunction-heavy constrained partial specializations, exhausting clang's stack
Keep clang-cl out of the MSVC AVX-512BW fallback by @stephenberry in Keep clang-cl out of the MSVC AVX-512BW fallback stephenberry/glaze#2750 — clang-cl defines _MSC_VER and the fine-grained __AVX512*__ macros, so an AVX-512F-only build selected the AVX-512BW validator without the feature
Time the variant growth test on CPU time, not wall time by @stephenberry in Time the variant growth test on CPU time, not wall time stephenberry/glaze#2755

Migration
The first four have no compiler diagnostic:

BEVE data at rest or on a wire with variants in it? Deploy v8 readers before v8 writers. v8 reads v1; v7 cannot read v8 variants.
Custom REPE call handlers? read_params now returns bool. == 0 still compiles and inverts.
std::monostate in an internally-tagged variant? Writes as {"tag":"ID"} now, not null. Readers still accept null, so only new output is affected.
Ranges of std::chrono::duration in BEVE? The encoding changed to a packed numeric array.
Non-UTF-8 bytes in your data? Fix the producer or set validate_utf8 = false on your options struct.
Documents nested deeper than 256 levels? They now error with exceeded_max_recursive_depth.
Checking for end_reached at a call site? It no longer escapes a read; check unexpected_end.
Reading into std::string_view from a streaming buffer? Now a compile error naming the owning equivalent. Buffered reads into views are unchanged.
Forward-declared glz::lazy_document? Change template <glz::opts Opts> to template <auto Opts>.

Several of these breaks announce themselves as static_asserts naming the offending type, so a clean rebuild will find most of what applies to you.
Full Changelog: stephenberry/glaze@v7.9.1...v8.0.0

View the full release notes at https://github.com/stephenberry/glaze/releases/tag/v8.0.0.

---

[github-actions]

🤖 An automated task has requested bottles to be published to this PR.

Caution

Please do not push to this PR branch before the bottle commits have been pushed, as this results in a state that is difficult to recover from. If you need to resolve a merge conflict, please use a merge commit. Do not force-push to this PR branch.