Route streaming buffers through read_streaming in glz::read

[stephenberry]

A streaming buffer satisfies contiguous, so it bound to the buffered glz::read and was parsed as a flat span: one window, with null_terminated left on, over memory that carries no terminator and ends wherever the last fill stopped. The parser drops its end checks in that mode, so it reads past the window.

std::istringstream iss{json};                    // 400-element array, larger than the window
glz::basic_istream_buffer<std::istringstream, 512> buf{iss};
std::vector<int> v;
glz::read_jsonc(v, buf);

Against main at 310f270:

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x615000000500
READ of size 1 at 0x615000000500 thread T0
    #0 glz::from<JSON, std::vector<int>>::op<glz::opts{10u, true, true, true, true}>   read.hpp:2441
    #1 glz::read_jsonc<std::vector<int>, glz::basic_istream_buffer<..., 512>&>         read.hpp:5292
    #2 main

0x615000000500 is located 0 bytes after 512-byte region [0x615000000300,0x615000000500)

read_json has is_input_streaming overloads — both the out-parameter form and the expected-returning one — and is safe. It is the only helper in the library that does. read_jsonc, validate_json, validate_jsonc, read_beve and direct glz::read calls all funnel into the same buffered overload, so each of them had to remember a rule that none of them stated. glz::read<opts{}>(v, buf) overruns identically.

The fix

Fixed where they meet rather than one helper at a time. The buffered overloads exclude streaming buffers, and a pair of dispatching overloads forwards them to read_streaming, which reads with null_terminated off and refills as it goes.

Doing it at read rather than per format is the point: every helper that funnels through it is covered by the one change, and a format added later inherits the behavior instead of having to opt in. The existing read_json overloads become redundant but are left in place — they route to the same function.

The context-taking overload has nowhere to hold the streaming state the parsers refill through, so it runs the read on a streaming_context and reports the outcome back through the caller's context.

The buffered overloads change from contiguous auto&& buffer to a named contiguous Buf so the constraint can name it. The invented parameter already sat in that position, so the template parameter order is unchanged.

Effect

Two separate things, worth not conflating.

The out-of-bounds read ends for every format. read_streaming forces null_terminated off, so nothing runs the sentinel-trusting parser over an unterminated window any more.

Streaming to completion is JSON-only, and stays that way here. consume_and_refill appears only in json/read.hpp and json/skip.hpp, so the JSON reader is the only one with refill points. read_jsonc and glz::read<opts{}> on JSON now parse a document larger than the window, which they could not do at all before. Every other format still sees one window — the second commit is about saying so.

Buffered reads are untouched: they take the same overloads they always did, and the dispatching pair is unreachable for them.

Second commit: a short read now says it is one

Routing a format to read_streaming removes the overrun. It does not give a reader refill points it never had, and what the non-refilling readers reported when a document outran the window blamed the document rather than the window. Measured, 400 elements through a 512-byte window:

format	before	after
JSON	400 elements, none	unchanged
NDJSON	156 elements, none	streaming_unsupported
BEVE	unexpected_end	streaming_unsupported

The NDJSON row is the one that matters: silent data loss, reported as success. It is not new — it is what reading NDJSON from a streaming buffer has always done — but this PR is what routes those calls here, so it is the right place to stop it.

glz::format_supports_streaming<Format> names which readers can refill, and sits next to the format identifiers in forward.hpp so it is visible where formats are. It describes the reader, not the grammar: NDJSON is false because its line loop cannot refill between lines, even though the per-line values it delegates to the JSON reader can. A user-defined format specializes it.

read_streaming then reports error_code::streaming_unsupported when a non-refilling reader was not shown the whole document. The check asks whether the source is exhausted, not whether the window is drained — hence streaming_state::source_at_eof. at_eof() also demands a drained window, which would call a value that legitimately left trailing bytes a truncated read. Once the source is exhausted the window holds everything there will ever be, so whatever the reader concluded, it concluded on the full input and stands.

The new code is appended to error_code so no existing value shifts; REPE puts these on the wire.

Tests

For the dispatch: a 400-element array through read_jsonc, generic read, and generic read with a caller-supplied context — the success path, plus an error path checking the error reaches the caller's context — with a guard that read_json still behaves. They trip ASAN against main as written and pass here.

For the reporting: both short reads, and the three legitimate cases the check must not touch — a document that fits, a value followed by trailing bytes in the same window, and genuinely malformed input that keeps its own unexpected_end. Plus static_asserts on the trait and a guard that JSON still streams to completion.

Full suite is 108/108. istream_buffer_test (222 tests, 11,727 asserts) and json_test (717 tests, 61,620 asserts) are clean under -fsanitize=address,undefined -fno-sanitize-recover=all.

Notes

Found while working on #2732, which bounded the same defect class — a JSON parser running with null_terminated = true over a buffer that has no terminator — for the RPC registry. This one is reached through a different door.

#2732 has since merged and this branch is rebased onto it; the overlap in glz::read was two additive hunks. Re-verified on the new base: the repro above still reproduces against main at 310f270, so #2732 did not incidentally fix this.

One interaction the rebase creates: the helpers this PR newly routes to read_streaming now inherit #2732's absent-value check, so read_jsonc on a stream holding only whitespace reports no_read_input instead of succeeding with the destination untouched. That is #2732's intended behavior reaching the callers this PR connects to it.

Three limitations are left alone:

• read_streaming ignores validate_trailing_whitespace; that block lives only in the buffered read.
• Leading whitespace wider than the window stops the parse short of a value that is really there, because nothing refills during the top-level whitespace skip.
• Truncated JSON read through read_streaming surfaces end_reached, which context.hpp documents as "a non-error code". It reaches the caller as one because the depth is non-zero, so finalize_read_context does not clear it. Pre-existing and unrelated to this change.

Giving the other readers real refill points is the follow-up this makes possible, and is deliberately not attempted here.