Derive the streamable operator surface from the MEOS catalog - #199
Merged
estebanzimanyi merged 1 commit intoSep 3, 2026
Merged
Conversation
`build_descriptor.py` reads `meos-idl.json` and classifies every public MEOS function into exactly one bucket: an operator the generator emits, a gap it owes, a structural residue, or a deferred category. The partition is committed as `tools/codegen/coverage-ledger.txt`, so a function MEOS adds cannot be dropped in silence -- it appears as a new row and the diff shows it. 814 operators are generated and 1571 remain owed. The generator states no MEOS fact of its own, and `check_no_hand_meos_facts.py` holds it to that: a module-level table may not be keyed by, or hold, a MEOS type or symbol name. The catalog says which header declares a symbol, which parser and constructor a type carries, what a function returns and what its parameters are, so a shape reads those rather than repeating them. What remains hand-written is named in the checker with the reason it belongs to NebulaStream rather than to MEOS -- the Nautilus type enum, the decomposition of a value into stream columns -- plus one outstanding table the checker reports on every run. An operand is built through the constructor the catalog names for its type, and a static value is parsed by the `_in` its `typeEncodings` entry names, so every type MEOS describes is answered by one shape instead of the one type a shape could spell. A type the catalog does not describe unambiguously declines rather than guess: `Span` encodes as `bigintspan_in`, one variety among several, and a `Span *` argument does not say which. A symbol declared only in an internal header is classified as residue before any shape sees it, because an operator reaches its MEOS entry through an umbrella header and no consumer can include the other kind. Each emitted operator is NebulaStream scaffolding around one lambda that calls MEOS, and that lambda compiles against the MEOS headers alone. All 814 of them do, with 0 errors and 0 warnings, against MEOS built from the master commit the catalog is read at. The Nix build works with `NES_ENABLE_MEOS=OFF` and with a stock paho-mqtt-cpp.
estebanzimanyi
force-pushed
the
codegen/catalog-first-ledger
branch
from
September 3, 2026 17:09
90ea854 to
7541478
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
build_descriptor.pyreadsmeos-idl.jsonand classifies every publicMEOS function into exactly one bucket: an operator the generator emits, a
gap it owes, a structural residue, or a deferred category. The partition
is committed as
tools/codegen/coverage-ledger.txt, so a function MEOSadds cannot be dropped in silence -- it appears as a new row and the diff
shows it. 814 operators are generated and 1571 remain owed.
The generator states no MEOS fact of its own, and
check_no_hand_meos_facts.pyholds it to that: a module-level table maynot be keyed by, or hold, a MEOS type or symbol name. The catalog says
which header declares a symbol, which parser and constructor a type
carries, what a function returns and what its parameters are, so a shape
reads those rather than repeating them. What remains hand-written is
named in the checker with the reason it belongs to NebulaStream rather
than to MEOS -- the Nautilus type enum, the decomposition of a value into
stream columns -- plus one outstanding table the checker reports on every
run.
An operand is built through the constructor the catalog names for its
type, and a static value is parsed by the
_initstypeEncodingsentrynames, so every type MEOS describes is answered by one shape instead of
the one type a shape could spell. A type the catalog does not describe
unambiguously declines rather than guess:
Spanencodes asbigintspan_in, one variety among several, and aSpan *argument doesnot say which.
A symbol declared only in an internal header is classified as residue
before any shape sees it, because an operator reaches its MEOS entry
through an umbrella header and no consumer can include the other kind.
Each emitted operator is NebulaStream scaffolding around one lambda that
calls MEOS, and that lambda compiles against the MEOS headers alone. All
814 of them do, with 0 errors and 0 warnings, against MEOS built from the
master commit the catalog is read at.
The Nix build works with
NES_ENABLE_MEOS=OFFand with a stockpaho-mqtt-cpp.