Skip to content

Derive the streamable operator surface from the MEOS catalog - #199

Merged
estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:codegen/catalog-first-ledger
Sep 3, 2026
Merged

Derive the streamable operator surface from the MEOS catalog#199
estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:codegen/catalog-first-ledger

Conversation

@estebanzimanyi

@estebanzimanyi estebanzimanyi commented Sep 3, 2026

Copy link
Copy Markdown
Member

build_descriptor.py reads meos-idl.json and classifies every public
MEOS function into exactly one bucket: an operator the generator emits, a
gap it owes, a structural residue, or a deferred category. The partition
is committed as tools/codegen/coverage-ledger.txt, so a function MEOS
adds cannot be dropped in silence -- it appears as a new row and the diff
shows it. 814 operators are generated and 1571 remain owed.

The generator states no MEOS fact of its own, and
check_no_hand_meos_facts.py holds it to that: a module-level table may
not be keyed by, or hold, a MEOS type or symbol name. The catalog says
which header declares a symbol, which parser and constructor a type
carries, what a function returns and what its parameters are, so a shape
reads those rather than repeating them. What remains hand-written is
named in the checker with the reason it belongs to NebulaStream rather
than to MEOS -- the Nautilus type enum, the decomposition of a value into
stream columns -- plus one outstanding table the checker reports on every
run.

An operand is built through the constructor the catalog names for its
type, and a static value is parsed by the _in its typeEncodings entry
names, so every type MEOS describes is answered by one shape instead of
the one type a shape could spell. A type the catalog does not describe
unambiguously declines rather than guess: Span encodes as
bigintspan_in, one variety among several, and a Span * argument does
not say which.

A symbol declared only in an internal header is classified as residue
before any shape sees it, because an operator reaches its MEOS entry
through an umbrella header and no consumer can include the other kind.

Each emitted operator is NebulaStream scaffolding around one lambda that
calls MEOS, and that lambda compiles against the MEOS headers alone. All
814 of them do, with 0 errors and 0 warnings, against MEOS built from the
master commit the catalog is read at.

The Nix build works with NES_ENABLE_MEOS=OFF and with a stock
paho-mqtt-cpp.

`build_descriptor.py` reads `meos-idl.json` and classifies every public
MEOS function into exactly one bucket: an operator the generator emits, a
gap it owes, a structural residue, or a deferred category. The partition
is committed as `tools/codegen/coverage-ledger.txt`, so a function MEOS
adds cannot be dropped in silence -- it appears as a new row and the diff
shows it. 814 operators are generated and 1571 remain owed.

The generator states no MEOS fact of its own, and
`check_no_hand_meos_facts.py` holds it to that: a module-level table may
not be keyed by, or hold, a MEOS type or symbol name. The catalog says
which header declares a symbol, which parser and constructor a type
carries, what a function returns and what its parameters are, so a shape
reads those rather than repeating them. What remains hand-written is
named in the checker with the reason it belongs to NebulaStream rather
than to MEOS -- the Nautilus type enum, the decomposition of a value into
stream columns -- plus one outstanding table the checker reports on every
run.

An operand is built through the constructor the catalog names for its
type, and a static value is parsed by the `_in` its `typeEncodings` entry
names, so every type MEOS describes is answered by one shape instead of
the one type a shape could spell. A type the catalog does not describe
unambiguously declines rather than guess: `Span` encodes as
`bigintspan_in`, one variety among several, and a `Span *` argument does
not say which.

A symbol declared only in an internal header is classified as residue
before any shape sees it, because an operator reaches its MEOS entry
through an umbrella header and no consumer can include the other kind.

Each emitted operator is NebulaStream scaffolding around one lambda that
calls MEOS, and that lambda compiles against the MEOS headers alone. All
814 of them do, with 0 errors and 0 warnings, against MEOS built from the
master commit the catalog is read at.

The Nix build works with `NES_ENABLE_MEOS=OFF` and with a stock
paho-mqtt-cpp.
@estebanzimanyi
estebanzimanyi force-pushed the codegen/catalog-first-ledger branch from 90ea854 to 7541478 Compare September 3, 2026 17:09
@estebanzimanyi estebanzimanyi changed the title Derive the operator surface from the catalog and ledger what it covers Derive the streamable operator surface from the MEOS catalog Sep 3, 2026
@estebanzimanyi
estebanzimanyi merged commit a7dd28b into MobilityDB:main Sep 3, 2026
19 of 34 checks passed
@estebanzimanyi
estebanzimanyi deleted the codegen/catalog-first-ledger branch September 3, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant