Conversation
|
Important Review skippedToo many files! This PR contains 804 files, which is 654 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (804)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…-author operators and keep eql_v3 surface
… client-side config round-trip
…inks to client-side config
…ts, permissions, getting started, versioning)
…solete documentation-inventory.md
The doc migration predated the version() port, so version() was documented
nowhere and docker/README still referenced the removed eql_v2 surface.
- README.md: add a 'SELECT eql_v3.version()' check to the Versioning section.
- docker/README.md: repoint the intro prose ('eql_v2 schema') and the version
example onto eql_v3 (the file was missed by the migration).
- docs_v3_grep.sh: add docker/README.md to the Tier-1 gate so it stays
eql_v2-free.
eql_v3.version() ships from the remove-eql-v2 branch.
…urface Replace the removed eql_v2 architecture (eql_v2_encrypted composite, config table, old src/ layout, db-side index config) with the v3 model: per-scalar eql_v3.<T> domains, the catalog/codegen build, functional indexes on extractors, and client-side configuration.
…md links - json-support: fix ste_vec ORE leaf term key ocv/ocf -> oc (matches src/v3/jsonb) - releasing-an-alpha: correct artifact model to the two real release artifacts (cipherstash-encrypt.sql is the self-contained eql_v3 surface) + docs bundle - sql-documentation-templates: drop dead @see add_search_config; AS [base_type] -> AS jsonb (no domain-over-domain footgun) - repoint inbound PAYLOAD.md links (docs index, eql-functions, proxy-configuration) to crates/eql-types (canonical wire types) + json-support.md - WHY: markdown spacing fix
…ng docs Correct the remaining "eql_v2 coexists / is the unchanged public API" framing to match reality: eql_v2 was removed in 3.0.0 and eql_v3 is the sole shipped surface. - DEVELOPMENT.md: replace the coexistence "Schemas" section with an accurate "The eql_v3 surface" section (v2 removed, fork-provenance/historical mentions flagged as deliberate); fix TOC anchor. - CLAUDE.md: fix the Schema bullet and Versioning paragraph that still asserted eql_v2 coexists/unchanged. - SUPABASE.md: reword the two remaining v2 callouts to teach the v3 way with no v2 references (a separate v2->v3 migration guide can come later). - docs/development/reference-sync-rules.md: update stale eql_v2 examples to verified v3 symbols (eql_v3.ciphertext, eql_v3.eq_term). Remove obsolete/invented artifacts and dead v2 fixtures: - delete docs/decisions/0001-remove-eql-v2.md and docs/plans/add-doxygen-sql-comments-plan.md; strip the 3 dangling ADR links from CHANGELOG (entries already explain the why inline). - delete tests/ORE_FIXTURES.md (documented the removed v2 proxy fixture flow; v3 ORE coverage is subsumed by catalog-generated scalar fixtures) and the orphaned tests/ore.sql / tests/ore_text.sql data files (loaded by nothing). Guard: refactor tasks/test/docs_v3_grep.sh from a hand-maintained allowlist to scan-all-with-exclusions over git-tracked docs (new reference/tutorial/concept pages covered automatically; untracked scratch ignored), and wire it into the docs-static CI job (runs on every PR, already in ci-required).
Migrate user-facing docs to the eql_v3
Re-verify the eql_v3 implementation audit against the current tree and correct drift introduced by the 3.0.0 overhaul (single self-contained installer; eql_v2 removed): - §1/§6: eql_v2 is removed, not 'the documented public API, unchanged'; reframe the ORE-comparator note as historical fork provenance. - §5: replace the 4-variant build table with the single-artifact build; re-cite build.sh and rename pin_search_path.sql -> pin_search_path_v3.sql. - §3.3: drop stale 'Supabase variant' exclusion; explain the functional- index rationale that made the subset build redundant. - §4.3: v3_ste_vec.sql is now generated/gitignored, not a committed fixture. - §2.4: fix drifted scalar_domains.rs / spec.rs line citations. De-enumerate snapshot baseline counts so they can't rot: CLAUDE.md and audit §4.2 now point at tests/sqlx/snapshots/README.md as the source of truth, and that README now documents matrix_jsonb_entry_tests.txt.
…ng comment The `eql_v3.json` domain flattens to native `jsonb` when an operator's RHS is an unknown-typed literal, so a bare `col -> 'sel'` binds the NATIVE `jsonb -> text` (a root-key lookup on the envelope) instead of the v3 selector-lookup operator — a silent wrong answer for direct-SQL callers (the Proxy is unaffected because it always sends typed `$n`). This is intrinsic to the domain type-kind and cannot be closed by an extra operator/blocker; it can only be pinned. - Re-land v3_jsonb_bare_operand_flattens_to_native (blocker face: `?` / `||` succeed as native on a bare RHS, raise on a typed RHS), recovered from the dropped commit 817a9660. - Add v3_jsonb_arrow_bare_operand_flattens_to_native (supported-operator face): assert via pg_typeof which operator binds AND the user-visible value divergence, so a resolution change in either direction goes red. Verified empirically against PG 17 (bare `-> 'sv'` -> jsonb `[]`; typed `-> 'sv'::text` -> eql_v3.ste_vec_entry NULL). - Regenerate snapshots/v3_jsonb_tests.txt (74 -> 76) for test:v3-jsonb:inventory. - Fix the inline comment in operators.sql (integer `->` overload): it claimed a bare `e -> 'sv'` would bind the v3 operator — empirically false and contrary to the file's own @warning. Bare binds native; the custom operator does not capture an untyped literal. Comment-only, no behaviour change.
…allthrough test(eql-v3): pin ->/->> bare-literal domain-flattening; fix misleading comment
Encrypting the empty string "" as ordered text produces an empty ORE term (ob: []) -- the only value that does. The ORE-bearing eql_v3 domains (_ord / _ord_ore, and text _search) now carry a CHECK requiring ob to be a non-empty array, so casting or inserting an empty-ob payload into an ordered column fails loudly with a check violation (23514) rather than producing an unorderable row. The rule lives where term behaviour belongs (CLAUDE.md): a typed per-term property Term::nonempty_array_key (Ore => "ob"), collected by Term::nonempty_array_keys symmetric to term_json_keys, with unit tests. DomainBlock derives nonempty_array_keys from the domain's terms exactly like keys, and the types.sql.j2 template renders a non-empty-array CHECK per key generically -- no hardcoded ob, no ORE concept leaking into the codegen-context or template layers. Storage / equality / match / bool domains are unaffected; fixed-width scalars never produce an empty ob, so the clause is a structural invariant that is a no-op for them. The comparator's "empty sorts first" cardinality guard (02ab24b) is retained as defense-in-depth for any path that bypasses the domain (e.g. a composite built directly). - crates/eql-scalars: Term::nonempty_array_key{,s} + tests. - crates/eql-codegen: DomainBlock.nonempty_array_keys + template clause + unit test pinning the constraint to ORE-bearing domains only. - tests/codegen/reference/*/*_types.sql regenerated. - Rewrote v3_text_empty_order_tests -> v3_text_empty_constraint_tests: empty "" rejected on text_ord/text_ord_ore; controls accepted/ordered. - CHANGELOG + adding-a-scalar docs updated. Refs: #262
…st-rebased fix(eql_v3): empty-string ordered text sorts first instead of dropping out
The four scalar-matrix shape snapshots in tests/sqlx/snapshots/ are not independent — eq-only is derived from the ordered baseline — and were regenerated by four separate hand-run commands. A macro change touching every shape could update 3 of 4 and silently drift the fourth, exactly how the storage-only snapshot drifted in #300 when a count_distinct dispatch arm switched to a runtime early-return. Add test:matrix:snapshots:regen, which lists the encrypted_domain binary once and rewrites all four shapes (ordered, eq-only, text, storage-only) in dependency order. Point snapshots/README.md at the single command. The inventory gate (test:matrix:inventory) is unchanged and remains the validator; CI still checks via git diff. Regen is deterministic — on the current branch it reproduces the committed snapshots byte-for-byte. Closes #300.
…o place CodeRabbit flagged that writing snapshots/ directly is unsafe: the '>' redirect truncates the committed file before the pipeline's grep runs, so a driver type whose matrix arms vanished would leave a truncated baseline behind — defeating the task's atomicity guarantee. Build all four shapes into a mktemp dir, assert each is non-empty, then mv them into place together, so any failure leaves the committed snapshots untouched. Add an explicit non-empty guard on the test listing for an early, clear message. Avoid a competing EXIT trap (stub-fixtures.sh owns it) by removing the temp dir explicitly. Verified: regen reproduces the four snapshots byte-for-byte and test:matrix:inventory passes.
…ot-regen test(v3): atomic regen task for the four scalar matrix snapshots (#300)
…ckstop Cover every reachable scalar_matrix! arm at the body level: int4 ([eq, ord]), text ([eq, ord, search]), bool ([storage]). text does not subsume int4 (its ord btree combo omits =, which the int4 arm proves rides the ORE ordered index), so all three are kept. Refactor test:matrix:expand to loop over TARGETS=(int4 text bool), emitting snapshots/<token>_expanded.rs per target (fixture/migration normalization generalized, bash-3.2 safe). int4_expanded.rs is also refreshed: its embedded matrix.rs line markers were stale (2-line shift) since a7b29b5 changed matrix.rs without a regen; the non-blocking nightly lane had not run to flag it. Body logic is unchanged (diff is start_line/end_line metadata only). Refs #321.
Pure mechanical rename (PR 1 of unified-catalog-codegen refactor): crate dir, Cargo package name, import path eql_scalars->eql_domains, all referrers, mise/task scripts, and load-bearing doc paths. No type/field/SQL changes.
Pure mechanical rename (PR 1 of unified-catalog-codegen refactor): crate dir (carrying committed bindings/+schema/ via git mv), Cargo package name, import path eql_types->eql_bindings, mise types:* tasks, and load-bearing doc paths. EQL_TYPES_SCHEMA_DIR env var name kept. No type/field/TS/JSON changes.
…hots test(v3): expand text + bool matrix body snapshots in the nightly drift lane
Three pre-existing inconsistencies in the scalar catalog crate, surfaced by code review of the crate-rename PR (they predate the rename; the rename only moved these files): - ScalarKind::rust_type() returned the SQL token "text" for Text, but the function documents itself as the canonical Rust plaintext type name and every other arm names a Rust type (i32, chrono::NaiveDate, rust_decimal::Decimal). Return "String". - proptest_invariants::any_kind() omitted ScalarKind::Bool, so the bool branch of bounded_int_ranges_are_ordered was never exercised. Add it (and fix the "ten"/"eleven" count in the doc comment). - Fixture::numeric_value()'s Int(n) arm bypassed the as_bounded_int() gate its Min/Max/Zero siblings all use, fabricating Some(n) for a hand-built literal on a non-integer kind. Gate it like the sentinels. No CATALOG row pairs Int with a non-integer kind, so generated output is unchanged. Adds regression tests pinning Text's rust_type and the Int(n) non-integer-kind gate. cargo test -p eql-domains: 76 passed.
Three more pre-existing items from the crate-rename code review: - export.rs (dump_v3_json_schemas): clear schema/v3 before regenerating so a JSON file for a domain later removed from the catalog cannot linger as a stale checked-in artifact. The dir holds only this test's output, so recreating it from scratch is safe; with the catalog unchanged the regen is byte-identical. - v3_conformance.rs (int4_ord_rejects_missing_ore_term): drop the stray `hm` key from the fixture. `hm` is not an Int4Ord field, so under deny_unknown_fields the rejection could pass for the wrong reason; the payload now carries only the base envelope, so the sole cause of failure is the missing `ob`. - tests/codegen/reference/README.md: timestamptz is no longer equality-only (promoted to ordered, native 12-block ORE). Replace the stale example with an accurate shape-divergence one (bool is storage-only). cargo test -p eql-bindings: 11 passed. clippy clean. No schema/v3 drift.
The scalar domains are 100% generated from eql_domains::CATALOG, so read the domain/variant matrix straight from the source of truth via `eql-codegen dump-catalog` (which already emits JSON) instead of parsing the generated *_types.sql. No drift from SQL-format changes, authoritative type tokens (integer/bigint/…, not int4/int8), and the exact SQL operators each domain supports (new `supportedOperators` field). More accurate, too: `_ord` domains now correctly report equality + order — ORE comparison collapses to equality, so `=`/`<>` are supported — which the CHECK-key derivation missed. json.sh emits docs/api/json/eql-catalog.json (cargo run -p eql-codegen dump-catalog) and passes it to the converter. Covers the 48 catalog (scalar) domains; the 3 hand-written jsonb domains (json/jsonb_entry/jsonb_query) are not in the catalog and remain a follow-up. Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
…ifest Resolves #365 and #366 — both via the catalog dump, no SQL autogen. eql_domains::CATALOG (Shape::SteVec), but their SQL is deliberately hand-written (hand-tuned CHECKs, #354) and `scalar_families()` filters them out of the dump. Add them as a new **additive** `stevec` field on `CatalogDump` — scalar-only consumers (the fixture-coverage task) read only `types[]`, so they're unaffected. The manifest now carries all 51 domains (48 scalar + 3 jsonb). from eql_domains::Term) to each `DomainEntry`, linking a domain to its extractor functions (e.g. integer_ord -> eql_v3.ord_term). Authoritative — resolves the docs drift-lint false-flags on eq_term / ord_term / match_term. Both fields are additive; all eql-codegen + eql-domains tests pass (89 + 91). The manifest generator maps them through (termFunctions on scalars; the jsonb family as `json`-capability domains). Full pipeline verified end-to-end (doxygen -> XML + dump-catalog -> 984 functions + 51 domains). Stacked on #364. Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
docs: emit a JSON API manifest from the Doxygen XML (EQL v3)
Per @tobyhede's review on #368: - dump.rs doc comments referenced `eql_v3.json` / "eql_v3-relative" — the jsonb domains resolve under `public` (only the extractor *functions* are eql_v3). Corrected the CatalogDump.stevec and SteVecEntry.full_name docs. - The catalog models no per-SteVec-entry terms (JSONB_DOMAINS carry `terms: &[]`, enforced by `shape_and_terms_are_consistent`), so `term_infos(d.terms)` was provably empty and the searchable SteVec family rendered inert (no termFunctions). Hardcode the real hand-written ste_vec extractors for now: `hm` -> eq_term, `oc` -> ore_cllw (src/v3/jsonb/operators.sql). The manifest now links public.json / jsonb_entry / jsonb_query to eql_v3.eq_term / eql_v3.ore_cllw. Operators for the SteVec family remain a follow-up (DB introspection of pg_operator is the reliable source). Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
The inline TermInfo { .. } literals added in the review-response commit
tripped cargo fmt --check (Rust workspace crates CI). Expand them to the
one-field-per-line form rustfmt wants.
Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
The hardcoded hm/oc terms were stamped onto all three SteVec domains, but
per src/v3/jsonb/{functions,operators}.sql the eq_term/ore_cllw extractors
take public.jsonb_entry (the sv *element* type) exclusively:
- eq_term(jsonb_entry) -> = / <> (coalesce(hm, oc))
- ore_cllw(jsonb_entry) -> < <= > >= (oc)
The public.json container and public.jsonb_query domains carry no term
extractors — their surface is containment (@>, <@) and path navigation.
stevec_terms() now keys on the catalog domain name and returns terms only
for `entry`; json/query resolve to []. Tests assert the per-domain split.
Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
The Markdown/JSON API extraction runs Doxygen's C++ parser over raw SQL, which silently dropped functions and misclassified visibility: - `::type` casts in `LANGUAGE sql` bodies read as C++ `::` scope resolution and dropped the whole enclosing CREATE FUNCTION memberdef — losing jsonb_path_query / jsonb_path_query_first and ~hundreds of generated eql_v3_internal extractor overloads. - Regular `-- …` SQL comments parsed as code, minting phantom functions (overloads, extractor, queries, accessor, inlinable). - `is_private` keyed on a leading `_` in the function NAME, but internal functions live in the `eql_v3_internal` schema, so none were flagged (984 functions, all reported public). - `@example` in version.sql detached its doc comment (dropping `version` entirely); and `*.template` sources were documented alongside their generated `.sql`, duplicating `version` with $RELEASE_VERSION placeholder text. Fixes: - doxygen-filter.sh: strip `$$…$$` bodies and neutralize `--` comments so Doxygen sees only clean declarations + `//!` doc comments. - xml-to-markdown.py: treat the `eql_v3_internal` schema as private. - Doxyfile: document only `*.sql` (drop `*.template`; version.sql is build-generated from the template before Doxygen runs). - version.template: replace `@example` with plain prose. Result: manifest 984 → 1680 functions (986 public, 694 private, 0 false-positive private); version + jsonb_path_query(_first) now present; phantom functions gone. Domains unchanged (catalog-sourced). All doc-gen tests + SQL coverage validation pass. Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
Address Copilot review on #369: - test_xml_to_markdown.py: add test_internal_schema_is_private, exercising process_function directly — asserts eql_v3_internal.* -> private and eql_v3.* -> public, so the schema-based visibility can't silently regress to the old leading-underscore heuristic. - doxygen-filter.sh: set -euo pipefail + explicit one-arg check, so a stray invocation fails fast (exit 2) instead of awk blocking on stdin. Claude-Session: https://claude.ai/code/session_01CqDNqLSEEkCi7xAJFq7HJA
…rivate fix(docs): repair Doxygen SQL extraction — coverage gaps + private split
Client-side half of the EQL v3 query-term surface: the enveloped,
per-capability query operand `{v, i, <terms>}` (envelope minus the
ciphertext `c`) for every term-bearing scalar domain, plus its
conversion from the v2 payload.
Generator (eql-codegen/src/bindings.rs):
- render_query_struct: a `<Name>Query` twin per term-bearing domain =
the storage struct minus `c`, on `public.<name>_query`, with
deny_unknown_fields enforcing the no-`c` contract. Storage-only
domains (no operators) get no twin.
- render_query_payload_rs: `QueryPayload` is now catalog-generated (one
variant per twin + the SteVec needle), superseding the hand-written
single-variant enum — enveloped + per-capability makes it
catalog-per-domain.
- all_query(): a separate query inventory, kept OUT of all() so query
domains never resolve as stored from_v2 conversion targets.
Bindings (eql-bindings, regenerated + hand-written):
- Regenerated family files + query_payload.rs + inventory.rs.
- 38 TypeScript bindings (bindings/v3/*Query.ts) + 38 JSON Schemas
(schema/v3/*_query.json); export.rs chains all_query().
- from_v2::convert_scalar_query hoists the v2 payload's required terms
into `{v:3, i, <terms>}` (drops c/k; bf reinterpreted to smallint[]);
both query entry points route through QueryPayload::parse per target;
storage-only scalars stay UnsupportedQueryTarget.
Verified: `mise run test:crates` (fmt + clippy -D warnings + tests) green.
SQL surface (public.<name>_query domains + consuming operators/functions)
and sqlx conformance are NOT in this commit — they need Postgres + CS
creds to validate. See the PR description.
CIP-3432
…432)
The SQL half, part 1: a `public.<name>_query` domain per term-bearing
scalar domain — the index-terms-only twin (`{v, i, <terms>}`) that query
operators will consume. CHECK asserts the envelope-minus-`c` + terms and
FORBIDS `c` (a query operand carries no ciphertext), mirroring the Rust
binding's deny_unknown_fields.
- context.rs: DomainBlock gains `forbidden_keys`; `query_domain_block`
builds the twin (keys = v/i + terms, forbidden = [c]).
- templates/query_types.sql.j2: idempotent CREATE DOMAIN with the
no-`c` CHECK.
- generate.rs: `render_query_types_file` → `<T>_query_types.sql`, wired
into `render_type` for families with any term-bearing domain.
Validated: `mise run build` assembles; the full surface (38 query
domains) installs cleanly into a fresh PG (`test:clean_install_v3`); and
the CHECK semantics verified directly — `{v,i,hm}` accepted, a payload
with `c` or a missing term rejected.
Still to do: query extractors + wrappers + operators binding
(storage_domain, <name>_query), and sqlx conformance (needs CS creds).
CIP-3432
…ins (CIP-3432)
The SQL half, part 2: per term-bearing domain, a `<name>_query_functions.sql`
+ `<name>_query_operators.sql` giving query operands a public SQL entry point.
- `render_query_functions_file`: query-operand extractor OVERLOADS (the same
eq_term/ord_term, on `public.<name>_query`) + comparison WRAPPERS binding
`(storage, query)` and its `(query, storage)` commutator — supported
operators only, `extractor(a) <op> extractor(b)`, no ciphertext cast.
- `render_query_operators_file`: `CREATE OPERATOR` for each, both directions,
reusing the existing planner metadata (COMMUTATOR/NEGATOR/RESTRICT/JOIN).
- Wired into `render_type` for term-bearing domains; reuses the storage
`functions.sql` / `operators.sql` templates unchanged.
Semantically validated against REAL fixture ciphertext (local PG): a
term-only operand `{v,i,hm}` (no `c`) matches the stored row through
`= (public.integer_eq, public.integer_eq_query)` — self-match hit all 17
integer fixture rows, and a plaintext=0 operand matched exactly the
plaintext=0 row. Full surface installs clean (`test:clean_install_v3`).
Next: sqlx conformance test with fresh in-test encryption (increment 6).
CIP-3432
…(CIP-3432)
End-to-end proof of the query-operand surface with FRESH ZeroKMS
encryption for both stored values and the query value (gated behind
`proptest-e2e`, like the rest of the fresh-encryption suite):
- eq: a term-only operand `{v,i,hm}` (no `c`), INDEPENDENTLY encrypted,
matches exactly the equal stored rows via `= (integer_eq,
integer_eq_query)`; a never-stored value matches nothing.
- ord: a term-only ORE operand orders correctly (`< 25` → the rows below
25), and the `(query, storage)` commutator direction resolves too.
- security: a ciphertext-bearing operand (full storage payload) is
rejected by the `<name>_query` domain's no-`c` CHECK.
Verified locally against Postgres 17: 3 passed.
CIP-3432
…en (CIP-3432) Regenerated eql_v3_public_surface.txt — 423 additions, 0 removals, all `*_query` wrappers/extractors. Confirms the query surface is purely additive (no existing operator changed).
…CIP-3432) Extends the shared oracle engine so EVERY generated `<name>_query` domain is exercised against real ciphertext, across all scalar types — folded into existing round trips (no added DB load, no new flakiness): - assert_eq_oracle / assert_ord_oracle: each all-pairs SELECT now also drives the term-only query operand (payload minus `c`) through the `(storage, <name>_query)` operators, both directions. Covers <T>_eq_query, <T>_ord_query, <T>_ord_ore_query for all 9 families in BOTH the fixture and e2e (fresh-encryption) suites. - Overload::DomainQuery: a fourth named-function overload (RHS = <domain>_query), reaching text_search_query, which the operator oracle (text runs via _eq/_ord/_ord_ore) never touches. - assert_match_smoke: four query-operand containment rows cover text_match_query (`contains`/`contained_by` with a bloom needle). - ope_ord_fixture_smoke!: each range/equality predicate also runs against the term-only operand, covering <T>_ord_ope_query for all families. jsonb_query already has semantic coverage (v3_jsonb_tests D4). With the standalone fresh-encryption conformance, all 38 scalar query domains + jsonb_query are now tested. Full fixture oracle suite: 42 passed. CIP-3432
…-3432) The new (storage, <name>_query) + commutator operators add 2 arg shapes per operator, so every term-bearing domain now has ops×5 operators (3 storage + 2 query), not ops×3. CI's SQLx shards caught this. Verified: planner_metadata 48/48 pass.
…te (CIP-3432) - generate.rs: comment in render_query_functions_file explaining why query twins emit no blockers (the realistic col<op>operand path is covered by the storage domain's jsonb blockers; blocking operand<op>operand would be the full matrix for zero real-world coverage). [review finding #1] - query_types.sql.j2: @note that query operands must be cast to their _query domain in a predicate (uncast literal RHS is ambiguous with the jsonb overload). [review finding #3]
… non-superuser) CREATE OPERATOR FAMILY / CLASS require superuser, so the single-transaction eql_v3 installer aborted at the first CREATE OPERATOR FAMILY on Supabase and other managed Postgres (SQLSTATE 42501, "must be superuser"), leaving eql_v3 uninstallable there despite the surface being otherwise managed-Postgres compatible. Wrap both src/v3/sem/*/operator_class.sql family+class creations in a DO block that catches insufficient_privilege and continues with a NOTICE. One artifact now installs everywhere: superuser installs create the default btree opclass as before (self-managed PG, SQLx matrix); non-superuser installs skip it and fall back to the OPE ordering domains, whose extractor return types carry a native btree opclass. Non-privilege errors still propagate. Also removes stale in-file comments claiming these files were excluded by a `**/*operator_class.sql` build glob — the v3 build globs src/v3 wholesale. Verified: live Supabase (both skipped, install commits, 0 opclasses, 36 OPE domains) and a local superuser cluster (both opclasses created).
The v3 encrypted domains are jsonb-backed, so introspection that resolves a domain to its base type (e.g. Supabase's table editor via postgres-meta) shows them as bare `jsonb` with no hint they are EQL encrypted columns. Attach a one-line COMMENT ON DOMAIN to every public encrypted domain so the type is self-documenting: visible via psql \dD, obj_description(), and any tool that reads pg_type comments (Supabase's types introspection surfaces it). Scalar domains: the comment is code-generated. A new DomainBlock.comment field derives capability text from the domain's terms (Term::operators_for_terms), so it tracks the generated CHECK/operator surface and can't drift; the DO-block templates emit COMMENT ON DOMAIN after each idempotent CREATE DOMAIN (re-applied on reinstall so comment-text changes propagate). Query-operand (_query) twins get a matching 'index terms only; no ciphertext' comment. The three hand-written jsonb SteVec domains (json / jsonb_entry / jsonb_query) get hand-written comments. No behaviour change; comments only. Generated SQL regenerated in place.
The first cut wrapped to ~3 lines in Supabase Studio's type picker. Drop the 'jsonb-backed CipherStash searchable-encryption domain.' boilerplate and compress capability to plain words (equality / ordering / containment / storage only), derived from the operator set. E.g. 'EQL encrypted numeric (equality, ordering)'. Longest is now 62 chars vs ~110.
feat(v3): COMMENT ON DOMAIN for every encrypted domain type
…operator-class fix(v3): install SEM btree operator classes conditionally (Supabase / non-superuser)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Long-running integration branch for the EQL v3 major release. Tracks
eql_v3againstmain.v3 introduces a new, additional
eql_v3schema of type-safe, per-capability encrypted-domain types. The existingeql_v2surface is unchanged and remains the documented public API.Scalar encrypted-domain families
Per-type jsonb-backed domains, each with
_eq(=/<>via HMAC) and_ord/_ord_ore(<<=>>=,MIN/MAXvia ORE block terms) variants, plus a storage-only base domain. Supported operators resolve to inlinable wrappers; nativejsonboperators reachable via domain fallback are blocked rather than silently mis-resolving.int2,int4,int8(feat(encrypted-domain): add int2 scalar domain family #243, feat: eql_v2_int4 variant family (v3) #239, feat(int8): add eql_v3.int8 encrypted-domain type family #253)date,timestamptz(UTC-normalized),numeric— ordered, ORE wider than 8 blocks (feat(scalars): add eql_v3.date encrypted-domain type + temporal wiring #256, feat(scalars): add eql_v3.timestamptz encrypted-domain type (equality-only) #257, Generalize ORE block comparator to N blocks (not hardcoded 8) for wide-term ordering (decimal/timestamp) #241/feat(v3): N-block ORE comparator + ordered numeric & timestamptz #276)float4/float8— ordered (feat(v3): float4/float8 encrypted-domain types #299)bool— storage/encryption-only (feat(eql_v3): addboolstorage-only encrypted-domain type #295)text—text_eq,text_match,text_ord,text_search(feat(scalars): add eql_v3.text encrypted-domain family (eq / match / ord) #260)Encrypted-JSONB (SteVec) document type
Self-contained encrypted-JSONB surface:
eql_v3.jsonstorage domain plusste_vec_entry/ste_vec_query. Searchable without decryption via containment (@>,<@), field/array access, entry equality, and entry-level ordered range via CLLW ORE. Comparisons are leaf-level only; root-document and native-jsonb operators are blocked. (#267)SEM index-term types
eql_v3owns its searchable-encrypted-metadata index-term types directly —hmac_256,ore_block_256,ore_cllw,bloom_filter— with noeql_v2dependency. The ORE block comparator orders ciphertexts of any block count (#241).=/<>on the v2 ORE term now declare aCOMMUTATOR(#239).Codegen
Scalar domains are generated from a Rust catalog (
eql-scalars::CATALOG) by theeql-codegenmaterializer. The Python codegen toolchain is removed. Supporting crates:eql-types(canonical payload types + JSON Schemas),eql-scalars,eql-codegen. (#252, #236, #269)Packaging
Self-contained, standalone
release/cipherstash-encrypt-v3.sqlinstaller + uninstaller, attached to GitHub Releases. (#255, #307)LOC breakdown
git diff --shortstatreports 498 files, 102,511 insertions / 32,897 deletions, but that number is dominated by machine-produced artifacts. Verified against the merge base (80a7a2bc):tests/sqlx/snapshots/cargo expandmacro snapshot + matrix baselinestests/codegen/reference/tests/sqlx/{src,tests}/crates/Cargo.locksrc/docs/tasks/+.github/+mise.tomlSummary:
eql-scalarscatalog +eql-codegenrenderers. Adding a scalar type is one catalog row plus ~one line of test wiring.Review implication: the generated buckets do not need line-by-line review — the parity/inventory tests verify them. Reviewer attention belongs on the ~5k catalog + codegen and the bespoke JSONB/SEM/ORE test suites. Full analysis:
docs/development/2026-06-24-eql-v3-pr-loc-analysis.md.