You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: accept an org's Trino catalog name as a logical catalog alias (#1177)
* feat: accept an org's Trino catalog name as a logical catalog alias
A Duckgres session may now connect with `database=org_<database_name>` —
the same catalog name the org has on Trino — and get the physical DuckLake
catalog under that name. current_database(), pg_database,
information_schema, three-part references, and `USE` all answer to it.
The point is SQLMesh: it sees ONE catalog name on both the Duckgres and
Trino engines, so moving a project between engines needs no state rewrite.
PR #651's invariant is preserved. The startup `database` is still never
used to find, select, or route to an org. Identity stays SNI-only, and the
alias is validated AGAINST the org SNI has already resolved — it is
compared to that org's own catalog name, never used as a lookup key. A
sibling tenant's catalog name is just another unrecognized string and
fails closed, exactly as today. Sessions connecting with "ducklake" or
nothing are unchanged; the alias is opt-in per connection.
The name derivation moves to configstore (untagged) so every build has it;
provisioner.TrinoCatalogName now delegates to that one definition.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
* test(e2e): assert the logical catalog alias against mw-dev
logical_catalog_alias connects the cnpg tenant with its own Trino catalog
name and asserts the alias renames without redirecting: current_database()
and pg_database report it, a three-part reference and `USE <alias>` reach
the real catalog, and a session connected the ordinary way sees the same
row. It also asserts the security half — a sibling tenant's catalog name
and an arbitrary name both get 3D000.
Docs: CLAUDE.md gains the alias contract and the PR #651 invariant that
governs it; the harness path references are corrected to the directory
that exists (tests/mw-dev/e2e/, not tests/e2e-mw-dev/).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
* refactor: drop two unreachable guards in the alias checks
`requestedCatalog != ""` cannot be false where it sits — an empty startup
database already set CatalogValid — and `c.database != ""` is subsumed by
the EqualFold against a non-empty USE target.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
* fix: restore the empty-database guard in namesDuckLakeCatalog
dcb043d dropped `c.database != ""` on the claim that it was "subsumed by
the EqualFold against a non-empty USE target". That claim was wrong: the
USE target is only checked for emptiness BEFORE quote-stripping, so `USE ""`
passes that check as the two-character `""` and then unquotes to "". A
session whose database is unset then matched its own empty name, and invalid
SQL was silently rewritten into `USE ducklake.main` instead of passing
through to DuckDB and erroring. No cross-tenant exposure — it resolves to
the catalog the session already executes against — but an invalid statement
must not silently succeed.
The other guard that commit dropped (`requestedCatalog != ""` in store.go)
is genuinely unreachable and stays dropped.
TestRewriteDirectQueryEmptyQuotedUseIsNotAnAlias pins it at the rewrite
entry point, plus an `USE ""` case in the alias table.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
* fix(e2e): send USE as its own message in the logical-alias assertion
The e2e assertion batched `USE <alias>; SELECT ...` into one psql -c, which
duckgres cannot split. handleQuery splits a multi-statement simple query only
when pg_query parses it (conn.go: `parseErr == nil && len(tree.Stmts) > 1`),
and `USE` is not PostgreSQL syntax — so the batch reached rewriteDirectQuery
whole, its USE target was `<alias>; SELECT ...` (matching no catalog name, so
correctly left alone), and DuckDB split it and failed the bare USE.
The product code was right; the assertion was written in a shape the simple
query protocol does not support here. `USE ducklake; SELECT ...` fails the
same way on main, so this is pre-existing and not alias-specific.
pg_script feeds a script on stdin, where psql sends each statement as its own
simple-query message on ONE session. Verified against a local standalone
server: state set by the first statement is visible to the third, on one pid.
Tests: TestLogicalCatalogAliasThroughConnectionSetup drives the alias through
the REAL setup path — NewClientConn plus the exported setters control.go
calls, in order, including the post-worker-switch replay — for both the simple
and the extended query composition. That closes the gap that let this reach
CI: the old tests hand-built a clientConn and never exercised how the session
fields get populated. TestUseStatementIsNeverSplitOutOfASimpleQueryBatch pins
the batching limitation so the next harness author does not rediscover it, and
fails loudly if pg_query ever learns to parse USE.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
* docs: record the catalog-qualified SET search_path gap
`SET search_path = '<alias>.main'` is not rewritten and fails on the worker,
while the physical `'ducklake.main'` works. The catalog name sits in a string
literal rather than a RangeVar, so LogicalCatalogTransform has nothing to
match and the USE pass does not look at SET. Deliberately left alone: SQLMesh
selects a catalog with `USE <catalog>` as its own statement, and touches
search_path only in dbt code it marks unsupported.
Also record in the harness that project_reader_isolation already issues its
USE as a separate message (psql -c/-c) and is not a latent failure, and that
the retry case list now has four copies worth folding together later.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: CLAUDE.md
+36-3Lines changed: 36 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -180,7 +180,7 @@ The project uses [just](https://github.com/casey/just) as a command runner. Run
180
180
181
181
**Every feature, behavior change, bugfix, AND refactor that affects runtime or
182
182
cluster behavior MUST ship with a solid end-to-end test case in
183
-
`tests/e2e-mw-dev/` (`harness.sh`).** This is not just for new features — any
183
+
`tests/mw-dev/e2e/` (`harness.sh`).** This is not just for new features — any
184
184
change to how the system behaves at runtime (new capability, changed semantics,
185
185
a fixed bug, a new config knob, an activation/routing/teardown tweak) extends or
186
186
adds a harness assertion in the same PR. Refactors count too: when you move or
@@ -202,11 +202,11 @@ Three test lanes worth knowing about, in increasing order of blast radius:
202
202
203
203
-**Unit / package tests** (`go test ./...`): in-process, no external deps. Where most coverage lives. Includes `tests/manifests/` (static-manifest artifact asserts for `k8s/rbac.yaml` + `k8s/networkpolicy.yaml`).
204
204
-**`tests/integration/`** (`just test-integration`): spins up the standalone server binary against a real MinIO + Postgres metadata store via docker compose. Covers wire protocol, DuckLake on real S3-compatible storage, transpilation against a live server.
205
-
-**`tests/e2e-mw-dev/`** (per-PR GitHub workflow `e2e-mw-dev.yml`): the full multi-tenant activation pipeline against the **real posthog-mw-dev EKS cluster** — real Cilium, real Crossplane ducklings, real cnpg-shard + external-RDS metadata, real AWS S3. A shell harness (`harness.sh`) runs as an in-cluster Job per PR; `run.sh` orchestrates deploy/test/teardown/e2e-cleanup. **Replaces the retired kind suite** (`tests/k8s/`) — that suite's `k8s-integration-tests` CI job and its Go tests are gone; the supporting `k8s/` scripts/manifests + Dockerfiles are kept for now. See `tests/e2e-mw-dev/README.md`.
205
+
-**`tests/mw-dev/e2e/`** (per-PR GitHub workflow `e2e-mw-dev.yml`): the full multi-tenant activation pipeline against the **real posthog-mw-dev EKS cluster** — real Cilium, real Crossplane ducklings, real cnpg-shard + external-RDS metadata, real AWS S3. A shell harness (`harness.sh`) runs as an in-cluster Job per PR; `run.sh` orchestrates deploy/test/teardown/e2e-cleanup. **Replaces the retired kind suite** (`tests/k8s/`) — that suite's `k8s-integration-tests` CI job and its Go tests are gone; the supporting `k8s/` scripts/manifests + Dockerfiles are kept for now. See `tests/mw-dev/README.md`.
206
206
207
207
### When code changes obligate test changes
208
208
209
-
`tests/e2e-mw-dev/` is the only place we exercise the full activation pipeline (control plane → STS broker → worker pod → DuckDB → ATTACH against real cloud storage). If your change touches any of the following, treat updating the harness as part of the change, not a follow-up:
209
+
`tests/mw-dev/e2e/` is the only place we exercise the full activation pipeline (control plane → STS broker → worker pod → DuckDB → ATTACH against real cloud storage). If your change touches any of the following, treat updating the harness as part of the change, not a follow-up:
210
210
211
211
-`controlplane/shared_worker_activator.go`, `controlplane/sts_broker.go`, anything in the activation payload shape (`TenantActivationPayload`, `server.DuckLakeConfig`)
0 commit comments