Skip to content

feat(stack): prepare slim resources on demand - #6250

Open
jgoux wants to merge 48 commits into
developfrom
juliengoux/cli-2123-stack-prepare-slim-resources-safely-on-demand-or-upfront
Open

feat(stack): prepare slim resources on demand#6250
jgoux wants to merge 48 commits into
developfrom
juliengoux/cli-2123-stack-prepare-slim-resources-safely-on-demand-or-upfront

Conversation

@jgoux

@jgoux jgoux commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • introduce package-owned off, lazy, and eager preparation policies with enabled dependency closure and coalesced work
  • prepare native slim-service archives through verified manifests, exact checksums, in-process zstd decompression, host gates, and atomic cache publication
  • use exact canonical GHCR resources and activate PostgREST lazily as the first end-to-end tracer bullet
  • select one exclusive execution mode, persist it with the managed stack, and use the selected Docker or Podman executable consistently
  • preserve typed lifecycle failures across the daemon boundary and make overlapping reload, preparation, and disposal outcomes deterministic
  • collect service logs from the selected runtime socket when usable (readable and writable), degrade safely when it is unavailable, and reject incompatible native glibc artifacts before launch

Context

This keeps resource preparation, runtime selection, and activation inside @supabase/stack, with the CLI remaining a thin consumer. An omitted mode selects Docker when Docker or Podman is usable and otherwise selects native mode. Explicit native or Docker choices are strict, preparation never falls back to the other mode, and managed stacks pin their persisted selection once claimed, including after a failed startup. Changing modes requires deleting and recreating the stack and its managed data.

Stack creation stays side-effect free beyond runtime detection and port reservation. Eager resources are prepared at startup, while lazy resources are prepared through the same activation path used by proxy and programmatic callers. Concurrent work is coalesced, disposal completes waiters with typed failures before cancellation, and cached paths cannot start services after disposal begins. Download completion restores prior public state atomically without overwriting a newer lifecycle transition. Concurrent Functions and Edge Runtime reloads preserve each committed state.

Docker and Podman remain exclusive runtime selections. Vector consumes the selected runtime socket when one is usable (readable and writable); a socket-less host uses internal Vector logs instead of crash-looping the service, and Podman never implicitly adopts the Docker default socket.

The catalog uses the frozen service versions, including Postgres 17.6.1.163, and preserves exact published container tags. The canonical Postgres image starts through its published non-root entrypoint and database bootstrap is modeled as an observable one-shot dependency before consumers become ready. Docker-only services remain container-backed until they have a concrete native runtime consumer.

@jgoux

jgoux commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

Automated review triage is complete for this batch.

Fixed:

  • socket-less Docker or Podman now uses internal Vector logs instead of crash-looping the stack
  • a Podman-selected stack no longer implicitly mounts the Docker default socket
  • download completion restores public state atomically and cannot overwrite a newer lifecycle transition
  • the stale Downloading coverage now exercises real download events, including a concurrent explicit stop

Deliberately deferred:

  • graceful stop cancelling an in-progress cold start remains a product lifecycle decision; disposal is the force-cancelling boundary
  • layer-scope single-flight alignment, version-prefix leniency, a catalog mandatory trait, and archive streaming remain outside this tracer-bullet scope

This is the final automated-fix pass for this PR. The remaining items need human product and architecture review rather than another mechanical review loop.

@jgoux

jgoux commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

Final automated-review disposition:

  • Applied usable-socket checks before Vector selects container logs, label=disable for every explicitly selected Podman socket, and an atomic Downloading projection transition.
  • Added focused coverage for inaccessible-socket degradation and an explicit Docker-compatible Podman socket override.
  • Deliberately did not add an exhaustive XDG/per-UID path matrix or warning plumbing. Those path permutations do not add meaningful user-surface confidence, and a warning belongs at a later UX boundary.
  • The uncached core run exposed a separate test-only managed-control flake: its 2-second guard left little margin over a deliberate ~1.5-second retry schedule. The guard is now 5 seconds, still well below the real 35-second parent handshake; production timing is unchanged.

This is the final automated-review pass. The remaining recorded deferrals are unchanged; the PR is ready for human review.

@jgoux
jgoux marked this pull request as ready for review August 19, 2026 21:11
@jgoux
jgoux requested a review from a team as a code owner August 19, 2026 21:11
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Supabase CLI preview

npx --yes https://pkg.pr.new/supabase/cli/supabase@cfe200427d7b32e943d87d15863058f640385660

Preview package for commit cfe2004.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

// A fully staged replacement is now available, so an incomplete
// destination can be reclaimed without risking the last usable
// cache entry. Retry publication once; persistent filesystem
// failures still surface instead of looping forever.
yield* fs.remove(cacheDir, { recursive: true, force: true });
const retry = yield* fs.rename(stagingDir, cacheDir).pipe(Effect.result);

P2 Badge Avoid deleting a concurrently published cache

When two processes repair the same incomplete cache, both can fail the initial rename and observe the destination as incomplete; if one publishes after the other's check, this unconditional removal deletes the newly complete directory that the winner may already have returned to a starting service. That produces intermittent missing executables despite successful resolution. Use a cross-process lock or a replacement protocol that cannot remove a destination after another contender has published it.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/managed/document.ts
Comment thread packages/stack/src/BinaryResolver.ts Outdated
Comment thread packages/stack/src/versions.ts Outdated
Comment thread packages/stack/src/managed/lifecycle.ts
Comment thread packages/stack/src/StackPreparation.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e8372805bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/createStack.ts Outdated
@jgoux

jgoux commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the cache-publication finding from review 4976855133 in 4d31f6c. Invalid-destination repair now takes an atomic cross-process publication claim and revalidates the destination under that claim before removing anything. Private staging and the uncontended atomic-rename path remain unchanged. A deterministic resolver integration scenario covers a stale repairer observing another resolver publish and verifies that the winner is reused rather than replaced.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4d31f6c4aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/managed/manager.ts Outdated
Comment thread packages/stack/src/BinaryResolver.ts
Comment thread packages/stack/src/BinaryResolver.ts
Comment thread apps/cli/src/next/config/stack-config.ts
Comment thread packages/stack/src/LocalStack.ts
Comment thread packages/stack/src/LocalStack.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5f1603bf1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/PortAllocator.ts Outdated
Comment thread packages/stack/src/StackConfigResolver.ts Outdated
Comment thread packages/stack/src/ServiceCatalog.ts
Comment thread packages/stack/src/services/postgres-init.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 066c466b66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/PortAllocator.ts Outdated
Comment thread packages/stack/src/supervisor.ts Outdated
Comment thread packages/stack/src/PortAllocator.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

# Check if already migrated (authenticator role created by initial-schema.sql)
if ${psql} -U supabase_admin -d postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='authenticator'" 2>/dev/null | grep -q 1; then
echo "Database already initialized, updating passwords..."
else

P1 Badge Use a completion marker after all native migrations

If initial native initialization is interrupted or a migration fails after initial-schema.sql creates the authenticator role, the next start takes this branch and skips every remaining init script and migration. The one-shot can then report success against a partially initialized database, leaving later services to fail on missing schema objects until the data directory is deleted; detect completion using a marker written only after the entire migration sequence succeeds rather than an early-created role.


const psql = `${pgBinDir}/psql -h 127.0.0.1 -p ${opts.dbPort}`;
const psqlOpts = `-v ON_ERROR_STOP=1 --no-password --no-psqlrc`;

P2 Badge Quote native migration executable and file paths

When cacheRoot contains whitespace, the generated shell expands this composite psql command as multiple words, so native Postgres initialization tries to execute only the path prefix and fails. The later init_flags and migrate_flags strings also append unquoted SQL paths, causing the same splitting for every -f argument; invoke the quoted executable separately and preserve migration paths as shell-safe arguments or arrays.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/LocalStack.ts
Comment thread packages/stack/src/PortAllocator.ts Outdated
@jgoux

jgoux commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the two review-body findings in cbc66a7.

  • Native initialization now records transactional init and complete phases. A restart skips only a committed init phase and reruns the migration phase until its completion marker commits.
  • Native psql and migration paths now remain discrete shell arguments, including cache roots with whitespace. Postgres also uses a stable no-space bundle alias because the bundled pgsodium initializer persists and shell-executes its helper path.
  • The native tracer-bullet e2e now starts from a cache path containing spaces, removes the completion marker, restarts the public stack, and verifies the unfinished migration is restored.

The two inline findings from this review were already fixed and resolved in 59a2305.

Comment thread packages/stack/src/services/postgres.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cbc66a7a32

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack/src/supervisor.ts Outdated
Comment thread packages/stack/src/ContainerRuntime.ts
Comment thread packages/stack/src/StackPreparation.ts
jgoux added 26 commits August 20, 2026 20:32
…3-stack-prepare-slim-resources-safely-on-demand-or-upfront

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cfe200427d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +266 to +267
onSuccess: (value) =>
Effect.andThen(markDownloadFinished(), Deferred.succeed(deferred, value)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Complete shared preparation before publishing terminal events

When the caller that created this single-flight is interrupted or disposes its stack during a download, its prepareEvents queue is shut down while the coordinated fiber continues in the longer-lived layer scope. If another stack is awaiting the same inFlight entry, markDownloadFinished() then offers to that closed queue and is interrupted before Deferred.succeed runs; the ensuring removes the map entry, but the second stack's deferred is never completed, so its lifecycle request hangs indefinitely. Complete the shared deferred before attempting the caller-owned notification, or make that notification best-effort.

AGENTS.md reference: AGENTS.md:L62-L66

Useful? React with 👍 / 👎.

servicePolicies.auth !== "off" && config.auth !== false
? (config.auth ?? undefined)
: undefined;
const edgeRuntimeEnabled = servicePolicies["edge-runtime"] !== "off";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject function bundles when Edge Runtime is disabled

When runtime auto-selection falls back to native mode and the caller supplies functions without an explicit edgeRuntime config, this marks Edge Runtime off while resolveFunctionsConfig still accepts and stores the bundle. The stack then starts successfully without an Edge Runtime service, so the explicit functions are silently unavailable; this also occurs when servicePolicies["edge-runtime"] is off. The public functions golden path supplies only functions and relies on Edge Runtime being enabled, so reject this incompatible configuration instead of discarding the requested workload.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant