Skip to content

fix(init,up): the no-clone quickstart starts its egress proxy (#480) - #488

Merged
edgehero merged 1 commit into
mainfrom
fix/no-clone-egress-conf
Sep 29, 2026
Merged

edgehero merged 1 commit into
mainfrom
fix/no-clone-egress-conf

Conversation

@edgehero

@edgehero edgehero commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Closes #480.

The no-clone quickstart did not produce a working deployment. In an empty folder, npx @edgehero/pi-dispatch up skipped the egress proxy (deploy/egress-proxy.conf is not here), and with the egress policy on by default every job was then refused before it spent. up's own advice to run init again could not help, because init never wrote that file. /dispatch setup had the same hole: its up pass ran before it copied any deploy/ file in. Only a clone worked.

  • init writes deploy/egress-proxy.conf, a byte for byte copy of the package's own file, create only, and lists it. up then starts the proxy on a fresh folder, and doctor's check that both proxy files are present passes.
  • Every init scaffold is written create only (wx), so nothing that exists is written through, a dangling symlink included; that is reported as kept. init refuses a deploy/ that is a symlink or not a directory, and a directory where any scaffold's file belongs: it writes nothing there, says so beside the list of what it did write, and exits 1.
  • up goes on past an init that refused or threw, says so and names it in its summary (a file named deploy used to abort up after the image pull). A directory at either proxy file is never treated as the file: up starts no proxy on it, and doctor fails naming it.
  • init's Docker ladder names only files the folder has. Step 2 is pi-dispatch up, which starts Valkey and the egress proxy (unless PI_EGRESS=0); the old compose command needed a clone and never started the proxy. The podman ladder links the guide's URL instead of a path the folder does not have.
  • up runs init without its ladder (steps: false), so it no longer prints instructions to do what it is about to do. The created/kept lines stay.
  • The file column is as wide as the longest name.
  • Remedies name pi-dispatch up, and compose only where it is right:
    • The egress-proxy-down refusal comment, the sandbox's network hint and doctor's proxy fix line name pi-dispatch up from the deployment folder and no compose line. Only up knows the folder; a compose line printed without it would, in a folder /dispatch setup laid out, also start compose's valkey under project deploy. A proxy PI_EGRESS_PROXY names is the operator's to start, and the comment and the sandbox say so. Reason tokens, log events and the run record are unchanged.
    • Every later or fallback compose line up prints (declined Valkey or proxy, failed volume create or container start, an open Valkey it did not start) appears only where the folder holds deploy/docker-compose.yml; otherwise it names pi-dispatch up again.
    • The "could not reach Valkey" hint in the CLI, cancel, service restart and the panel names pi-dispatch up only for a loopback VALKEY_URL, and otherwise asks whether the Valkey at that host is running.
    • The worker unit template's Valkey comment names pi-dispatch up (deploy/worker.service and its worker/deploy mirror, byte identical).
    • The wizard's skipped-receiver note names no receiver container on the podman venue, and its create-only copy says "kept as it is" neutrally. setup github's closing steps offer the compose profile only for a folder holding the file.
    • A check in up.test.mjs requires --env-file on every compose command outside a comment in worker/src, receiver/src and admin/src (flags in any order, docker-compose too, its own judgement tested on sample lines), and names each prose mention; composeArgs is pinned to carry --env-file .env.
  • Docs: docs/egress.md's one-time steps are pi-dispatch up then doctor; the forge docs' receiver-in-a-container option and docs/podman.md say the compose file comes with a clone or /dispatch setup.

REQ-DEPLOYMENT-BOOTSTRAP and INT-EGRESS-POLICY-CONTRACT amended with revision rows. Every new test mutation checked. Full suite: 5237 tests, 0 fail.

Not in this PR: a folder copy of egress-proxy.conf is never refreshed on upgrade (#484).

@edgehero
edgehero force-pushed the fix/no-clone-egress-conf branch 3 times, most recently from 2a7a411 to eb3fd1a Compare September 29, 2026 10:49
In an empty folder, `npx @edgehero/pi-dispatch up` never started the
egress proxy. `up` needs both files the docker proxy mounts in the
folder (egress-allowlist.conf and deploy/egress-proxy.conf), and init
wrote only the allowlist, so on the default policy `up` printed "the
egress policy is on but deploy/egress-proxy.conf is not here" and every
job was then refused before it spent. Its own advice (run init, then up
again) could not help, because init did not write that file either.
Only a clone worked, since its root carries deploy/.

What changes:

- init scaffolds deploy/egress-proxy.conf, create-only like every other
  scaffold, from the package's own copy (../deploy/ from the module, the
  layout service.mjs resolves templates by: worker/deploy in a checkout,
  the shipped deploy/ under npm), creating deploy/ when needed, and lists
  it. A clone's own file is reported kept and never read or replaced.
- init's docker next steps name only files the folder has: step 2 was a
  compose command naming deploy/docker-compose.yml, which a no-clone
  folder lacks (and which started no proxy either); it is now
  `pi-dispatch up`, which starts Valkey and the proxy. Step 1's note no
  longer names image/Dockerfile.
- up runs init with `steps: false`: the created and kept lines stay, but
  init's "Next:" ladder no longer prints in the middle of a pass that is
  performing those same steps. up's own closing Next is unchanged.
- init's name column is padded to the longest name rather than 20, so
  egress-allowlist.conf's note lines up.
- The podman ladder named docs/podman.md, which a no-clone folder has
  no copy of either; it now names the Podman guide by its URL.
- Docs addressed to a no-clone operator: docs/egress.md's "What you
  have to do, once" is now `pi-dispatch up` then `doctor` from the
  deployment folder, with the compose route kept for a clone or a
  folder holding deploy/docker-compose.yml. The receiver compose
  alternative in docs/azure-devops.md, docs/forgejo.md and
  docs/gitlab.md says it needs deploy/docker-compose.yml (a clone
  carries it, /dispatch setup copies it in). docs/podman.md step 7
  says the compose file is a clone's and names `pi-dispatch up` for a
  folder without one (not separately measured on that venue).
  docs/backup.md's compose lines are conditional on a compose-started
  Valkey and stay as they are.

- The product's own remedies follow the same rule. The egress-proxy
  refusal comment (processor.mjs), the sandbox's network-failure
  message (sandbox.mjs) and doctor's proxy fix line name `pi-dispatch
  up` from the deployment folder and no compose line: only up knows
  the folder, and a compose line printed without it would, in a
  folder /dispatch setup laid out, also start compose's unprofiled
  valkey under project `deploy` (a second Valkey on a fresh volume,
  or a clash on its port). A proxy PI_EGRESS_PROXY names is the
  operator's own, which up never starts, and the comment and the
  sandbox say to start it themselves. Every later or fallback compose
  line up prints (declined Valkey, declined proxy, failed volume
  create, failed container start, an open Valkey up did not start) is
  printed only where the folder holds deploy/docker-compose.yml, and
  otherwise names `pi-dispatch up` again. The Valkey-unreachable hints
  in cli.mjs, cancel-cli.mjs, service.mjs and the panel
  (admin/src/index.ts) said "(docker compose up)"; they name
  `pi-dispatch up` only for a loopback VALKEY_URL, the only Valkey up
  starts, and otherwise ask whether the Valkey at that host is running
  (valkeyDownHint). The worker unit template's Valkey comment
  (deploy/worker.service and its worker/deploy mirror, byte identical)
  names `pi-dispatch up`. The wizard's skipped-edge note names no
  receiver container on the podman venue, its create-only copy says
  "kept as it is" neutrally (init now writes that file too), and the
  GitHub App setup's closing steps offer the compose profile only for
  a folder holding the file. Refusal reason tokens, log events and the
  run record are unchanged: the refusal text is a forge comment, not a
  record field. A static test in up.test.mjs requires --env-file on
  every compose command outside a comment in worker/src, receiver/src
  and admin/src and names each prose mention; composeArgs is pinned to
  carry --env-file .env for commands built at run time.
- Every init scaffold is written create-only with flag wx, so nothing
  that exists is written through, a dangling symlink included (which
  existsSync reports absent and a plain write followed out of the
  folder); EEXIST is reported kept. init refuses, writes nothing into
  it and exits 1 when deploy/ is a symlink or not a directory, and
  refuses a directory where any scaffold's file belongs (it used to call
  one "kept", and docker would mount it as squid.conf), listing every
  other file it wrote beside the refusal. The kept note is "already
  exists, left untouched".
- up goes on past an init that refused or threw, says so and names it
  in its summary: a FILE named deploy used to make init throw EEXIST
  and abort up after the image pull. up starts no proxy, and doctor
  fails naming the path, where a directory stands at either of the
  proxy's two files.

Verified with the real CLI in an empty folder before and after: before,
seven files and a ladder naming deploy/docker-compose.yml; after, eight
files, the rules byte-identical to the package's, a second run keeps an
edited copy, and the ladder names only .env.

Tests pin each acceptance point: up in a real empty folder with the real
init reaches the proxy start from the shown argv with no missing-file
line and one Next (its own); init lists the file and copies the package
bytes; init never overwrites an existing one; neither the docker nor the
podman ladder names a path the folder lacks; the column is one width;
steps: false prints no ladder on any venue; each changed product message
is pinned by its own test (processor, sandbox, doctor, up's two
declines, the wizard note, the App setup's steps), as are the review
round's cases (a dangling symlink, a symlinked or non-directory deploy/,
a directory at a scaffold's path in init, up and doctor, init refusing
or throwing under up, the scan's own judgement, the gated fallbacks, a
custom proxy, valkeyDownHint, composeArgs and the source scan). Each was
mutation-checked red. Two doctor --fix tests in doctor.test.mjs run init
in a temp folder beside a fake proxy mounting process.cwd()'s files;
with both files now in the folder doctor compares the mounts and called
that proxy stale, so their fake proxy now mounts the test's own folder
(a fixture change; re-checked after rebasing onto #485, still needed:
red without it). The doctor --fix podman ladder assertion follows the
new first line.

Spec IDs: REQ-DEPLOYMENT-BOOTSTRAP AMENDED (init scaffolds the rules
file, the ladders name only files the folder has, up runs init without
its ladder, init writes every scaffold wx and refuses a symlinked
deploy/ or a directory at a file's path, up survives init, the remedies
name `pi-dispatch up`); INT-EGRESS-POLICY-CONTRACT AMENDED (the rules
bullet: init copies the package's file into a folder that lacks one).
UNCHANGED, checked: REQ-EGRESS-ALLOWLIST (the rules content, the
allowlist scaffold and the policy), DES-CLI-SURFACE (init is still
create-only and runs nothing; up's consent tiers are as they were),
INT-CONTAINER-RUNTIME-CONTRACT (no job flag or mount moves), the podman
ladder's steps and the podman venue (its unit mounts an account-owned
copy that service install writes, never the folder's), the .env.example
mirror (not edited), and the deploy/ mirror (worker.service edited
identically in both copies).

Final review: a write that throws part way (a read-only deploy/, say)
still prints the list of what init had already created before the
error propagates, and docs/egress.md now says the refusal names
`pi-dispatch up`, as it does since this change.

Signed-off-by: Rob Boerman <robboerman@live.nl>
@edgehero
edgehero force-pushed the fix/no-clone-egress-conf branch from eb3fd1a to 15e2259 Compare September 29, 2026 11:07
@edgehero
edgehero merged commit ceaa43e into main Sep 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The no-clone quickstart starts no egress proxy: init never writes deploy/egress-proxy.conf

1 participant