feat(egress): a declared model endpoints file, its derivation and its squid rules (#503, 1/6) by edgehero · Pull Request #515 · edgehero/pi-dispatch

edgehero · 2026-09-30T14:23:41Z

Part of #503: PR 1 of 6. This adds the declared endpoints file, its parser, the derivation and the squid rules it renders. Nothing is enforced yet. The proxy include, the mounts, the leases and the credential gate come in the later PRs.

What changes

New worker/src/model-endpoints.mjs:
- Parser. Reads model-endpoints.json ({"version":1,"endpoints":[{"id","host","port","slots","keyless"}]}). It refuses and never repairs. It refuses:
  - a loopback, localhost or unspecified host, and ambiguous numeric names;
  - VALKEY_URL's port and 3128;
  - duplicate ids, and duplicate host plus port;
  - slots outside 1-64;
  - unknown keys;
  - a newer version.
- Derivation. A model uses endpoint E when its effective baseUrl has E's host and port. The model's own baseUrl beats the provider's, and a missing port means the scheme default. Only the main model counts until Model policy: a validated model on every trigger kind, allowed-model lists enforced at the provider wrapper, and per-model dollar windows #502 (a named residual).
- Render. Deterministic squid rules, CONNECT only, host ACL first. Literals are rendered as dstdomain -n <ip>: measured 2026-09-30 (see the comment on Local model servers: declared endpoints reached through the egress proxy, keyless providers that pass the credential gate, and fleet-wide slot leases #503), dst <ip> made squid resolve every CONNECT name that reaches the line, and admitted any name resolving to that IP. IPv6 literals are stored bracketed, compressed and lowercase, which is how squid canonicalises the CONNECT host (measured on squid 6.13).
init scaffolds model-endpoints.json (empty) and model-endpoints.conf, which is exactly the empty render. squid refuses to start on a missing include file, so the scaffold must exist.
The default path is <deployment>/model-endpoints.json, and PI_MODEL_ENDPOINTS_FILE overrides it. Doctor loads the file and names a refusal.
PI_DISPATCH_KEYLESS joins CONTAINER_ENV_NAMES, so run.secrets cannot bind it, and PI_FORWARD_ENV refuses it.
Specs: new INT-MODEL-ENDPOINTS-FILE-CONTRACT; INT-CONTAINER-RUNTIME-CONTRACT amended.

Tests

27 new parser, derivation and render tests, plus doctor, init and config tests.
Mutations, each shown red:
- a plain allow emitted
- the port ACL dropped
- derivation by host without port
- the provider baseUrl winning
- the host ACL rendered as dst
- the host ACL placed after the loopback ACL
- the reserved name removed
- the forward-env check removed
The golden render passed squid -k parse on the pinned squid. CONNECT probes gave the expected 200, 403 and 503 answers.

… squid rules (#503) The first change of #503. It adds the file that declares which local or LAN model servers a job may reach, the rule that derives which models use them, and the squid include rendered from them. Nothing reads the include yet: the proxy's include line, the mounts, the slot leases and the keyless credential gate are later changes, and they bind to this one implementation. - worker/src/model-endpoints.mjs: parseModelEndpoints (version rule, refuse never repair, unknown keys refused, loopback and localhost hosts, port 3128 and the Valkey port, duplicate ids and duplicate host and port refused), loadModelEndpoints (model-endpoints.json in the deployment folder, PI_MODEL_ENDPOINTS_FILE overrides, a missing default is no endpoints), readOverlayModels, endpointsForModel and endpointsForProvider (a model's own baseUrl beats its provider's, host AND port), renderEndpointsInclude (CONNECT rules only, the host ACL first, dstdomain -n for IP literals too). - IPv6 literals are stored and rendered in brackets. Measured on squid 6.13 on 2026-09-30: squid compares the bracketed canonical form, so `dstdomain -n [fd00::2]` matches and a bare `fd00::2` does not. The golden render was parsed and probed with CONNECT on the same squid. - init scaffolds an empty model-endpoints.json and model-endpoints.conf (the empty render; squid refuses a missing include file). - PI_DISPATCH_KEYLESS is reserved: run.secrets cannot bind it and PI_FORWARD_ENV refuses it at load. - doctor loads the file through the worker's loader, the default copy included, and names why it does not load. - One VALKEY_URL default, DEFAULT_VALKEY_URL in config.mjs, so doctor and the worker refuse an endpoint on the queue's port alike when VALKEY_URL is unset. - The loopback deny keeps 10.0.2.2 and leaves out 169.254.0.0/16 and fe80::/10 on purpose: Podman 5.3+ reaches the host at 169.254.1.2. - One trailing dot on a baseUrl host is dropped before matching, since squid tunnels `name.` to the same server. A $VAR baseUrl matches nothing, a named residual. Specs: INT-MODEL-ENDPOINTS-FILE-CONTRACT added. INT-CONTAINER-RUNTIME-CONTRACT amended (PI_DISPATCH_KEYLESS reserved). INT-EGRESS-POLICY-CONTRACT and INT-CONFIG-OVERLAY-CONTRACT unchanged, checked. Signed-off-by: Rob Boerman <robboerman@live.nl>

…ed by pi-dispatch egress render (#503) The second change of #503. The proxy's rules now read the include that the first change renders, every way of starting the proxy mounts it, and a new verb writes it and prints the reload. - deploy/egress-proxy.conf (and its mirror): `include /etc/pi-dispatch/model-endpoints.conf` directly after `http_access deny allowed to_host_local` and before `http_access deny CONNECT !SSL_ports`. pi tunnels every provider call and first match wins, so the include must come before the port-443 deny. It coexists with #508's port-80 rule. - The third mount, read-only with `z`, from the deployment folder's own model-endpoints.conf: compose (and mirror), up's run argv, the Quadlet unit (a third placeholder rendered to <deployDir>/model-endpoints.conf, mounted directly like the allowlist and never copied like the rules, because the render writes that file in place), the hand-started Podman recipe, podman-conformance.yml and the host check's mount list. squid refuses to start without the file, so up, service install and doctor name it missing (or a directory) and start nothing. - The proxy, never a job, gets host.docker.internal:host-gateway on Docker (compose and up). Docker Desktop resolves the name already and its host-gateway is the same host address, so one line serves both. Rootless Podman adds nothing: host.containers.internal resolves. - worker/src/egress-cli.mjs, `pi-dispatch egress render`: loads the declaration through the worker's loader, renders in memory, then writes the include IN PLACE (no O_CREAT, O_NOFOLLOW, truncate, fsync). A temp file renamed over a single-file bind mount is invisible to the running proxy and `squid -k reconfigure` then reloads the old rules silently (measured on Docker 29.1.3 and Podman 4.9.3 and 5.8.1, 2026-09-30). A symlink, a directory or a missing file is refused; an unchanged render writes nothing. It prints `docker exec` or `podman exec ... squid -k reconfigure` for the deployment's venues and never runs it. - One governing rule: the folder's rules include model-endpoints.conf only together with a proxy that mounts it. A two-mount proxy stays current; a third bind must be this folder's file; an absent one is drift only when the folder's rules include the file (its next start exits on squid's missing-include FATAL, measured), and such a proxy of this folder is said to be out of date. up's rules refresh and the replace of a proxy without the mount are one step, asked once: declined or not possible here, nothing is written; a failed replace puts the old rules back from the backup. - Endpoints declared under rules that predate #503 are not drift (a replace would cut tunnels and fix nothing): up, doctor and egress render print one shared line naming the rules refresh (docker: the folder's rules; podman: the account copy and service install --force). doctor says a missing include with a cross only when the running rules include it, else a warning. - render: truncates only after judging the file (a hard-linked include is refused), empties the include when a write fails rather than leaving a prefix, compares PI_MODEL_ENDPOINTS_FILE as resolved paths and warns when only the shell sets it. - A replaced proxy is removed with `rm -f -v`, so its anonymous squid volumes go with it. - canonicalHost: `a.lan:` (an empty port) is refused as carrying a port, not as an IPv6 address with an IPv4 tail (PR #515's review). - docs/egress.md gains a short "Local model servers" section (per-venue reachability, jobs cannot use it until the keyless gate and slots land), and says a hand-started proxy or an old compose copy must be recreated with the third mount, never just restarted. Specs: INT-EGRESS-POLICY-CONTRACT amended (the include, its order, its mounts, the in-place render, reload by reconfigure, the governing rule, the endpoint case in Acceptance). REQ-EGRESS-ALLOWLIST amended (endpoint acceptance). DES-EGRESS-DENY-ON-A-DEDICATED-NETWORK amended (through the proxy, never the host network). INT-MODEL-ENDPOINTS-FILE-CONTRACT amended (Status, the empty port). INT-CONTAINER-RUNTIME-CONTRACT unchanged, checked. Signed-off-by: Rob Boerman <robboerman@live.nl>

…ed by pi-dispatch egress render (#503) The second change of #503. The proxy's rules now read the include that the first change renders, every way of starting the proxy mounts it, and a new verb writes it and prints the reload. - deploy/egress-proxy.conf (and its mirror): `include /etc/pi-dispatch/model-endpoints.conf` directly after `http_access deny allowed to_host_local` and before `http_access deny CONNECT !SSL_ports`. pi tunnels every provider call and first match wins, so the include must come before the port-443 deny. It coexists with #508's port-80 rule. - The third mount, read-only with `z`, from the deployment folder's own model-endpoints.conf: compose (and mirror), up's run argv, the Quadlet unit (a third placeholder rendered to <deployDir>/model-endpoints.conf, mounted directly like the allowlist and never copied like the rules, because the render writes that file in place), the hand-started Podman recipe, podman-conformance.yml and the host check's mount list. squid refuses to start without the file, so up, service install and doctor name it missing (or a directory) and start nothing. - The proxy, never a job, gets host.docker.internal:host-gateway on Docker (compose and up). Docker Desktop resolves the name already and its host-gateway is the same host address, so one line serves both. Rootless Podman adds nothing: host.containers.internal resolves. - worker/src/egress-cli.mjs, `pi-dispatch egress render`: loads the declaration through the worker's loader, renders in memory, then writes the include IN PLACE (no O_CREAT, O_NOFOLLOW, truncate, fsync). A temp file renamed over a single-file bind mount is invisible to the running proxy and `squid -k reconfigure` then reloads the old rules silently (measured on Docker 29.1.3 and Podman 4.9.3 and 5.8.1, 2026-09-30). A symlink, a directory or a missing file is refused; an unchanged render writes nothing. It prints `docker exec` or `podman exec ... squid -k reconfigure` for the deployment's venues and never runs it. - One governing rule: the folder's rules include model-endpoints.conf only together with a proxy that mounts it. A two-mount proxy stays current; a third bind must be this folder's file; an absent one is drift only when the folder's rules include the file (its next start exits on squid's missing-include FATAL, measured), and such a proxy of this folder is said to be out of date. up's rules refresh and the replace of a proxy without the mount are one step, asked once: declined or not possible here, nothing is written; a failed replace puts the old rules back from the backup. In each of those cases the usual proxy step then runs on the old rules (start a stopped proxy, unpause a paused one, name declared endpoints), and a blocked step names the way out (rm -f -v the proxy, then up). - Endpoints declared under rules that predate #503 are not drift (a replace would cut tunnels and fix nothing): up, doctor and egress render print one shared line naming the rules refresh (docker: the folder's rules; podman: the account copy and service install --force). doctor says a missing include with a cross only when the running rules include it, else a warning. - render: truncates only after judging the file (a hard-linked include is refused), empties the include when a write fails rather than leaving a prefix, compares PI_MODEL_ENDPOINTS_FILE as resolved paths and warns when only the shell sets it. - A replaced proxy is removed with `rm -f -v`, so its anonymous squid volumes go with it. - canonicalHost: `a.lan:` (an empty port) is refused as carrying a port, not as an IPv6 address with an IPv4 tail (PR #515's review). - docs/egress.md gains a short "Local model servers" section (per-venue reachability, jobs cannot use it until the keyless gate and slots land), and says a hand-started proxy or an old compose copy must be recreated with the third mount, never just restarted. Specs: INT-EGRESS-POLICY-CONTRACT amended (the include, its order, its mounts, the in-place render, reload by reconfigure, the governing rule, the endpoint case in Acceptance). REQ-EGRESS-ALLOWLIST amended (endpoint acceptance). DES-EGRESS-DENY-ON-A-DEDICATED-NETWORK amended (through the proxy, never the host network). INT-MODEL-ENDPOINTS-FILE-CONTRACT amended (Status, the empty port). INT-CONTAINER-RUNTIME-CONTRACT unchanged, checked. Signed-off-by: Rob Boerman <robboerman@live.nl>

…ed by pi-dispatch egress render (#503, 2/6) (#517) The second change of #503. The proxy's rules now read the include that the first change renders, every way of starting the proxy mounts it, and a new verb writes it and prints the reload. - deploy/egress-proxy.conf (and its mirror): `include /etc/pi-dispatch/model-endpoints.conf` directly after `http_access deny allowed to_host_local` and before `http_access deny CONNECT !SSL_ports`. pi tunnels every provider call and first match wins, so the include must come before the port-443 deny. It coexists with #508's port-80 rule. - The third mount, read-only with `z`, from the deployment folder's own model-endpoints.conf: compose (and mirror), up's run argv, the Quadlet unit (a third placeholder rendered to <deployDir>/model-endpoints.conf, mounted directly like the allowlist and never copied like the rules, because the render writes that file in place), the hand-started Podman recipe, podman-conformance.yml and the host check's mount list. squid refuses to start without the file, so up, service install and doctor name it missing (or a directory) and start nothing. - The proxy, never a job, gets host.docker.internal:host-gateway on Docker (compose and up). Docker Desktop resolves the name already and its host-gateway is the same host address, so one line serves both. Rootless Podman adds nothing: host.containers.internal resolves. - worker/src/egress-cli.mjs, `pi-dispatch egress render`: loads the declaration through the worker's loader, renders in memory, then writes the include IN PLACE (no O_CREAT, O_NOFOLLOW, truncate, fsync). A temp file renamed over a single-file bind mount is invisible to the running proxy and `squid -k reconfigure` then reloads the old rules silently (measured on Docker 29.1.3 and Podman 4.9.3 and 5.8.1, 2026-09-30). A symlink, a directory or a missing file is refused; an unchanged render writes nothing. It prints `docker exec` or `podman exec ... squid -k reconfigure` for the deployment's venues and never runs it. - One governing rule: the folder's rules include model-endpoints.conf only together with a proxy that mounts it. A two-mount proxy stays current; a third bind must be this folder's file; an absent one is drift only when the folder's rules include the file (its next start exits on squid's missing-include FATAL, measured), and such a proxy of this folder is said to be out of date. up's rules refresh and the replace of a proxy without the mount are one step, asked once: declined or not possible here, nothing is written; a failed replace puts the old rules back from the backup. In each of those cases the usual proxy step then runs on the old rules (start a stopped proxy, unpause a paused one, name declared endpoints), and a blocked step names the way out (rm -f -v the proxy, then up). - Endpoints declared under rules that predate #503 are not drift (a replace would cut tunnels and fix nothing): up, doctor and egress render print one shared line naming the rules refresh (docker: the folder's rules; podman: the account copy and service install --force). doctor says a missing include with a cross only when the running rules include it, else a warning. - render: truncates only after judging the file (a hard-linked include is refused), empties the include when a write fails rather than leaving a prefix, compares PI_MODEL_ENDPOINTS_FILE as resolved paths and warns when only the shell sets it. - A replaced proxy is removed with `rm -f -v`, so its anonymous squid volumes go with it. - canonicalHost: `a.lan:` (an empty port) is refused as carrying a port, not as an IPv6 address with an IPv4 tail (PR #515's review). - docs/egress.md gains a short "Local model servers" section (per-venue reachability, jobs cannot use it until the keyless gate and slots land), and says a hand-started proxy or an old compose copy must be recreated with the third mount, never just restarted. Specs: INT-EGRESS-POLICY-CONTRACT amended (the include, its order, its mounts, the in-place render, reload by reconfigure, the governing rule, the endpoint case in Acceptance). REQ-EGRESS-ALLOWLIST amended (endpoint acceptance). DES-EGRESS-DENY-ON-A-DEDICATED-NETWORK amended (through the proxy, never the host network). INT-MODEL-ENDPOINTS-FILE-CONTRACT amended (Status, the empty port). INT-CONTAINER-RUNTIME-CONTRACT unchanged, checked. Signed-off-by: Rob Boerman <robboerman@live.nl>

edgehero force-pushed the feat/503-endpoints-file branch from 8a1e081 to 4730b8e Compare September 30, 2026 14:38

edgehero force-pushed the feat/503-endpoints-file branch from 4730b8e to e625dbb Compare October 2, 2026 06:30

edgehero merged commit 4e22b93 into main Oct 2, 2026
7 checks passed

edgehero mentioned this pull request Oct 2, 2026

Local model servers: declared endpoints reached through the egress proxy, keyless providers that pass the credential gate, and fleet-wide slot leases #503

Closed

8 tasks

Navigation Menu

Repository navigation

feat(egress): a declared model endpoints file, its derivation and its squid rules (#503, 1/6) - #515

feat(egress): a declared model endpoints file, its derivation and its squid rules (#503, 1/6)#515
edgehero merged 1 commit into
mainfrom
feat/503-endpoints-file

edgehero commented Sep 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

edgehero commented Sep 30, 2026

What changes

Tests

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant