Operator console and durable runs: features adopted from durable-pi #650

Goal

Track the operator console and the features adopted from disler/durable-pi.
Decided 2026-10-14: build our own, do not build on durable-pi or @earendil-works/pi-durable (the 2026-10-12 decision
in specs/design.md:6557-6588 stands for the dependency), but adopt its ideas.

Decisions taken

• Reverse "no web dashboard": a served console with full management, loopback only, SSH tunnel, token login.
• Adopt from durable-pi: durable run state with crash recovery and per-tool replay policy, approval gates, steering a
  running job, phase/step progress, fork / re-run with another model, and a terminal chat.
• 2026-10-18: the epic ends with release 5.0.0 (Release 5.0.0 #695), including new READMEs, images and a release video.
• 2026-10-18: NVIDIA OpenShell becomes a fully supported third backend. This reverses the constitution's rejection of
  OQ-011's TLS-terminating, secrets-injecting proxy (specs/constitution.md:346) for that venue (OpenShell: reverse OQ-011, allow a TLS-terminating secrets-injecting venue layer #680).
• 2026-10-19: project secret bundles (Project secret bundles: grant a project's env once, shared by every job, any manager or none #704): a project's env granted once and shared by every job of the project, vendor-neutral; the 1Password route is a read-only, Environment-scoped service account, and its desktop-app OpenShell provider integration is declined.
• 2026-10-19: the console is audited for people, not only for parity: /dispatch console: open, sign in to and check the web console from pi, and run it as a service #708, Console: explain every outcome, a truthful overview, one vocabulary with the panel #710, Console: feedback after writes, sessions and restarts, forms that refuse, phone width and accessibility #711, Console and panel: venues, doctor results, OpenShell runs and secrets shown truthfully #712 join the series before the release. A console security finding on main is tracked privately (GHSA-4qvc-f352-mm29) and must be fixed before 5.0.0 ships the console.
• 2026-10-19: models change only by a reviewed edit. A rate limit or quota moves the next attempt to a listed fallback before spend (Model failover: on a rate limit or quota, the next attempt runs on a listed fallback, chosen before spend #716); roles let one edit move every job, and new models are proposed with a canary, never adopted (Model roles: agents and flows name roles, the operator maps them once; new models proposed with a canary, never adopted #717). Subscription logins stay refused, and portfolio plans stay weights only.

Spec decisions these reverse (each issue amends its own, in its own PR)

• DES-ADMIN-VIA-PI-EXTENSION ("binds no network port") and the rejected options at specs/design.md:2857-2870: Operator console security model: loopback only, token login, CSRF and rebinding guards, audit log #640, Operator console: a live web dashboard for runs, parked runs, spend, queues, hosts and configuration #641.
• CONST-RETRY-INFRA-ONLY / REQ-RESUMABLE-SESSION ("retry is not continue") and the pi-durable rejection: Durable runs: checkpoint session and workspace so a crashed run continues, with a per-tool replay policy #645.
• DES-SANDBOX-IS-A-FRESH-CONTAINER ("a stdin channel to the running agent" rejected): Steer a running job: comments and console messages reach the agent through a durable inbox #646.
• OQ-011's rejected proxy, CONST-EGRESS-POLICY-IN-THE-ARGV, INT-CONTAINER-RUNTIME-CONTRACT,
  INT-EGRESS-POLICY-CONTRACT, DES-STEER-VIA-READONLY-INBOX-MOUNT, INT-SANDBOX-CONTRACT: OpenShell: reverse OQ-011, allow a TLS-terminating secrets-injecting venue layer #680.
• Kept: CONST-ISOLATION-CONTAINER-PER-JOB, CONST-BUDGET-BEFORE-TOKENS, CONST-TRIGGER-AUTHOR-GATE,
  DES-AWAIT-INPUT-ENDS-THE-JOB (approval gates end the job like parked runs).

Issues and order

1. Operator console security model: loopback only, token login, CSRF and rebinding guards, audit log #640 console security model (first; everything served depends on it)
2. Operator console: a live web dashboard for runs, parked runs, spend, queues, hosts and configuration #641 web console: server, read screens, live updates
3. Operator console management: cancel, retry, re-run, answer parked runs and edit configuration from the browser #642 console management: writes, answers, audit
4. Live run transcript from every host, streamed through Valkey for the console and the panel #643 live transcript stream from every host
5. Run progress: phases and steps visible while a job runs #644 phase and step progress
6. Durable runs: checkpoint session and workspace so a crashed run continues, with a per-tool replay policy #645 durable run state and crash recovery
7. Steer a running job: comments and console messages reach the agent through a durable inbox #646 steer a running job through a durable inbox
8. Approval gates: the agent asks permission for a named action, the run parks, a human approves or declines #649 typed approval gates
9. Fork a run: re-run from an earlier point in the session, optionally with another model #648 fork / re-run from a point with another model
10. Terminal chat: open a run's session in a terminal, read it, continue it, or attach live #647 terminal chat for a run

Status 2026-10-18: #640 to #647 are done. Open: #649, #648.

OpenShell backend

11. OpenShell: reverse OQ-011, allow a TLS-terminating secrets-injecting venue layer #680 decision and spec amendments (blocks the rest of this group)
12. Venue code assumes only docker and podman; a third venue's containers are checked with the docker CLI #681 venue code generic beyond docker and podman (bug; worth doing on its own)
13. OpenShell backend: job image, runner, backend lifecycle, doctor and CI #683 the backend: job image and runner, run/stop/reap/size, doctor and CI
14. OpenShell policy layer: egress network policy, per-job credential providers, MCP #684 the policy layer: egress network policy, per-job credential providers, MCP
15. OpenShell: steering and live transcripts over stdio, optional bind mode #686 steering and live transcripts over stdio, optional bind mode

Status 2026-10-19: #683 is done (#698, #700, #701, #702). #684 is done (#703, #705), and the openshell venue is blessed. #638's residual (held replies) is closed by #706. Follow-up: #707 (up and service install do not know openshell). Open: #686.

Status 2026-10-19 (later): #686 is done (#709 the live transcript and progress over stderr, #713 steering over framed stdin, #714 the opt-in openshell-bind venue with folder jobs, file steering and run.recover). Release debt: the job image gains transcriptStderr and steerStdin.
Status 2026-10-19 (later): #707 is done (#715): up and service install know both openshell venues. An absent declaration is now a doctor warning, not a failure.

Status 2026-10-21: #704 is done (#718, #719, #720, #721): a project's env is granted once in triggers.json (secretBundles) and reaches every trigger-made job of the project on every venue; on openshell a variable with a declared endpoint is a per-host placeholder. A job image release is owed (bundleEnvFile).

Status 2026-10-21 (later): #712 is done (#722, #724, #723, #726): the console and the panel say nothing wrong for a run's venue (a serves block in BACKENDS_TABLE, liveSkipped on the record), a Venues view, doctor results per host in Valkey, and OpenShell run facts plus a per-profile secrets view. Follow-up #725 (an openshell lane steering race). A worker release is owed for the new registry and record fields and doctor publishing.

Status 2026-10-21 (evening): #708 is done (#728, #729): /dispatch console prints the address, says whether the console runs and mints an audited one-time sign-in link, and pi-dispatch service install --console keeps the console running. #725 (the openshell lane's steering race) is in progress as #727: fixing it exposed that pi's own model calls fail with a connection error inside an OpenShell sandbox, under investigation.

Project secret bundles

16. Project secret bundles: grant a project's env once, shared by every job, any manager or none #704 grant a project's env once, shared by every job, any manager or none (1Password via a read-only service account)

Console experience (found auditing the console against OpenShell and secrets, 2026-10-19)

17. /dispatch console: open, sign in to and check the web console from pi, and run it as a service #708 /dispatch console: open, sign in to and check the console from pi, and run it as a service
18. Console: explain every outcome, a truthful overview, one vocabulary with the panel #710 explain every outcome, a truthful overview, one vocabulary with the panel
19. Console: feedback after writes, sessions and restarts, forms that refuse, phone width and accessibility #711 feedback after writes, sessions and restarts, forms that refuse, phone width and accessibility
20. Console and panel: venues, doctor results, OpenShell runs and secrets shown truthfully #712 venues, doctor results, OpenShell runs and secrets shown truthfully (bundles after Project secret bundles: grant a project's env once, shared by every job, any manager or none #704)

pi.dev

21. pi.dev: adopt pi 1.3.0 and support the Radius gateway under a cost cap #693 adopt pi 1.3.0 and support the Radius gateway under a cost cap

Models (researched 2026-10-19)

22. Model failover: on a rate limit or quota, the next attempt runs on a listed fallback, chosen before spend #716 failover: on a rate limit or quota, the next attempt runs on a listed fallback, chosen before spend
23. Model roles: agents and flows name roles, the operator maps them once; new models proposed with a canary, never adopted #717 model roles mapped once by the operator, agent files name roles; new models proposed with a canary, never adopted

Release 5.0.0

24. READMEs and images for 5.0.0: console, live runs, steering, chat, durable runs #692 READMEs and images
25. 5.0.0 release video: scripted, re-renderable, no model calls #691 release video
26. Release 5.0.0 #695 release 5.0.0 (last)

Dependencies: #641 needs #640; #642 needs #641; #644 and the live run view use #643; #647 live attach needs #643 and
#646; #648 mid-run workspace restore uses #645 checkpoints; approval decisions from the console need #642. OpenShell: #681 before #683; #684 and #686 need #683; #704 needs #684 (its OpenShell part). #712 shows #704 and #686 once they land; #710 holds the reason catalog #712 uses. Models: #717's role map can carry #716's fallback list; #716 skips an unpriced candidate, so Radius waits for #693.
The video (#691) is rendered from the 5.0.0 build. Decided 2026-10-18: #695 waits until every other issue here is finished.

Research notes

• Fork a run: re-run from an earlier point in the session, optionally with another model #648 conflicts with OQ-046 (specs/open-questions.md:2102: no console route serves the stored, unredacted
  transcript); settle that before the console fork. Durable runs: checkpoint session and workspace so a crashed run continues, with a per-tool replay policy #645 checkpoints exist only for run.recover runs, on one host,
  for 7 days.
• OpenShell (2026-10-18): NVIDIA/OpenShell v0.1.4, Apache-2.0, weekly releases, Experimental interfaces can change in
  a patch. Needs Linux 6.2+ (Landlock ABI 3) and Docker 28+ or Podman 5. Copies files into the sandbox, so steering,
  live transcripts, live checkpoints and local folders need new transport or bind mode.
• durable-pi: one squashed commit 37165c6 (2026-10-11), MIT, demo grade, pins pi-durable 1.0.3. Device server per
  machine, SQLite per session, Vue hub with SSE, bearer token per device ("a token is a shell").
• pi 1.2.0: SDK steer/followUp/fork(entryId, {position}), session entries appended per message without fsync,
  no remote attach, no resume of pending tool calls, AgentTool.replay declared but unused.