Skip to content

feat(worker,runner): the portfolio snapshot in, the priorities plan out, and the portfolio persona - #577

Merged
edgehero merged 1 commit into
mainfrom
feat/505-portfolio-plan
Oct 4, 2026
Merged

edgehero merged 1 commit into
mainfrom
feat/505-portfolio-plan

Conversation

@edgehero

@edgehero edgehero commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Refs #505 (part B, the last of two). Closes #505 together with #575.

A flagged cron job now sees the budget split as a read-only snapshot, can hand back a priorities plan through its outbox, and the worker applies that plan under the same rules as an operator's, after the container has exited.

The snapshot in

  • /job/portfolio.json, read-only, built when the job is prepared and only when the job was flagged at pickup and the live triggers file still flags the same entry. A plan is applied only when the pickup, the prepare step (which wrote the snapshot) and the live file at collection all agree.
  • It holds ids, digests, integers, ISO instants, enum tokens and operator labels only (a label with control, format or bidi characters is replaced by the member's ref): the envelope numbers, the current plan's id and writer, the last attempt's outcome, and per project its floor, weight, share and spend (spent plus still held, read from the same counters enforcement uses), its members, and its last seven days of run outcomes. No issue text, no titles, no plan reasons and no paths.
  • A Valkey fault while building it is retried before anything is spent; a snapshot over 64 KiB refuses the job before it spends as portfolio-snapshot-oversize.

The plan out

  • The job writes /outbox/priorities.json, a separate file from chain requests, so it never uses a chain slot and an older worker never misreads it.
  • After a completed exit, the collector checks, in order: still a portfolio job (plan-not-portfolio), a file it can read (plan-unreadable), size (plan-oversize), a regular file opened without following links (plan-not-regular-file), JSON (plan-parse-error), the plan contract (plan-invalid plus the field), and then applies it as writer portfolio-job under the envelope's rules. It never throws, so a refused plan never turns a completed job into a retry; a fault inside it is recorded as plan-collect-error.
  • Every outcome lands in the run record as plan. The portfolio job's own attempts also land in the audit file and in alloc:log, and the next snapshot's last attempt tells the next run what happened. A plan file left by any other job is refused in its run record only, so no ordinary job can crowd the panel's history. plan-collect-error means the outcome is unknown and the plan may have applied; the record still carries its plan id.

The persona and the pre-check

  • guardrails/PORTFOLIO_PROTOCOL.md is baked into the image and composed into the system prompt only when /job/portfolio.json exists, decided once when the loader is built, so the prompt stays the same on every turn. It explains the snapshot, the plan, the basis rule, and that the host decides after the job exits.
  • Before its exit line, the runner checks the plan's shape and logs refusals so the agent's log shows them. It never changes the exit code or the exit line.

Specs

AMENDED: INT-CONTAINER-JOB-INPUTS, INT-OUTBOX-CONTRACT (a second file and its order of checks), INT-RUN-HISTORY-FILE-CONTRACT (plan, the new reasons), INT-PRIORITIES-PLAN-CONTRACT, DES-JOB-OUTBOX-CHAINING (a second kind of request; the rejected type key on a chain request), DES-DELEGATED-ALLOCATION-INSIDE-ENVELOPE, SECURITY.md (the portfolio plan rows). UNCHANGED, checked: REQ-AI-TRIGGERED-RUNS (a plan enqueues nothing).

Tests

The snapshot byte for byte with a planted issue title absent, every collector refusal in order, a symlinked plan file, manual and chained jobs refused, a flag removed after queueing, re-collection as plan-duplicate, the processor collecting only on a completed exit, the persona present or absent with an identical prompt across turns, the pre-check never changing the exit, and a live Valkey race between a job's plan and an operator's apply. Each rule was checked by reverting it and watching its test fail. The by-hand steps of #505 were run with a local model priced through the overlay.

@edgehero
edgehero force-pushed the feat/505-portfolio-plan branch from 313db20 to 4deac08 Compare October 4, 2026 12:47
…ut, and the portfolio persona

Part B of issue #505. A flagged cron job now reads the budget as
/job/portfolio.json, writes /outbox/priorities.json, and the worker
applies that plan under the envelope's rules after the job completes,
with no keypress.

The snapshot (worker/src/portfolio-snapshot.mjs): written by
prepareLocalWorkspace through an injected builder, 0444 beside
event.json on the existing /job mount. Only for a job the processor
confirmed as a portfolio job at pickup AND the live triggers file still
flags at prepare; a chained child and a manual run never get one. It
holds ids, digests, integers, ISO instants, enum tokens, project ids
and member labels (a folder as local:<basename>; a label holding a
control, format or bidi character is replaced by the member's ref),
never issue text, a title, a plan reason or a path. Money comes from
the fleet-wide budget:usd:s counters of the envelope's window. A
member's key comes from memberDollarKeyPrefix (allocation.mjs), the
helper governedDollars now uses too, so a member matched by a bare
row such as acme/web is read from the key enforcement settles into. spentMicros is the
counter itself (spent plus held, what the next job is admitted
against); the issue's sketch also named reservedMicros, which the
counters cannot split, so it is left out and the spec says why.
runs7d merges the local history with the run mirror, and
fleet.runsComplete is false without PI_WORKER_NAME. lastAttempt is the
newest alloc:log row of this trigger's portfolio jobs. A Valkey fault
is an InfraRetry before any reserve; over 64 KiB the job is refused as
portfolio-snapshot-oversize, and the log line names the project count.

The collector (worker/src/outbox-plan.mjs, makeCollectPlan): wired
beside collectChain (start.mjs, index.mjs rewrap with the real job) and
called on EXIT_COMPLETED after it, with the pickup's portfolio
decision. Ladder: no file (plan null; only ENOENT and ENOTDIR mean
none, any other lstat answer is judged after authority, and a job the
pickup never confirmed then has no plan, so its record is unchanged),
plan-not-portfolio (the pickup decision, prepare, which marks
prepared.portfolio only when it wrote the snapshot, and the live file
at collection must all say yes), plan-oversize and
plan-not-regular-file by lstat, then open with O_NOFOLLOW and
O_NONBLOCK and fstat the descriptor for both rules again, a read capped
at 16 KiB plus one, plan-unreadable, plan-parse-error, then applyPlan
with the writer portfolio-job. The collector's own refusals are written
by a new recordRefusal (an audit row and an alloc:log row, the token
and no body), except for a job the pickup never confirmed (a manual
run, a chained child, an unflagged cron job): its plan-not-portfolio
goes to the run record and the log line only, because alloc:log keeps
500 rows and is the panel's revert history, so stray files from any
local job would wipe it. It never throws: a fault is
plan-collect-error, which means the outcome is unknown (a lost reply
may hide a plan that applied), so it writes no audit or alloc:log
row and the record carries the plan's id;
the job stays completed. Log line plan_collected { jobId, outcome,
reason }.

The run record gains the tail field plan { outcome, reason, planId,
clamped } after project, rebuilt and enum-checked; the byte pins move.

The persona: guardrails/PORTFOLIO_PROTOCOL.md with its sentinel,
copied beside OUTBOX_PROTOCOL.md (Dockerfile.azure inherits it), and
composed by the loader only when /job/portfolio.json exists, read once
at build, in the order guardrails, outbox, portfolio, global, project.
verify-image.sh checks both sentinels, and a new test reads the real
files.

The runner pre-check (image/runner/src/plan-check.mjs): at exit,
before the exit line, a guarded and discarded look at
/outbox/priorities.json under the host's file rules, with a vendored
copy of parsePlan's structural half bolted to the worker's by a
cross-workspace corpus test. It logs plan_precheck with enum tokens
and can change neither the exit code nor the exit line.

Specs: INT-CONTAINER-JOB-INPUTS (/job/portfolio.json), INT-OUTBOX-
CONTRACT (the second file and its ladder), INT-RUN-HISTORY-FILE-
CONTRACT (plan, portfolio-snapshot-oversize), INT-PRIORITIES-PLAN-
CONTRACT (one sentence on the pre-check's maxPlanDays),
DES-JOB-OUTBOX-CHAINING (a second kind of request; the rejected type
discriminator), DES-DELEGATED-ALLOCATION-INSIDE-ENVELOPE (the
portfolio-job writer path, three decisions that must agree, a hand
fire deferred behind a tick by the folder mutex, never-confirmed
refusals kept out of alloc:log, plan-collect-error as an unknown
outcome) amended, and two
SECURITY.md rows (the trigger table and the trust zones).
REQ-AI-TRIGGERED-RUNS UNCHANGED, checked: a plan enqueues nothing.
HOOK_POLICY_REASONS UNCHANGED, checked: portfolio-snapshot-oversize is
decided before the reserve, so there is nothing spent to page about.
INT-TRIGGERS-FILE-CONTRACT, INT-ENVELOPE-FILE-CONTRACT and
INT-RUNNER-EXIT-CODE-PROTOCOL UNCHANGED, checked. Docs: allocation.md
(portfolio jobs; the manager's own job needs a share with a floor;
portfolio-no-envelope on a host with no envelope is reachable only
while the fleet has no applied split, else envelope-mismatch comes
first) and the operate-pi-dispatch SKILL.md.

Refs #505

Signed-off-by: Rob Boerman <robboerman@live.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unattended project manager: a portfolio snapshot into a flagged cron job, a priorities plan back out through the outbox

1 participant