Skip to content

fix(ci): keep the openshell lane's steering run alive for its mid-run frame (#725) - #727

Open
edgehero wants to merge 3 commits into
mainfrom
fix/openshell-lane-steer-race-725
Open

edgehero wants to merge 3 commits into
mainfrom
fix/openshell-lane-steer-race-725

Conversation

@edgehero

Copy link
Copy Markdown
Owner

Fixes #725.

The copy-mode steering case ran at maxCostMicros: 0 against a real model, so the runner hit its cost cap within milliseconds of the first ack. The worker closes stdin on the verified exit line, so the second message's frame was never written. It now runs on a zero-rated, keyless fake model served by the stand-in:

  • The first request waits for an event, not a timer. The stand-in holds it until the conformance script signals, by touching STEER_SIGNAL_FILE, that the relay has written the second frame (at most 30 s). It then asks for a read of the job's prompt file.
  • Any request carrying a tool result gets "done". The reply depends on the request body, not on a request counter, so a retry of the first request is held again.
  • The pass condition stays strict. Exit 0 with a verified exit line naming both messages, both delivered, at least 2 frames written and 2 acks verified, and no cost spent.

The stand-in's other routes answer exactly as before, and the request body is never logged.

Rejected: rerunning a relay tick requested mid-pass. That was not the cause, and in production it would only add Valkey load when Valkey is slow.

Residual: this case is the first to stream a model answer through OpenShell 0.1.4's protocol: rest layer.

Verification:

  • node --check passes, and the workflow YAML parses.
  • The extracted stand-in, driven locally, holds until the signal file appears, holds a retry again, and answers "done" once a tool result arrives.
  • The real runner against it: exit 0, 2 turns, cost 0.
  • The openshell lane will be rerun three times on this PR before merge.

… frame (#725)

The copy-mode steering case ran with maxCostMicros 0 and a real model, so
the runner hit its cost cap within milliseconds of the first ack. The
worker ends the exec's stdin on the verified exit line, so the second
message's frame was never written, and the case failed whenever the
relay had not written it first: on the #723 rerun and on #724.

The case now runs on a zero-rated, keyless fake model served by the
stand-in (POST /v1/chat/completions, streamed):
- The first request is held until the conformance script signals, by
  touching STEER_SIGNAL_FILE, that the relay has written the second
  frame. It waits at most 30 s. It then asks for a `read` of the job's
  prompt file.
- Any request carrying a tool result answers "done".
- The reply is chosen from the request body, not a counter, so a retry of
  the first request is held again. The body is never logged, and every
  other route answers exactly as before.

The assertion stays strict: exit 0 with a verified exit line naming both
messages, both delivered, at least 2 frames written and 2 acks verified,
no cost refusal, and a cost of 0.

Rejected: a relay change that reruns a tick requested mid-pass. Nothing
in production ticks from a mark, the failure was the closed sink, and in
production it would only add back-to-back Valkey passes when Valkey is
slow.

Specs: DES-OPENSHELL-VENUE (the CI bullet, with the residual that this is
the first model stream through OpenShell 0.1.4's rest layer).
UNCHANGED, checked: DES-STEER-VIA-READONLY-INBOX-MOUNT and
INT-STEER-INBOX-CONTRACT.

Signed-off-by: Rob Boerman <robboerman@live.nl>
…in's completion requests (#725)

The steering case's run ended exit 1 "Connection error." on every model call from the
copy-mode sandbox, while the policy probes' plain-node GETs to the same host answered.
No lane case had made a real provider call from inside OpenShell before (the cost cap
of 0 refused it first), so the difference between those probes and a job's call is
unmeasured. A printed, unasserted `netpath` probe now walks it one step at a time: the
image's node and the runner's own binary, a GET and a streamed POST, then the same after
pi is loaded (which replaces the fetch dispatcher, issue #427), then through the OpenAI
SDK as pi-ai calls it. On a steering failure the case also prints how many completion
requests reached the stand-in during that run (method and path only, never a body).

Test-only: no spec entry changes (DES-OPENSHELL-VENUE UNCHANGED, checked).
Signed-off-by: Rob Boerman <robboerman@live.nl>
…it, and print its exits (#725)

The first run printed no netpath line: the probe file sat at /, which the sandbox policy's read_only list does not include, so node could not load it, and its stderr was not shown. It now lives under /opt, and each binary's exit and the head of its stderr are printed. Test-only.

Signed-off-by: Rob Boerman <robboerman@live.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openshell lane: the copy-mode "mid-run second" steering assertion races the run's end

1 participant