RFC: bounded, cancellable, stream-preserving execution results

[vulragrag-star]

RFC: bounded, cancellable, stream-preserving execution results

Motivation

The persistent PowerShell host delivered the intended latency win, but the current one-line response frame now crosses several boundaries that were not exercised by the existing suite:

1. Native stderr is outside the frame. [Console]::SetError() captures PowerShell/.NET writes, but stderr from git, npm, node, etc. still reaches the host process stderr pipe. Node appends those bytes to stderrChunks and never consumes them. A 256 KiB native stderr probe returned { stderr: "", exitCode: 7 } while all 256 KiB remained retained after later commands.
2. One result is unbounded. PowerShell buffers both streams in memory, base64-encodes them, and sends one JSON line. A 9 MB MCP result succeeded; an 11 MB result exceeded the SDK's 10 MiB read buffer and closed the entire connection.
3. Cancellation is not execution cancellation. The MCP SDK provides extra.signal, but the handler ignores it. Cancelling sleep 3 returned to the client at ~204 ms while the command kept the session lock; the next call waited ~2.95 s.
4. Lifecycle operations race. Three concurrent resets produced three live PowerShell hosts. Closing stdio does not dispose the session, so the server can hang and leave host/cwd/env files behind.
5. The MCP result loses structure. stdout, stderr, and exit status are flattened into one text block; whitespace-only output is dropped, and infrastructure failures are indistinguishable from normal shell exit codes.

These are one architectural cluster: framing, limits, cancellation, and lifecycle need one explicit contract.

Constraints

• Windows PowerShell 5.1 remains the execution baseline.
• One persistent host per FauxnixSession remains the latency model.
• Existing MCP clients must keep receiving readable text content during migration.
• The stdio transport has a finite frame limit and cannot be treated as an unbounded byte pipe.
• Timeout/cancellation must terminate the owned process tree or explicitly report any weaker guarantee.

Proposed host protocol v2

Version the handshake and use typed frames rather than one unbounded response line:

{"v":2,"type":"ready","capabilities":{"cancel":true,"maxChunkBytes":65536}}
{"v":2,"type":"run","id":"r1","scriptB64":"...","env":{},"stdoutLimit":8388608,"stderrLimit":1048576}
{"v":2,"type":"stdout","id":"r1","seq":0,"dataB64":"..."}
{"v":2,"type":"stderr","id":"r1","seq":0,"dataB64":"..."}
{"v":2,"type":"end","id":"r1","exitCode":0,"timedOut":false,"cancelled":false,"truncated":false}
{"v":2,"type":"cancel","id":"r1"}

Required behavior:

• bounded chunks with backpressure and separate stdout/stderr accounting
• per-request byte budgets with an explicit truncation marker; never sever the MCP connection because one command is verbose
• request IDs on every frame; strict schema/sequence validation
• a cancellation path wired from RequestHandlerExtra.signal
• one serialized lifecycle lock covering run/reset/dispose
• idempotent cleanup on stdio EOF, transport close, SIGINT, and SIGTERM
• a documented process-tree termination guarantee (Windows Job Object with kill-on-close is the strongest option)

Native stderr needs an explicit boundary marker or launcher mechanism; consuming whatever happens to have arrived when the stdout JSON line is read is not deterministic across two OS pipes.

MCP result contract

During migration, return both the current text content and versioned structured content:

{
  "schemaVersion": 1,
  "stdout": "...",
  "stderr": "...",
  "exitCode": 0,
  "timedOut": false,
  "cancelled": false,
  "truncated": false,
  "sessionId": "..."
}

Normal shell nonzero exits should remain command results, not automatically become MCP protocol errors. Host startup failure, malformed frames, and a dead transport should be marked as infrastructure errors.

Migration

1. Add output budgets/truncation metadata and structured MCP results while keeping the v1 host frame.
2. Serialize reset/dispose and clean up on EOF/signals.
3. Wire AbortSignal; initially cancellation may cold-restart the host, matching current timeout recovery.
4. Introduce the v2 handshake and chunked frames, including deterministic native stderr capture.
5. Add process-tree ownership and then optional fine-grained runspace cancellation.

Acceptance criteria

• native stderr is returned exactly once and retained buffers return to baseline after every request
• results above the configured budget truncate explicitly without closing MCP
• cancellation unblocks the next request within a bounded interval and prevents later list segments
• concurrent reset/run/close leaves exactly zero orphan hosts after disposal
• whitespace-only stdout remains byte-faithful
• protocol and structured-result tests run through the official MCP client

Non-goals

• Linux/macOS execution
• turning fauxnix into a security sandbox
• changing bash's normal exit-code semantics
• requiring every MCP client to consume streaming chunks directly

This RFC complements roadmap B3/B4 but makes the wire/result contract explicit before the v1 interface freeze.

[20000419]

Accepted — this becomes the v0.8 anchor (slotted into the roadmap doc alongside #130). The framing diagnosis matches everything we measured: the one-line response frame was a spawn-era convenience that the persistent host outgrew.

Review notes for the implementation wave:

1. Versioned handshake + capability negotiation ({"v":2,...,"capabilities":{...}}) — yes, and keep v1 accepted during migration so fauxnix translate and any pinned old clients keep working.
2. Chunked frames with per-stream seq — yes. Define the merge policy for interleaving explicitly (agents read combined text today): stdout/stderr seq per stream, Node-side ordered merge for the MCP text block, exact bytes available on request. The audit's "whitespace-only output is dropped" belongs in the same fix.
3. Result limits: stdoutLimit/stderrLimit per run are right; when truncated, the result must say so explicitly (marker + byte counts), never silently.
4. Cancellation: extra.signal → per-frame runspace Stop() (finally the B3 lever), then a health check before reuse — a stopped frame may leave the host in an unknown state; cold-restart on any doubt. The audit's "cancel returned in 204ms but kept the lock" is the exact contract to kill.
5. Lifecycle: all of run/reset/dispose behind the existing session lock (concurrent reset = three hosts is a plain race); register stdio-close → dispose so the server never hangs; sweep temp cwd/env/host files in dispose.
6. Contract tests: official MCP client, 11MB probe, concurrent reset, cancel-then-next-call latency — adopt the audit's reproductions verbatim as the test plan.

Ship it as an RFC doc in docs/ (like the persistent-host one) when you start; that doc's failure-mode table pattern worked well.

[r3wretrhy]

First slice merged (#138): RFC, structured MCP results, extra.signal cancel (kill+cold-start, exit 130), serialized run/reset/dispose, native stderr drain, whitespace-only stdout.

Still open (PR-B): chunked host protocol v2 handshake, per-stream seq frames, stdoutLimit/stderrLimit + truncated, and FAUXNIX_ERR_END so native stderr returns exactly once. Working that next.

[r3wretrhy]

PR-B is up: #142 — v2 handshake, chunked stdout/stderr + end, FAUXNIX_ERR_END native-stderr marker. Cancel remains kill+cold-start (capabilities.cancel: false).