[Plugin] run_code (PTC): programmatic tool calls — experimental, DANGEROUSLY_ gated

[eshwar-sundar-glean]

Status: experimental, off by default. The entire feature is gated behind the env flag DANGEROUSLY_ENABLE_UNSTABLE_RUN_CODE_FEATURE. With the flag off, the tool surface and behavior are byte-identical to what's deployed today (run_tool only). node:vm is not a security boundary yet — see Security & deferred work.

What this is

run_code (Programmatic Tool Calls / PTC) lets the host LLM write real JavaScript in which each Glean tool is an in-scope async function PTC_<TOOL_NAME>(args). The plugin intercepts each call, performs the real run_tool (or direct) MCP call in the trusted parent, and feeds the result back into the running cell. Loops, filtering, fan-out, and chaining happen in code, in one turn, with large payloads kept out of the model's context.

It's added alongside run_tool (not replacing it): run_tool for a single one-off call, run_code for batches (2+ calls / chaining / a loop).

The four runtime flows

Flow A — Discovery (find_skills in code mode → PTC_ bindings)

sequenceDiagram
    participant M as Model
    participant H as find_skills handler
    participant D as disk (skills cache)
    participant R as Glean gateway
    M->>H: find_skills({queries})
    H->>R: callRemoteTool("find_skills")
    R-->>H: skills + tool JSON
    H->>D: writeSkillsToDisk(skills)
    H->>D: writeCoreTools(headTools) → _core/tools/<name>.json (direct:true)
    H->>H: formatAvailableSkillsPrompt(index, {codeMode:true})
    H-->>M: <available_skills> + code-mode guide<br/>(read tool JSON → call PTC_<NAME>(args))
    Note over M,D: Each tool JSON becomes a PTC_<NAME> binding at run_code time<br/>(discoverTools reads the cache; head tools bind as direct PTC_search/etc.)

Loading

Flow B — Happy-path execution (single or batch)

sequenceDiagram
    participant M as Model
    participant H as run_code handler (parent)
    participant V as node:vm cell
    participant B as __ptcDispatch (host bridge)
    participant R as Glean gateway / remote

    M->>H: run_code({ code, reset? })
    H->>H: acquireLock (FIFO mutex)
    H->>H: ensureContext(reset) — persistent vm ctx + PREAMBLE (ToolResult/inspect/print)
    H->>H: writeCoreTools + discoverTools → toolsByName
    H->>H: scanReferencedTools(code); bindingsSource(names) + async-IIFE wrapper
    H->>V: vm.runInContext(script)  (Promise.race vs wall-clock timeout)
    V->>B: await PTC_FOO(args)
    B->>B: meta lookup · budget (MAX_CALLS) · deadline check
    alt meta.direct (head tool)
        B->>R: callRemoteTool(name, args)
    else skill tool
        B->>R: invokeTool → run_tool gateway
    end
    R-->>B: CallToolResult { content[].text, structuredContent? }
    B-->>V: { content, text, structured } → __mkResult → ToolResult (r)
    Note over V: r.json()/r.get(path,fb)/r.text/.format · inspect(r) for shape<br/>bare assignment (x = …) persists across run_code calls
    V-->>H: cell returns <value> (and/or print() → stdout)
    H->>H: value + stdout VERBATIM (host/harness handles oversized output)
    H-->>M: structuredContent + content[].text: { ok:true, value, stdout?, session? }

Loading

Flow C — Failure path (a failed PTC_ call throws)

sequenceDiagram
    participant M as Model
    participant H as run_code handler
    participant V as node:vm cell
    participant B as __ptcDispatch

    M->>H: run_code({ code })
    H->>V: run cell
    V->>B: await PTC_FOO(args)
    B->>B: tool isError? transport throw? approval declined?
    B-->>V: throw Error("PTC_FOO failed: <reason>")
    alt uncaught
        V-->>H: rejects with `PTC_FOO failed: <reason>`
        H->>H: read thrown message
        H-->>M: { ok:false, error:{ message: "PTC_FOO failed: <reason>" } }
    else caught (self-contained batch)
        Note over V: try { await PTC_FOO() } catch (e) { out.push({error: e.message}) }
        V-->>H: cell returns the report it built
        H-->>M: { ok:true, value: out }  ← failures captured IN the value
    end
    Note over B,H: No execution ledger. Failure rides the normal exception channel.<br/>Writes already made before a throw are NOT rolled back.

Loading

Flow D — Approval / HITL (ENABLE_HITL=true)

sequenceDiagram
    participant M as Model
    participant H as run_code handler
    participant E as elicitInput (host)
    participant V as node:vm cell
    participant B as __ptcDispatch

    M->>H: run_code({ code })
    H->>H: scan → needApproval (requires_approval, not already granted)
    opt needApproval && HITL && canElicit  (BEFORE the cell)
        H->>E: ONE bulk card listing all PTC_ tools that will run
        alt accept
            E-->>H: accept → grant for the session (sessionApproved)
        else decline / channel error
            E-->>H: decline → { ok:false, "Bulk approval declined; nothing ran." } (fail-closed)
        end
    end
    H->>V: run cell (only if not declined)
    V->>B: await PTC_DYNAMIC(args)  (a tool the static scan missed)
    opt requires_approval && not granted  (INSIDE the cell — JIT backstop)
        B->>E: requestToolApproval(failClosed:true)
        alt declined / elicitation-error
            B-->>V: throw "Approval declined for PTC_DYNAMIC."
        else accept
            E-->>B: granted (session)
        end
    end

Loading

Suggested review order (dependency order)

The engine file was split into focused modules; read in this order:

1. src/tools/run-code/limits.ts — env-overridable limits + shape tuning.
2. src/tools/run-code/shape.ts — shape inference (powers inspect).
3. src/tools/run-code/output.ts — value serialization (serialize), extractText, normalizeForSummary.
4. src/tools/run-code/envelope.ts — RunCodeEnvelope + assembly helpers.
5. src/tools/run-code/preamble.ts — the in-VM PREAMBLE (ToolResult/inspect/print), bindingsSource (PTC_ binding generator), scanReferencedTools.
6. src/tools/run-code.ts — the engine: persistent vm context, ensureContext, the __ptcDispatch bridge, handleRunCode orchestrator.
7. src/skill-tools.ts — discoverTools / findToolMeta / writeCoreTools (NEW; not used by main yet).
8. src/tools/run-tool.ts — additive invokeTool + requestToolApproval (used by run_code; run_tool's own path unchanged).
9. src/index.ts — RUN_CODE_TOOL def, RUN_CODE_ENABLED gate, registration + dispatch (all flag-gated).
10. src/tools/find-skills.ts + src/skill-writer.ts — additive codeMode branch (inert when off).
11. tests/run-code.test.ts — 22 tests incl. throw model, verbatim output, structuredContent, approval.

Result envelope

Emitted on both MCP channels: structuredContent (typed object) + the same JSON in content[].text. A loose outputSchema is declared on the tool. Values/stdout come back verbatim — no file redirection; the host/harness handles oversized output.

{ "ok": true|false,
  "value": <return value VERBATIM>,        // any JSON type; omitted on error
  "stdout": "...",                          // print() output; only when non-empty
  "session": { "fresh": true },             // only on a fresh/just-reset context
  "error": { "message": "PTC_X failed: …" } // only on a throw
}

Env flags

Flag	Default	Effect
DANGEROUSLY_ENABLE_UNSTABLE_RUN_CODE_FEATURE	off	Master gate. On → run_code is exposed alongside run_tool.
ENABLE_HITL	(repo default)	Gates bulk pre-scan + JIT approval.
GLEAN_PTC_TIMEOUT_MS	60000	Wall-clock per cell.
GLEAN_PTC_MAX_CALLS	200	Tool-call budget per cell.

Security & deferred work

• node:vm is not a security boundary. Injected host bridges are reachable from the cell; the sandbox shares the process (and the OAuth token on disk). Acceptable only because the feature is off by default + experimental — hence the DANGEROUSLY_/UNSTABLE flag name.
• Roadmap: worker/QuickJS-WASM isolation plus a parent-mediated, path-scoped readFile/writeFile that denies the token dir. Process isolation alone doesn't close token exfiltration; the capability boundary is the real fix.
• The parked fake-backend / e2e harness (gateway-free failure injection) lives on branch ptc-e2e-fake-backend.

🤖 Generated with Claude Code