perf: agent SDK snapshot wiring + bench harness + preview secure-exec dep

.github/workflows/publish.yaml

@@ -112,6 +112,12 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-${{ matrix.target }}-rusty-v8-
       - run: pnpm install --frozen-lockfile
+      # crates.io has no preview track, so a preview pin builds the sidecar
+      # against secure-exec cloned at the pinned commit (no-op for releases).
+      - name: Prepare secure-exec crates
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: node scripts/secure-exec-dep.mjs prepare-build
       - name: Build sidecar binary
         id: build
         run: |
@@ -174,6 +180,12 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-${{ matrix.target }}-rusty-v8-
       - run: pnpm install --frozen-lockfile
+      # crates.io has no preview track, so a preview pin builds the plugin
+      # against secure-exec cloned at the pinned commit (no-op for releases).
+      - name: Prepare secure-exec crates
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: node scripts/secure-exec-dep.mjs prepare-build
       - name: Build plugin cdylib
         id: build
         run: |

CLAUDE.md

@@ -6,7 +6,7 @@ Agent OS is the agent-facing wrapper around secure-exec. It provides ACP session
 
 - secure-exec dependency workflow. Manage the secure-exec dependency ONLY through `scripts/secure-exec-dep.mjs` (the `just secure-exec-*` recipes); never hand-edit the `path` / `version` / `catalog:` pins.
   - Testing against local secure-exec changes: run `just secure-exec-local` to repoint npm (`link:`) and crates (`path = "../secure-exec/..."`) at the sibling checkout, then `node scripts/secure-exec-dep.mjs set-crate-version <sibling-version>` so the Cargo version requirement matches the sibling crate version (otherwise cargo cannot resolve the path deps). Also run `pnpm install` in `../secure-exec` first, or cargo panics in `v8-runtime/build.rs` with "missing Node dependencies at .../packages/build-tools/node_modules" (the V8 bridge assets are built from there). Use `just secure-exec-status` to inspect. This mode is for local builds/tests ONLY.
-  - Pushing changes that depend on secure-exec changes: NEVER push with local (`path:` / `link:`) dependencies. First preview-publish the secure-exec changes to their own secure-exec branch (the `preview-publish-secure-exec` flow), then point agent-os back at that exact published version with `just secure-exec-pinned` + `just secure-exec-set-version <version>` (and `set-crate-version <version>` for the crates). Only commit/push the pinned-to-remote state.
+  - Pushing changes that depend on secure-exec changes: NEVER push with local (`path:` / `link:`) dependencies — this rule still holds. First preview-publish the secure-exec changes to their own secure-exec branch (the `preview-publish-secure-exec` flow), then point agent-os back at that exact published version with `just secure-exec-pinned` + `just secure-exec-set-version <version>`. Only commit/push the pinned state. Note the committed `Cargo.toml`/`Cargo.lock` stay pinned to a **crates.io** version (release-clean), NOT to a path/clone: a preview pin keeps the crate version at the last crates.io release and the secure-exec *crate* changes are picked up at CI build time by `prepare-build`, which clones secure-exec at the pinned `<sha>` and builds cargo in local mode (crates.io has no preview track). So you never commit a local path dep, yet a preview's crate changes still build. See "Depending on unreleased secure-exec changes".
 - Keep generic runtime, kernel, VFS, language execution, and registry software behavior in secure-exec.
 - Agent OS owns ACP, sessions, agent adapters, toolkit semantics, quickstarts, and the AgentOs facade.
 - Call OS instances VMs, never sandboxes.
@@ -17,17 +17,21 @@ Agent OS is the agent-facing wrapper around secure-exec. It provides ACP session
 ### secure-exec dependency versions (`just`)
 
 Two independent version tracks:
-- **secure-exec** — the `@secure-exec/*` npm packages and the `secure-exec-*` Cargo crates always share **one** version (npm and crates are kept in sync; pin both to the same `<v>`).
+- **secure-exec** — the `@secure-exec/*` npm packages and the `secure-exec-*` Cargo crates share **one** version on a real **release** (npm + crates publish together). On a **preview** they diverge: npm pins to the `0.0.0-<branch>.<sha>` preview tag while the crate version requirement stays at the last crates.io release (crates.io has no preview track), and CI's `prepare-build` clones secure-exec at `<sha>` to build the crates. See "Depending on unreleased secure-exec changes".
 - **`@agentos-software/*`** software packages (registry agents / WASM commands) are on a **separate** track and version independently of secure-exec.
 
 Manage them ONLY via these recipes (never hand-edit `path`/`version`/`catalog:` pins):
 - `just secure-exec-local` — point deps at the sibling `../secure-exec` checkout for local hacking.
-- `just secure-exec-set-version <v>` — pin secure-exec to a published version: sets the `@secure-exec/*` npm packages **and** the `secure-exec-*` crates (same `<v>`, they're in sync) and switches to pinned mode.
+- `just secure-exec-pinned` — switch deps back to pinned (published) mode without changing the pinned versions.
+- `just secure-exec-status` — show the current dep mode + pinned npm/crate versions.
+- `just secure-exec-set-version <v>` — pin secure-exec to a published version and switch to pinned mode. For a **release** `<v>` it pins both the `@secure-exec/*` npm packages and the `secure-exec-*` crates to `<v>`; for a **preview** `<v>` it pins only npm to the tag and leaves the crate version at the last crates.io release (crates come from the `prepare-build` clone).
 - `just agentos-pkgs-set-version <v>` — pin the `@agentos-software/*` software packages (separate version track).
 
 ### Depending on unreleased secure-exec changes
 
-agent-os builds against secure-exec crates + npm packages, so a secure-exec change must reach agent-os before it can be pushed. NEVER push with local (`path:`/`link:`) deps. Flow: preview-publish the secure-exec branch (the `preview-publish-secure-exec` skill), then `just secure-exec-set-version <published-version>` (pins npm + crates + switches to pinned mode), and push only that pinned state. Caveat: a preview publishes npm but the crates.io job is dry-run/skipped — a secure-exec *crate* change only flows locally (`secure-exec-local`) or via a real crates.io release.
+agent-os builds against secure-exec crates + npm packages, so a secure-exec change must reach agent-os before it can be pushed. NEVER push with local (`path:`/`link:`) deps. Flow: preview-publish the secure-exec branch (the `preview-publish-secure-exec` skill), then `just secure-exec-set-version <published-version>` (pins npm to the preview tag and switches to pinned mode; for a preview the crate version stays at the last crates.io release and crate changes flow via the `prepare-build` clone, while a release pins npm + crates together), and push only that pinned state.
+
+How crate changes flow on a preview (crates.io has no preview track): npm has dist-tags, so a secure-exec preview publishes the `@secure-exec/*` packages under a branch tag with version `0.0.0-<branch>.<sha>` (the `<sha>` is the secure-exec commit it was cut from). crates.io has no equivalent non-prod track, so secure-exec only publishes **crates** on real rc/releases — its publish workflow skips the crates.io job for previews. To still build the agent-os sidecar (a Rust binary embedding the secure-exec crates) against an unreleased secure-exec, the build **clones secure-exec at the pinned `<sha>` and compiles cargo in local path-dep mode** against that clone. This is automated: `node scripts/secure-exec-dep.mjs prepare-build` (run by every `cargo build` job in `.github/workflows/publish.yaml` and `docker/build/darwin.Dockerfile`) reads the pinned `@secure-exec/core` version — for a preview it clones `rivet-dev/secure-exec` (public, no token) at the sha into the sibling `../secure-exec` and flips cargo local; for a release pin it is a no-op and cargo resolves crates from crates.io. The **committed** `Cargo.toml` therefore always stays pinned to a crates.io version (release-clean and pushable); the preview clone happens only in CI at build time. So a preview-published secure-exec crate change DOES flow into an agent-os preview build — you no longer need a real crates.io release just to validate crate changes in a preview.
 
 ### Preview-publishing agent-os
 

crates/agentos-sidecar/src/acp_extension.rs

@@ -182,6 +182,7 @@ impl AcpExtension {
         mut ctx: ExtensionContext<'_>,
         request: AcpCreateSessionRequest,
     ) -> AcpHandlerOutput {
+        let __t0 = Instant::now();
         let process_id = self.allocate_process_id("acp-agent");
         let mut args = request.args.clone();
         let mut env = hash_to_btree(request.env.clone());
@@ -195,6 +196,7 @@ impl AcpExtension {
         {
             return AcpHandlerOutput::response(Err(error));
         }
+        tracing::info!(target: "agentos_sidecar::perf", phase = "prompt_injection", elapsed_ms = __t0.elapsed().as_millis() as u64, "create_session phase");
 
         let started = match ctx
             .spawn_process_wire(ExecuteRequest {
@@ -212,10 +214,12 @@ impl AcpExtension {
             Ok(started) => started,
             Err(error) => return AcpHandlerOutput::response(Err(error)),
         };
+        tracing::info!(target: "agentos_sidecar::perf", phase = "spawn_process", elapsed_ms = __t0.elapsed().as_millis() as u64, "create_session phase");
 
         let bootstrap = self
             .create_session_inner(&mut ctx, &request, &process_id)
             .await;
+        tracing::info!(target: "agentos_sidecar::perf", phase = "session_inner_done", elapsed_ms = __t0.elapsed().as_millis() as u64, "create_session phase");
         if bootstrap.is_err() {
             kill_process_best_effort(&mut ctx, &process_id).await;
         }
@@ -288,6 +292,7 @@ impl AcpExtension {
         request: &AcpCreateSessionRequest,
         process_id: &str,
     ) -> Result<CreateSessionBootstrap, SidecarError> {
+        let __ti = Instant::now();
         let mut stdout = String::new();
         let mut notifications = Vec::new();
         let client_capabilities =
@@ -317,6 +322,7 @@ impl AcpExtension {
         notifications.extend(initialize_response.notifications);
         let init_result = response_result(initialize_response.response, "ACP initialize")?;
         validate_initialize_result(&init_result, request.protocol_version)?;
+        tracing::info!(target: "agentos_sidecar::perf", phase = "acp_initialize", elapsed_ms = __ti.elapsed().as_millis() as u64, "create_session_inner phase");
 
         let session_new = json!({
             "jsonrpc": "2.0",
@@ -341,6 +347,7 @@ impl AcpExtension {
         notifications.extend(session_response.notifications);
         let session_result = response_result(session_response.response, "ACP session/new")?;
         let session_id = session_id_from_session_result(&session_result, process_id);
+        tracing::info!(target: "agentos_sidecar::perf", phase = "acp_session_new", elapsed_ms = __ti.elapsed().as_millis() as u64, "create_session_inner phase");
 
         let mut config_options = init_result
             .get("configOptions")

crates/agentos-sidecar/src/main.rs

@@ -3,6 +3,7 @@ use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt, EnvFilte
 
 fn main() {
     init_tracing();
+    tracing::info!(target: "secure_exec_sidecar::perf", "sidecar process started");
     if let Err(error) =
         secure_exec_sidecar::stdio::run_with_extensions(agentos_sidecar_wrapper::extensions())
     {

crates/client/src/agent_os.rs

@@ -870,6 +870,12 @@ fn serialize_create_vm_config_for_sidecar(
                 platform: vm_config::JsRuntimePlatform::default(),
                 module_resolution: vm_config::JsModuleResolution::default(),
                 allowed_builtins: Some(allowed.clone()),
+                // Agent SDK snapshotting is driven by the TypeScript client
+                // (`packages/core` resolves the per-agent `dist/sdk-snapshot.js`
+                // bundle). The Rust client does not resolve npm package bundles, so
+                // it forwards no snapshot. TODO: expose a snapshot bundle input on
+                // the Rust client config for parity if a Rust consumer needs it.
+                snapshot_userland_code: None,
             }
         }),
     })

docker/build/darwin.Dockerfile

@@ -24,6 +24,11 @@ RUN rustup toolchain install stable --profile minimal && \
 RUN corepack enable && \
     pnpm install --frozen-lockfile
 
+# crates.io has no preview track: a secure-exec preview pin builds the crates
+# from a clone at the pinned commit (../secure-exec == /secure-exec here);
+# a release pin is a no-op and resolves crates from crates.io.
+RUN node scripts/secure-exec-dep.mjs prepare-build
+
 RUN tu=$(echo "$TARGET" | tr 'a-z-' 'A-Z_') && \
     tl=$(echo "$TARGET" | tr - _) && \
     export BINDGEN_EXTRA_CLANG_ARGS_${tl}="--sysroot=$SDK -isystem $SDK/usr/include" && \

justfile

@@ -10,13 +10,24 @@ preview-publish REF:
 secure-exec-local:
 	node scripts/secure-exec-dep.mjs local
 
-# Pin secure-exec to a published version. The @secure-exec/* npm packages and the
-# secure-exec-* crates are always the same version, so this sets both and switches
-# to pinned mode.
+# Switch deps back to pinned (published) mode without changing the pinned versions.
+secure-exec-pinned:
+	node scripts/secure-exec-dep.mjs pinned
+
+# Show the current dep mode + the pinned npm/crate versions.
+secure-exec-status:
+	node scripts/secure-exec-dep.mjs status
+
+# Pin secure-exec to a published version and switch to pinned mode.
+#   Release <v>  -> @secure-exec/* npm AND secure-exec-* crates both pin to <v>
+#                   (a release publishes npm + crates together).
+#   Preview <v>  -> @secure-exec/* npm pin to the 0.0.0-<branch>.<sha> tag; the
+#                   crate version is left at the last crates.io release because
+#                   crates.io has no preview track. CI's `prepare-build` clones
+#                   secure-exec at <sha> and builds the crates from that clone.
 secure-exec-set-version VERSION:
 	node scripts/secure-exec-dep.mjs pinned
-	node scripts/secure-exec-dep.mjs set-secure-exec-version "{{ VERSION }}"
-	node scripts/secure-exec-dep.mjs set-crate-version "{{ VERSION }}"
+	node scripts/secure-exec-dep.mjs pin-secure-exec "{{ VERSION }}"
 
 # Pin the @agentos-software/* software packages (separate version track).
 agentos-pkgs-set-version VERSION:

package.json

@@ -15,11 +15,13 @@
 		"check-types": "npx turbo check-types --concurrency=1",
 		"lint": "pnpm biome check .",
 		"fmt": "pnpm biome check --write --diagnostic-level=error .",
-		"shell": "pnpm --filter @rivet-dev/agentos-shell shell"
+		"shell": "pnpm --filter @rivet-dev/agentos-shell shell",
+		"bench": "bash scripts/benchmarks/run-benchmarks.sh"
 	},
 	"devDependencies": {
 		"@biomejs/biome": "^2.3",
 		"@copilotkit/llmock": "^1.6.0",
+		"@mariozechner/pi-coding-agent": "^0.60.0",
 		"@rivet-dev/agentos-core": "workspace:*",
 		"@agentos-software/claude-code": "catalog:",
 		"@agentos-software/codex": "catalog:",
@@ -37,7 +39,9 @@
 	},
 	"pnpm": {
 		"overrides": {
-			"@rivetkit/rivetkit-wasm": "2.3.2"
+			"@rivetkit/rivetkit-wasm": "2.3.2",
+			"@mariozechner/pi-tui": "0.60.0",
+			"@mariozechner/pi-agent-core": "0.60.0"
 		}
 	}
 }

packages/core/src/agent-os.ts

@@ -211,6 +211,7 @@ import {
 import {
 	type CommandPackageMetadata,
 	processSoftware,
+	resolveAgentSnapshotBundle,
 	resolvePackageDir,
 	type SoftwareInput,
 	type SoftwareRoot,
@@ -2741,6 +2742,9 @@ export class AgentOs {
 				? (options.software ?? [])
 				: [commonSoftware, ...(options?.software ?? [])];
 		const processed = processSoftware(software);
+		// Agent-SDK snapshot bundle (loaded once per sidecar into the V8 startup
+		// snapshot, reused across sessions) for any snapshot-enabled agent.
+		const snapshotUserlandCode = resolveAgentSnapshotBundle(software);
 		const localMounts = await resolveCompatLocalMounts(options?.mounts);
 		const toolKits = options?.toolKits;
 		if (toolKits && toolKits.length > 0) {
@@ -2835,12 +2839,18 @@ export class AgentOs {
 					// module resolution keep their engine defaults (full Node
 					// emulation), matching the prior behavior where Agent OS only
 					// constrained the builtin allow-list.
-					...(options?.allowedNodeBuiltins !== undefined
+					...(options?.allowedNodeBuiltins !== undefined ||
+					snapshotUserlandCode !== undefined
 						? {
 								jsRuntime: {
 									platform: "node" as const,
 									moduleResolution: "node" as const,
-									allowedBuiltins: options.allowedNodeBuiltins,
+									...(options?.allowedNodeBuiltins !== undefined
+										? { allowedBuiltins: options.allowedNodeBuiltins }
+										: {}),
+									...(snapshotUserlandCode !== undefined
+										? { snapshotUserlandCode }
+										: {}),
 								},
 							}
 						: {}),