Skip to content

refactor(api)!: prune internals and unify built-in grep - #5002

Merged
OwenKephart merged 3 commits into
v27/lab-fixesfrom
api-cleanup/surface-pruning
Oct 20, 2026
Merged

OwenKephart merged 3 commits into
v27/lab-fixesfrom
api-cleanup/surface-pruning

Conversation

@OwenKephart

@OwenKephart OwenKephart commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Authoring entries should not promise wire framing, loader recognition, or deprecated extension plumbing. Remove those exports, retire the code extension's GitHub compatibility path, and give authors importable callback and configuration types. Both grep mounts now use one bounded workspace search with explicit output modes. Audience callbacks keep only the privacy-limited caller projection. Related to #4930; stacked on #5000.

The grep default changes from content to matching file paths. Limits now bound returned rows, and paths cannot escape the workspace. Native ripgrep and just-bash use structured content output; native fallback uses its filename delimiter. just-bash now supplies filesystem-backed realpath resolution. Pipeline failures remain failures, including invalid regular expressions.

Disabled-marker types and guards remain public. GitHub credentials move to the git extension; the retired code config is rejected rather than ignored. Internal PR-watch callers supply their GitHub settings explicitly. Shared built-in implementation entries live under eve/internal/*.

Validation

  • eve package typecheck, JS build, and declaration emit passed.
  • 147 focused eve unit tests passed across nine files, including privacy projections and Teams sign-in authorization. The latter's stale session fake now implements the real stream contract.
  • All 105 code-extension tests and all 18 git-extension tests passed across their unit/integration/scenario tiers before the final native grep regression was added. The 22 grep scenarios then passed, including that new real-shell regression.
  • Two just-bash filesystem scenarios passed, covering workspace containment, search output, and real command failure propagation.
  • Code and git extension typechecks passed before the final grep scenario was added; the final checks are recorded in follow-up commits if needed.
  • All docs checks and the invariant guard passed. Public snapshots were regenerated. Published capability history remains unchanged; new extension/config epochs describe removed arity and additive named types.
  • The sandbox file-tool e2e prompt now requests content mode. E2E and full eve integration/scenario tiers were not run locally.

Review follow-through

Fixed the long-line regression: a capped JSON record retains a confirmed match and bounded preview, with an explicit truncated-position marker when needed. The strengthened native case fails before repair and passes afterward; it also compares context separators with native NUL framing. Home-scoped paths now fail immediately with workspace guidance. Retired the obsolete 29-case executor suite and test-only adapter; the canonical 23-case grep suite and two real just-bash filesystem cases pass. Stale code-extension, audience, and model-alias docs are updated. Restacked on reviewed #5000.

Checklist

  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

@OwenKephart
OwenKephart requested a review from a team as a code owner October 17, 2026 22:57
@vercel

vercel Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Ready Ready Preview, v0 Oct 18, 2026 1:54am UTC
eve-pkg Ready Ready Preview, v0 Oct 18, 2026 1:54am UTC

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs api-cleanup/owned-types (6a90b45).

Delta vs api-cleanup/owned-types (6a90b45)

Area Metric Baseline Current Delta
Package Packed tarball 7.68 MB 7.68 MB -1.2 kB ✅
Package Unpacked publish size 26.96 MB 26.96 MB -7.4 kB ✅
Package Installed footprint 75.10 MB 75.09 MB -7.4 kB ✅
Package Published files 4270 4274 +4
Package Installed files 8281 8285 +4
Package Installed package instances 34 34 0
Package Distinct installed package names 32 32 0
Package Installed dependency edges 51 51 0
Package Installed optional peer edges 9 9 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 21.03 MB 21.03 MB -2.9 kB ✅
Runtime Public routes 17 17 0
Changed function payloads vs api-cleanup/owned-types (6a90b45) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 10.52 MB 10.51 MB -1.5 kB ✅ none
functions/.well-known/workflow/v1/flow.func changed 10.52 MB 10.51 MB -1.5 kB ✅ none

eve init install

Metric Baseline Current Delta
Installed footprint 113.48 MB 113.48 MB -7.4 kB ✅
Installed packages 97 97 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 45.24 MB 45.23 MB -7.4 kB ✅
devDependency package bytes 5.11 MB 5.11 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260903-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-10-18T01:55:16.043Z
  • Route aliases: 17 public, 1 internal (18 total aliases)
  • Vercel routes in config: 20
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.79.2
  • Package directory: packages/eve
  • Tarball: 7.68 MB (eve-0.79.2.tgz)
  • Unpacked payload: 26.96 MB across 4274 published files
  • Installed footprint: 75.09 MB across 8285 installed files
  • Installed root package: 26.96 MB
  • Installed dependencies: 48.14 MB
  • Installed package instances: 34
  • Distinct installed package names: 32
  • Installed dependency edges: 51
  • Installed optional peer edges: 9
  • Runtime dependencies: 2
  • Peer dependencies: 7 (6 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

  • eve: 26.96 MB (35.9%)
  • @rolldown/binding-linux-x64-gnu: 19.81 MB (26.4%)
  • ai: 8.24 MB (11.0%)
  • zod: 6.15 MB (8.2%)
  • undici: 3.58 MB (4.8%)
Publish payload breakdown
Published file size
🟠 dist/src/compiled/shadcn-registry/index.js       [#####...................] 3.85 MB 14.3%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.29 MB 8.5%
🟠 dist/src/compiled/@ai-sdk/code-mode/index.js     [#.......................] 1.03 MB 3.8%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 604.9 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-D-I... [#.......................] 521.9 kB 1.9%
🔴 Other published files                            [########################] 18.67 MB 69.2%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 26.96 MB 35.9%
🔴 @rolldown/binding-linux-x64-gnu [##################......] 19.81 MB 26.4%
🔴 ai                              [#######.................] 8.24 MB 11.0%
🔴 zod                             [#####...................] 6.15 MB 8.2%
🟠 undici                          [###.....................] 3.58 MB 4.8%
🟠 nitro                           [##......................] 1.89 MB 2.5%
🔴 Other installed packages        [########................] 8.46 MB 11.3%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260903-beta
undici 8.10.2
Peer dependencies (7)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
chat ^4.42.0 optional peer
dd-trace ^6.13.0 optional peer
just-bash ^3.1.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 113.48 MB across 10202 installed files
  • Installed packages: 97 total (91 transitive-only)
  • dependencies: 4 direct packages totaling 45.23 MB
  • devDependencies: 2 direct packages totaling 5.11 MB
  • Other transitive package files: 63.14 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • @typescript/typescript-linux-x64: 27.95 MB (24.6%)
  • eve: 26.96 MB (23.8%)
  • @rolldown/binding-linux-x64-gnu: 19.81 MB (17.5%)
  • zod: 9.76 MB (8.6%)
  • ai: 8.24 MB (7.3%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 24.6%
🔴 eve                              [#######################.] 26.96 MB 23.8%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.81 MB 17.5%
🔴 zod                              [########................] 9.76 MB 8.6%
🔴 ai                               [#######.................] 8.24 MB 7.3%
🟠 undici                           [###.....................] 3.58 MB 3.2%
🔴 Other installed packages         [###############.........] 17.19 MB 15.1%
dependencies (4)
Package Range Installed size Share
@vercel/connect 2.5.2 276.3 kB 0.2%
ai ^7.0.137 8.24 MB 7.3%
eve file:eve-0.79.2.tgz 26.96 MB 23.8%
zod 4.5.4 9.76 MB 8.6%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.61 MB 2.3%
typescript 7.0.2 2.50 MB 2.2%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 10.51 MB 50.0%
🔴 functions/__server.func                         [########################] 10.51 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 10.51 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 10.51 MB
Function files 10.51 MB across 126 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 2.48 MB (23.6%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [############............] 2.48 MB 23.6%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 9.6%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 732.7 kB 7.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.7 kB 6.7%
🔴 Other bundled files                            [########################] 4.78 MB 45.4%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 16 public routes, 1 internal alias • 10.51 MB
Metric Value
Public routes /
/.well-known/workflow/v1/webhook/[token]
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[attemptId]/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
/eve/v1/session/[sessionId]/stubs
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 10.51 MB
Function files 10.51 MB across 126 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 2.48 MB (23.6%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [############............] 2.48 MB 23.6%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 9.6%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 732.7 kB 7.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.7 kB 6.7%
🔴 Other bundled files                            [########################] 4.78 MB 45.4%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 2.45 s -> 2.68 s (+229.0 ms) vs api-cleanup/owned-types (6a90b45).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `api-cleanup/owned-types (6a90b45)`
Phase Baseline Current Delta
extension.check 14.8 ms 16.4 ms +1.6 ms
project.resolve 4.3 ms 0.3 ms -4.0 ms
workspace.create 3.1 ms 0.4 ms -2.7 ms
host.prepare 577.7 ms 432.7 ms -145.0 ms
vercel.service-prefix.resolve 0.9 ms 0.9 ms 0.0 ms
nitro.create 401.0 ms 416.4 ms +15.4 ms
sandbox.prewarm 228.0 ms 618.1 ms +390.1 ms
nitro.cache.prepare 0.2 ms 0.2 ms 0.0 ms
nitro.prepare 0.5 ms 0.5 ms 0.0 ms
nitro.public-assets 0.6 ms 0.6 ms 0.0 ms
nitro.prerender 0.3 ms 0.3 ms 0.0 ms
nitro.bundle 1.09 s 1.08 s -3.2 ms
nitro.cache.write 0.2 ms 0.2 ms 0.0 ms
vercel.workflow-function.materialize 69.3 ms 55.5 ms -13.8 ms
agent-summary.emit 0.4 ms 0.5 ms +0.1 ms
connect-manifest.emit 0.2 ms 0.2 ms 0.0 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 24.3 ms 12.9 ms -11.4 ms
workspace.remove 1.5 ms 1.5 ms 0.0 ms

import type { GrepToolInput } from "#tools/provided/grep.js";
export type GrepInput = GrepToolInput;
export type GrepResult = GrepSearchResult;
export function executeGrepOnSandbox(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adapter now only forwards to executeGrepSearch, and its main consumer, packages/eve/test/grep-tool.test.ts, still tests the old contract. That file isn't touched by this PR and vitest.unit.config.ts still includes it (test/**/*.test.ts). It expects the default --max-count 100 and the 1000 cap. Its fake run also only handles the old ripgrep probe and $HOME probe, so the new realpath calls from resolveInWorkspace go to commandHandler and fail with "could not be resolved" before any search runs. I'd expect the eve unit tier to go red on this head. The PR body says that tier only ran on nine focused files.

Suggested fix: delete this file, point just-bash.scenario.test.ts at executeGrepSearch directly, and rewrite or retire test/grep-tool.test.ts against the new modes and limits.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retired the test-only adapter and its obsolete 29-case suite after reading all cases. The canonical 23-case suite covers the new modes/bounds, plus real native and virtual filesystem regressions. just-bash now calls the canonical executor directly.

parts.push("--count");
break;
case "content":
parts.push("--line-number", "--with-filename", "--json");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving content mode from --null to --json silently drops matches on long lines. Each JSON record carries the whole line, so any match line over the 256 KiB command cap (minified bundles, lockfiles, data dumps) becomes one partial record. The partial-record skip throws it away, and the caller gets a truncated notice with nothing from that file.

I reproduced this locally with rg 15.1 and a stubbed sandbox: a big.js holding one 300 KB line containing needle, plus a small.txt with two matches.

  • base (--null): content returns 3 matches, and big.js:1: comes back trimmed to the 8 KiB line bound
  • head (--json): content returns 2 matches with truncated: true, big.js is missing, while files_with_matches still lists it

JSON mode also drops rg's -- separators between context groups. That's minor, but it's another drift from the code-extension behavior this PR says it preserves.

One fix: cap each line inside rg (--max-columns with --max-columns-preview) so no single record can exceed the byte cap. Or keep --null framing on native rg and use JSON only where just-bash needs it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed confirmed-match loss at the byte cap: a partial JSON record retains a bounded preview and marks unavailable position metadata explicitly. The strengthened long-line case fails on the old parser and passes after repair. Context separators are reconstructed and compared against native NUL output.

"The directory or file to search in. Defaults to /workspace. " +
"Must be an absolute path or begin with $HOME/. Omit to use the default.",
"Directory or file inside the workspace. Defaults to the workspace root. " +
"Relative paths resolve within the workspace.",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropping resolveSandboxModelPath means built-in grep no longer accepts the $HOME/ paths that read_file and glob still advertise. For example, $HOME/.agents/skills/<skill> (MODEL_SKILL_ROOT in shared/skill-paths.ts) now goes to sandbox.resolvePath. On Vercel, resolveVercelSandboxPath turns that into /workspace/$HOME/.agents/skills/..., and realpath then fails with an opaque resolution error. Even when the path does resolve, skills under the real home sit outside the workspace, so containment rejects them. On base, the model could grep its skill files.

If workspace-only search is the intent, reject $HOME/ inputs with an error saying grep is limited to the workspace. Otherwise, expand $HOME/ first and allow the skill root explicitly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workspace-only search is intentional. $HOME paths now fail before any command with explicit workspace guidance, and the regression verifies that no sandbox command ran.

Comment thread packages/eve-code/README.md Outdated
@@ -7,7 +7,6 @@ It ships inside the `eve` package. This private `@eve/code` workspace package is
- `eve/extensions/code`: the extension
- `eve/extensions/code/sandbox`: sandbox bootstrap and credential helpers
- `eve/extensions/code/tools`: `apply_patch` and `grep` (`gh` is a deprecated re-export)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is now stale: eve/extensions/code/tools exports only apply_patch and grep (packages/eve/src/extensions/code/tools.ts), so gh is no longer a re-export. Line 25 of this README also still says code({ github }) contributes code__gh, its GitHub instructions, and code__pr, but the config is now .strict() and rejects github. Both should point readers to eve/extensions/git.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed both stale compatibility claims. The README now directs GitHub contributions to eve/extensions/git and describes only the remaining optional Vercel connector.

Audience callbacks take `AudienceContext` and its privacy-limited `caller`; `auth`,
`AudienceInput`, and `AudiencePrincipal` are removed. Event framing records and loader-only
guards are not public authoring APIs. `JsonValue`, `JsonObject`, `JsonArray`, and `JsonPrimitive`
have one public home, `eve/events`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section retires these APIs, but three published pages still document them as supported:

  • docs/code-extension.mdx:131-133: "Migrate from code({ github })" says the option is deprecated and still contributes code__gh, its instructions, and code__pr. It is now rejected by the strict schema.
  • docs/channels/custom.mdx:486-493: says AudienceInput<TState> "remains supported" and shows audience({ auth }). Neither exists after this PR.
  • docs/reference/typescript-api.md:148,185: lists experimental_chatgpt and says it "remains as a deprecated alias". eve/models/openai now exports only openai and chatgpt.

Update those pages to match, or point them at this section.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated all three published pages. They no longer teach the retired GitHub config, audience auth input, or experimental_chatgpt alias. Docs checks pass.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

eve benchmark

Model Dataset Harnesses Source
anthropic/claude-sonnet-5.5 (released 2026-09-28, knowledge 2026-06) swe-lean@v3 eve-code@95bf09f, opencode, pi, eve-code@2cc43ca, opencode@2cc43ca, pi@2cc43ca Run

Results

Correctness

Harness Position Score pass^5 Δ 95% CI
opencode (Winner) 1st 32/35 (91%) 71% ±0 —
eve-code@2cc43ca (Baseline) 2nd 30/35 (86%) 57% -5% (tie) -26% to +11%
eve-code@95bf09f (Current) 5th 18/35 (51%) 43% -40% -71% to -3%

95% CI: paired bootstrap over tasks; tie: the interval includes ±0, so this run cannot separate the row from the winner

All harnesses and tasks
Task eve-code@95bf09f opencode pi eve-code@2cc43ca opencode@2cc43ca pi@2cc43ca
cancel-async-tasks 5/5 4/5 5/5 5/5 2/5 5/5
customer-health-dossier 0/5 5/5 4/5 2/5 5/5 0/5
customer-sku-breakdown 0/5 5/5 1/5 5/5 5/5 2/5
git-multibranch 5/5 5/5 5/5 5/5 5/5 5/5
growth-driver-diagnosis 0/5 5/5 5/5 4/5 5/5 5/5
log-summary-date-ranges 5/5 5/5 5/5 5/5 5/5 5/5
sanitize-git-repo 3/5 3/5 4/5 4/5 3/5 3/5
Total 18/35 32/35 29/35 30/35 30/35 25/35

Latency

Harness Position Score Δ 95% CI
pi@2cc43ca (Baseline, Winner) 1st 15s ±0 —
eve-code@2cc43ca (Baseline) 2nd 15s ±0 (tie) -2s to +8s
eve-code@95bf09f (Current) 6th 14m30s +14m15s +14m14s to +19m20s

95% CI: paired bootstrap over tasks; tie: the interval includes ±0, so this run cannot separate the row from the winner

All harnesses and tasks
Task eve-code@95bf09f opencode pi eve-code@2cc43ca opencode@2cc43ca pi@2cc43ca
cancel-async-tasks 14m30s 14s 12s 13s 12s 14s
customer-health-dossier 19m30s 18s 17s 18s 21s 15s
customer-sku-breakdown 19m30s 16s 10s 15s 18s 11s
git-multibranch 14m30s 21s 18s 25s 31s 16s
growth-driver-diagnosis 19m30s 19s 18s 24s 18s 21s
log-summary-date-ranges 14m30s 12s 11s 12s 13s 10s
sanitize-git-repo 14m30s 16s 16s 15s 21s 17s
Median 14m30s 16s 16s 15s 18s 15s

Token usage

Harness Position Score Δ 95% CI
pi (Winner) 1st 854K↑ 51K↓ ±0 —
eve-code@2cc43ca (Baseline) 3rd 2.3M↑ 58K↓ +1.5M↑ +7K↓ +1.0M to +2.0M

eve-code@95bf09f reports no tokens; 95% CI: paired bootstrap over tasks on input + output; tie: the interval includes ±0, so this run cannot separate the row from the winner

All harnesses and tasks
Task eve-code@95bf09f opencode pi eve-code@2cc43ca opencode@2cc43ca pi@2cc43ca
cancel-async-tasks — 21K↑ 1K↓ 6K↑ 1K↓ 28K↑ 1K↓ 21K↑ 946↓ 6K↑ 1K↓
customer-health-dossier — 101K↑ 1K↓ 36K↑ 2K↓ 73K↑ 2K↓ 101K↑ 1K↓ 33K↑ 2K↓
customer-sku-breakdown — 114K↑ 947↓ 30K↑ 768↓ 88K↑ 1K↓ 115K↑ 934↓ 30K↑ 773↓
git-multibranch — 44K↑ 2K↓ 14K↑ 2K↓ 63K↑ 2K↓ 80K↑ 2K↓ 10K↑ 1K↓
growth-driver-diagnosis — 111K↑ 1K↓ 50K↑ 2K↓ 131K↑ 3K↓ 110K↑ 1K↓ 50K↑ 3K↓
log-summary-date-ranges — 46K↑ 933↓ 11K↑ 1K↓ 30K↑ 972↓ 35K↑ 961↓ 10K↑ 888↓
sanitize-git-repo — 46K↑ 1K↓ 20K↑ 1K↓ 43K↑ 1K↓ 94K↑ 2K↓ 17K↑ 1K↓
Total — 2.4M↑ 43K↓ 854K↑ 51K↓ 2.3M↑ 58K↓ 2.8M↑ 48K↓ 906K↑ 51K↓

Compared inputs

  • live harnesses only

Diagnostics

26 trials to inspect
  • eve-code/customer-health-dossier#1 (reward 0; lost to opencode, pi, eve-code@2cc43ca, opencode@2cc43ca):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-health-dossier
    
  • eve-code/customer-health-dossier#2 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-health-dossier --attempt 2
    
  • eve-code/customer-health-dossier#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-health-dossier --attempt 3
    
  • eve-code/customer-health-dossier#4 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-health-dossier --attempt 4
    
  • eve-code/customer-health-dossier#5 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-health-dossier --attempt 5
    
  • eve-code/customer-sku-breakdown#1 (reward 0; lost to opencode, pi, eve-code@2cc43ca, opencode@2cc43ca, pi@2cc43ca):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-sku-breakdown
    
  • eve-code/customer-sku-breakdown#2 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-sku-breakdown --attempt 2
    
  • eve-code/customer-sku-breakdown#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-sku-breakdown --attempt 3
    
  • eve-code/customer-sku-breakdown#4 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-sku-breakdown --attempt 4
    
  • eve-code/customer-sku-breakdown#5 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task customer-sku-breakdown --attempt 5
    
  • eve-code/growth-driver-diagnosis#1 (reward 0; lost to opencode, pi, eve-code@2cc43ca, opencode@2cc43ca, pi@2cc43ca):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task growth-driver-diagnosis
    
  • eve-code/growth-driver-diagnosis#2 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task growth-driver-diagnosis --attempt 2
    
  • eve-code/growth-driver-diagnosis#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task growth-driver-diagnosis --attempt 3
    
  • eve-code/growth-driver-diagnosis#4 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task growth-driver-diagnosis --attempt 4
    
  • eve-code/growth-driver-diagnosis#5 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task growth-driver-diagnosis --attempt 5
    
  • eve-code/sanitize-git-repo#2 (reward 0; lost to pi, eve-code@2cc43ca):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task sanitize-git-repo --attempt 2
    
  • eve-code/sanitize-git-repo#4 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness eve-code --task sanitize-git-repo --attempt 4
    
  • opencode/cancel-async-tasks#2 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness opencode --task cancel-async-tasks --attempt 2
    
  • opencode/sanitize-git-repo#2 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness opencode --task sanitize-git-repo --attempt 2
    
  • opencode/sanitize-git-repo#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness opencode --task sanitize-git-repo --attempt 3
    
  • pi/customer-health-dossier#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness pi --task customer-health-dossier --attempt 3
    
  • pi/customer-sku-breakdown#1 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness pi --task customer-sku-breakdown
    
  • pi/customer-sku-breakdown#2 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness pi --task customer-sku-breakdown --attempt 2
    
  • pi/customer-sku-breakdown#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness pi --task customer-sku-breakdown --attempt 3
    
  • pi/customer-sku-breakdown#5 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness pi --task customer-sku-breakdown --attempt 5
    
  • pi/sanitize-git-repo#3 (reward 0):
    eve-bench trials logs https://github.com/vercel/eve/actions/runs/39088079556 --harness pi --task sanitize-git-repo --attempt 3
    

More details on the workflow run

vercel Bot and others added 3 commits October 18, 2026 01:52
Signed-off-by: owenkephart <owen.kephart@vercel.com>

Co-Authored-By: owenkephart <owen.kephart@vercel.com>
Signed-off-by: owenkephart <owen.kephart@vercel.com>

Co-Authored-By: owenkephart <owen.kephart@vercel.com>
Signed-off-by: owenkephart <owen.kephart@vercel.com>

Co-Authored-By: owenkephart <owen.kephart@vercel.com>

@vercel vercel Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

docs/reference/cli.md still documents importing serializeModelInputSchema from eve/tools, but this PR removed that export from the public surface, leaving a broken public-API doc reference.

Fix on Vercel

Base automatically changed from api-cleanup/owned-types to v27/lab-fixes October 20, 2026 22:58
@OwenKephart
OwenKephart merged commit 95bf09f into v27/lab-fixes Oct 20, 2026
145 of 151 checks passed
@OwenKephart
OwenKephart deleted the api-cleanup/surface-pruning branch October 20, 2026 22:58
@OwenKephart

Copy link
Copy Markdown
Contributor Author

Recovery note: this PR was automatically marked merged when its commits were included in #4855's branch. That consolidation has been unwound. #4855 is v27-only again; this change is independently reviewable in #5144, based on #5143. The original merged state remains historical GitHub metadata. Nothing was merged into main by this recovery.

This branch was successfully deployed

2 active deployments
Preview – eve-pkg — 95bf09f1 Deployed Oct 18, 2026 by vercel[bot]
Preview – eve-docs — 95bf09f1 Deployed Oct 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant