Skip to content

test(eve): cover v27 sessions through their odd paths - #4726

Open
OwenKephart wants to merge 7 commits into
v27/observable-sessionfrom
v27/coverage
Open

OwenKephart wants to merge 7 commits into
v27/observable-sessionfrom
v27/coverage

Conversation

@OwenKephart

@OwenKephart OwenKephart commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The v27 migration needs proof that committed facts, resumed execution, and rendered results agree through failure and suspension. This PR exercises those boundaries and retains the runtime fixes found there, including cancellation during admission and instrumentation hearing session creation. It now also tests real idle/stale cancellation receipts and schema-4 delivery attribution through eve's OTel provider. Stacked on #4854; #4855 follows. Related to #4930.

Coverage includes steering, approvals, child relays, interrupted sign-ins, compaction, call-ID collisions, and eval ordering/accounting. A cancellation after admission consumes the selected message into its cancelled turn, so retained history and the stream agree. Hook cancellation does not escape admission and retry that message.

Scope boundaries:

  • Failure placement, child-delivery attribution, shared activity selection, and acceptance-aware frontend tests belong to refactor(eve)!: let the session stream own the frontend lifecycle #4854; the child-delivery golden integration case remains here.
  • Repeated-call-ID protection covers IDs retained in the operational view or model prompt, not IDs discarded by retention or clear/compaction.
  • Native fact-to-span tests replace relevant old tracing coverage; this does not restore every deleted tracing suite.
  • The existing expected failure remains: a failed session closes its open turn as cancelled, so the turn span has no session error.
  • Reactions feat(eve)!: compute dynamic capabilities with functions and memos #4616 is untouched. Most release notes remain collected here; the cancellation and instrumentation notes now accompany their own lower PRs.

It also checks the stream's promises on random streams, and logs a turn step that disagrees with waiting() under EVE_CHECK_SESSION_EVENTS. Waits are announced from the tables alone, and only the writer closes and scopes facts.

Production code: 524 more lines (+729/−205). Growth comes from runtime fixes uncovered by coverage: Teams' private sign-in reveal, call-ID collision protection, and cancellation-safe publication. The dev-only step-decision check adds 41 lines; announcing from the tables and one cascade site remove 10. Excludes tests/fixtures, generated copies, docs, and tooling.

Validation

At assembled head b1c123f55 (runtime unchanged from the validated D8 snapshot):

  • Sixteen explicitly selected unit files: 269 passed, one pre-existing expected failure, in 5.15 seconds. No whole test tier ran.
  • Package TypeScript check/declaration build, docs, invariant guard, targeted lint, formatting, and diff checks passed.
  • Unknown-tool/sign-in and task-cancel-origin regressions fail on the prior owners, then pass with the fixes.
  • pnpm --filter eve build:js resolved the earlier config-import blocker. Two real idle/stale receipt cases pass (17.94 seconds) and fail against the prior owners on absent settlements. The actual SDK retry/per-call metadata case passes (11.11 seconds), as does the concurrent-approval requester case (16.44 seconds).
  • Raw workflow fixtures now supply dispatch identities at their sender boundaries, including their initial contexts. They do not inject admission or settlement facts. The stranded-reset assertions remain CI-owned.
  • No full integration/scenario/e2e or browser run. Capability reports were not regenerated during this restack; final generation is coordinated against the assembled cleanup source.

Latest type-only follow-through at assembled head 390f9e5b3: AgentMessageResult.error.message
is restored to string. Package typecheck/declaration build, targeted lint, formatting, and diff
checks pass. Runtime tests were not rerun for this annotation-only correction. The invariant guard
reports only the unreleased tool/v93 contract hash; generation remains deferred to the agreed
final assembled-source pass. No reports or baselines were rewritten.

Follow-through at 77e487f18: four commits announce waits from the tables alone, cascade only in the writer, log step decisions that disagree with waiting(), and check the stream's promises on random streams. The step check labels deferred dispatch and doesn't log it. The property test runs 20 seeded streams in the unit tier (about 0.6 seconds), and EVE_STREAM_PROPERTY_RUNS raises the count. It fails if the reply retention change in #4567 is reverted. At this head, the package typechecks, and src/protocol, src/harness, and src/execution unit tests have one pre-existing failure (v26-transcript-lines). The scoped integration suites show the same 4 failures with and without the commits, and no new checker rule fires. At #4855's head, lint, formatting, and pnpm docs:check pass, and the invariant guard reports only the existing tool/v93 hash.

Checklist

  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

@OwenKephart
OwenKephart requested a review from a team as a code owner October 13, 2026 18:37
@vercel

vercel Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Error Error v0 Oct 20, 2026 5:34pm UTC
eve-pkg Ready Ready Preview, v0 Oct 20, 2026 5:34pm UTC

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs v27/observable-session (67e1dc2).

Delta vs v27/observable-session (67e1dc2)

Area Metric Baseline Current Delta
Package Packed tarball 7.65 MB 7.65 MB +286 B ⚠️
Package Unpacked publish size 26.90 MB 26.90 MB +4.1 kB ⚠️
Package Installed footprint 74.97 MB 74.98 MB +4.1 kB ⚠️
Package Published files 4198 4200 +2
Package Installed files 8207 8209 +2
Package Installed package instances 34 34 0
Package Distinct installed package names 32 32 0
Package Installed dependency edges 51 51 0
Package Installed optional peer edges 9 9 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 24.53 MB 24.54 MB +9.4 kB ⚠️
Runtime Public routes 17 17 0
Changed function payloads vs v27/observable-session (67e1dc2) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 12.27 MB 12.27 MB +4.7 kB ⚠️ none
functions/.well-known/workflow/v1/flow.func changed 12.27 MB 12.27 MB +4.7 kB ⚠️ none

eve init install

Metric Baseline Current Delta
Installed footprint 113.26 MB 113.26 MB +4.1 kB ⚠️
Installed packages 97 97 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 45.04 MB 45.04 MB +4.1 kB ⚠️
devDependency package bytes 5.11 MB 5.11 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260903-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-10-14T23:56:15.143Z
  • Route aliases: 17 public, 1 internal (18 total aliases)
  • Vercel routes in config: 20
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.79.1
  • Package directory: packages/eve
  • Tarball: 7.65 MB (eve-0.79.1.tgz)
  • Unpacked payload: 26.90 MB across 4200 published files
  • Installed footprint: 74.98 MB across 8209 installed files
  • Installed root package: 26.90 MB
  • Installed dependencies: 48.08 MB
  • Installed package instances: 34
  • Distinct installed package names: 32
  • Installed dependency edges: 51
  • Installed optional peer edges: 9
  • Runtime dependencies: 2
  • Peer dependencies: 7 (6 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

  • eve: 26.90 MB (35.9%)
  • @rolldown/binding-linux-x64-gnu: 19.81 MB (26.4%)
  • ai: 8.20 MB (10.9%)
  • zod: 6.15 MB (8.2%)
  • undici: 3.56 MB (4.8%)
Publish payload breakdown
Published file size
🟠 dist/src/compiled/shadcn-registry/index.js       [#####...................] 3.85 MB 14.3%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.29 MB 8.5%
🟠 dist/src/compiled/@ai-sdk/code-mode/index.js     [#.......................] 1.03 MB 3.8%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 604.9 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-D-I... [#.......................] 521.9 kB 1.9%
🔴 Other published files                            [########################] 18.61 MB 69.2%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 26.90 MB 35.9%
🔴 @rolldown/binding-linux-x64-gnu [##################......] 19.81 MB 26.4%
🔴 ai                              [#######.................] 8.20 MB 10.9%
🔴 zod                             [#####...................] 6.15 MB 8.2%
🟠 undici                          [###.....................] 3.56 MB 4.8%
🟠 nitro                           [##......................] 1.89 MB 2.5%
🔴 Other installed packages        [########................] 8.46 MB 11.3%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260903-beta
undici 8.10.2
Peer dependencies (7)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
chat ^4.41.0 optional peer
dd-trace ^6.13.0 optional peer
just-bash ^3.1.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 113.26 MB across 10104 installed files
  • Installed packages: 97 total (91 transitive-only)
  • dependencies: 4 direct packages totaling 45.04 MB
  • devDependencies: 2 direct packages totaling 5.11 MB
  • Other transitive package files: 63.11 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • @typescript/typescript-linux-x64: 27.95 MB (24.7%)
  • eve: 26.90 MB (23.7%)
  • @rolldown/binding-linux-x64-gnu: 19.81 MB (17.5%)
  • zod: 9.76 MB (8.6%)
  • ai: 8.20 MB (7.2%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 24.7%
🔴 eve                              [#######################.] 26.90 MB 23.7%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.81 MB 17.5%
🔴 zod                              [########................] 9.76 MB 8.6%
🔴 ai                               [#######.................] 8.20 MB 7.2%
🟠 undici                           [###.....................] 3.56 MB 3.1%
🔴 Other installed packages         [###############.........] 17.09 MB 15.1%
dependencies (4)
Package Range Installed size Share
@vercel/connect 2.2.0 188.7 kB 0.2%
ai ^7.0.137 8.20 MB 7.2%
eve file:eve-0.79.1.tgz 26.90 MB 23.7%
zod 4.5.4 9.76 MB 8.6%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.61 MB 2.3%
typescript 7.0.2 2.50 MB 2.2%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 12.27 MB 50.0%
🔴 functions/__server.func                         [########################] 12.27 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 12.27 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 12.27 MB
Function files 12.27 MB across 122 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 4.26 MB (34.8%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [######################..] 4.26 MB 34.8%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 8.2%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 6.6%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 733.9 kB 6.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.7 kB 5.7%
🟠 Other bundled files                            [########################] 4.75 MB 38.7%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 16 public routes, 1 internal alias • 12.27 MB
Metric Value
Public routes /
/.well-known/workflow/v1/webhook/[token]
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[attemptId]/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
/eve/v1/session/[sessionId]/stubs
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 12.27 MB
Function files 12.27 MB across 122 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 4.26 MB (34.8%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [######################..] 4.26 MB 34.8%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 8.2%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 6.6%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 733.9 kB 6.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.7 kB 5.7%
🟠 Other bundled files                            [########################] 4.75 MB 38.7%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 3.14 s -> 3.07 s (-72.4 ms) vs v27/observable-session (67e1dc2).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `v27/observable-session (67e1dc2)`
Phase Baseline Current Delta
extension.check 0.4 ms 15.2 ms +14.8 ms
project.resolve 0.2 ms 0.4 ms +0.2 ms
workspace.create 0.4 ms 0.5 ms +0.1 ms
host.prepare 650.4 ms 576.8 ms -73.6 ms
vercel.service-prefix.resolve 1.2 ms 1.1 ms -0.1 ms
nitro.create 625.3 ms 634.3 ms +9.0 ms
sandbox.prewarm 319.6 ms 270.5 ms -49.1 ms
nitro.cache.prepare 0.2 ms 0.2 ms 0.0 ms
nitro.prepare 0.6 ms 0.5 ms -0.1 ms
nitro.public-assets 0.6 ms 0.6 ms 0.0 ms
nitro.prerender 0.4 ms 0.5 ms +0.1 ms
nitro.bundle 1.40 s 1.43 s +29.4 ms
nitro.cache.write 0.4 ms 0.3 ms -0.1 ms
vercel.workflow-function.materialize 79.0 ms 76.8 ms -2.2 ms
agent-summary.emit 0.5 ms 0.5 ms 0.0 ms
connect-manifest.emit 0.3 ms 0.2 ms -0.1 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 2.9 ms 2.7 ms -0.2 ms
workspace.remove 2.2 ms 2.1 ms -0.1 ms

Comment thread packages/eve/src/harness/model-call/unique-call-ids.ts Outdated
Comment thread packages/eve/src/harness/model-call/call.ts Outdated
Comment thread packages/eve/src/harness/model-call/unique-call-ids.ts Outdated
Comment thread packages/eve/src/harness/model-call/call.ts Outdated
Comment thread packages/eve/src/harness/model-call/unique-call-ids.ts Outdated
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg October 14, 2026 22:42 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg October 14, 2026 23:01 Inactive
@OwenKephart
OwenKephart removed this pull request from stack #4696 October 14, 2026 23:46
@OwenKephart

Copy link
Copy Markdown
Contributor Author

Latest-main restack: combined top green

Head: 2a23bff78f49855886060a2db6ed973cd41ed5d9
Base: 4210766b5c5449e5b59bbddd5bb80a42111666ed

All current-head workflows pass:

The newer coverage commit is preserved, including its focused span, eval, framework-binding, dynamic-resolver, route, and channel-notice contracts. Main's managed connection callbacks, replay-stable session deadlines, retention rules, and empty-root-response fix are preserved. The v27 extension epochs now use tool 93, dynamic-tool 87, and connection 51 without rewriting main's published history.

The first CI run exposed a stale message.completed assertion in main's new Workflow retention eval. It now checks one native text content.completed reply, retaining its history-retention assertions. The fixture typecheck and fresh combined-top CI pass. Superseded workflows were cancelled; these are not runtime failures.

Local validation passed package and basic-runtime fixture typechecks, 179 unit tests, and nine scoped integration cases, plus lint/format, guards, extension contracts/history, docs, and template drift. One span-error-status test inherited from the newer coverage commit remains an explicit expected failure. Expired-history rejection, empty-root acceptance, and deadline preservation failed with their respective main fixes disabled and passed after restoration.

All 19 PRs and their order remain intact in native stack #4913, rooted at main. All 109 stack commits have verified SSH signatures and DCO trailers. The 19 release notes remain centralized in #4726, and the production LOC deltas for #4854 and #4855 are unchanged. Unrelated reactions/research edits were left untouched.

This current-head result supersedes the earlier pending-CI notes in the PR descriptions.

Signed-off-by: owenkephart <owen.kephart@vercel.com>
The announcement gate kept its own waitsOnPerson, which read sign-ins differently from park's.
announceWaiting already reads waiting(), so the gate only says whether the step stopped. Park's
predicate is renamed resumesOnPerson: a child's relayed sign-in resumes the turn on its call.

Signed-off-by: owenkephart <owen.kephart@vercel.com>
step.apply and the publisher both ran the cascade. The publisher is the writer; a step without
one cascades in its detached recorder.

Signed-off-by: owenkephart <owen.kephart@vercel.com>
Under EVE_CHECK_SESSION_EVENTS, a turn step whose decision disagrees with waiting() logs the
disagreement. Deferred dispatch, the one known kind, is labelled and not logged.

Signed-off-by: owenkephart <owen.kephart@vercel.com>
Random commits go through the writer's cascade, the checker, and both retentions: the cascade
is idempotent, its lines check, and an operational view answers what a complete one does.

Signed-off-by: owenkephart <owen.kephart@vercel.com>

@AndrewBarba AndrewBarba left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed with a fresh-context reviewer (tests per the test-audit gate, plus every stack changeset). Nothing critical; the breaking PRs are correctly minor. Approving.

Fix before release:

  • .changeset/v27-turns-wait-on-people.md: "Text produced while waiting on tasks is narration" only holds with finalReplyOnly (subagents). Root sessions still get reply, as this PR's own held-turn.root.eval.ts change shows.
  • tracing/eve/session-spans.test.ts:278 it.fails records that a failed session leaves invoke_agent without error status (it was set on main via turn.failed). Meanwhile instrumentation-vocabulary.md says the trace schema is unchanged. Fix it, or disclose it and link a tracking issue.

Minor: edge-cases.integration.test.ts:864 races on setTimeout(…, 500); per-read 20s timeouts vs the 30s test timeout; eve-events-entry.md describes intermediate states users never had; the changeset audit comment is stale (20 files, next minor is 0.82.0).

Keeps the restored Teams sign-in reveal route alongside main's inactive prompt retirement (#4840), and moves the new session span test to @vercel/agent-runs.

Signed-off-by: Andrew Barba <barba@hey.com>
Signed-off-by: Andrew Barba <barba@hey.com>

@vercel vercel Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

Registry channel files import a non-existent export messageToUserContent from eve/channels/chat-sdk, causing TS2724 build failures

Fix on Vercel

This branch had an error being deployed

1 failed and 1 active deployments
Preview – eve-pkg — c99ecc4b Deployed Oct 20, 2026 by vercel[bot]
Preview – eve-docs — c99ecc4b Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants