Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 5 additions & 0 deletions .changeset/command-deliveries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"eve": minor
---

Session commands now return serializable, followable `Delivery` receipts, with request statuses distinct from session termination and explicit retryability on `SessionNotActiveError`. Session handles use `id`/`stream()` with one line-indexed `SessionStreamOptions`, `SessionInfo` matches `ctx.session`, client handles are type-only exports, local resets become `newSession()`, eval/CLI results use `reply`/`data`, and `invokeTool` returns call-shaped outcomes.
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ export function AgentChatSession({
const toastError = displayError && dismissedError !== displayError ? displayError : null;

const resetSession = useCallback(() => {
agent.reset();
agent.newSession();
setActiveChatId(null);
activeChatIdRef.current = null;
eventIndexRef.current = 0;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ export function CouncilApp() {
setResult(undefined);
setResultError(undefined);
setSummaryUsage({});
agent.reset();
agent.newSession();
void agent.send(message, { outputSchema: councilResultSchema });
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ export function VoiceCall() {
setError(null);
setStarting(true);
// One call is one eve session.
agent.reset();
agent.newSession();
setTurns((turnsRef.current = []));
setAnchors([]);
delegatedTurnsRef.current = 0;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ export function removeEveAgent(chatId: string) {

export function resetAllEveAgents() {
for (const agent of agentsByChatId.values()) {
agent.reset();
agent.newSession();
}
agentsByChatId.clear();
}
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ export function createEveChatSession(
}

function reset() {
agent.value.reset();
agent.value.newSession();
}

async function retry() {
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/execution-model-and-durability.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ The resolver runs before the replacement's first model call, and its result ente

The read and the prompt are yours to bound:

- `startIndex: -2000` reads only the newest 2,000 events, and `signal` limits how long the read may take. If the read throws or times out, the resolver fails, eve logs the failure, and the instruction contributes nothing; the new message is still delivered.
- `startIndex: -2000` reads only the newest 2,000 stored lines (each can hold several events), and `signal` limits how long the read may take. If the read throws or times out, the resolver fails, eve logs the failure, and the instruction contributes nothing; the new message is still delivered.
- Keep the instruction's role `"user"` and label the text as historical data. Text the earlier user wrote should not reach the model as system instructions.
- `sessions` reads any session ID, including a session that set `experimental.workflow.retention: 0`, until its data is deleted. Skip such sessions if your agent sets zero retention. Pass only the ID eve provides in `ctx.session.predecessor`, never one taken from a request or a model.

Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/sessions-runs-and-streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ for await (const event of events) {
}
```

`stream()` accepts `startIndex` (negative values count back from the tail), `follow`, and `signal`. Key stored events on [`meta.position`](#positions), which stays stable across reads. With `follow: false`, the read ends at the durable tail observed when it opens; otherwise it keeps following new events until the stream closes or the signal aborts.
`stream()` takes `SessionStreamOptions` from `eve/client`: `startIndex` (a durable line position, with negative values counting back from the tail), `follow`, and `signal`. Client readers also accept `streamReconnectPolicy`. Key stored events on [`meta.position`](#positions), which stays stable across reads. With `follow: false`, the read ends at the durable tail observed when it opens; otherwise it keeps following new events until the stream closes or the signal aborts.

A read reflects events that are already durable. A hook runs after its line is written, so a bounded read there includes the event that triggered it. `sessions` reads any session ID without channel auth, so check that a caller may read a session before passing an ID from a request. It is available only in code that runs inside the eve server.

Expand Down
73 changes: 57 additions & 16 deletions docs/concepts/upgrade-to-v27-events.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,15 +261,38 @@ in [Instrumentation](/docs/observability/instrumentation#lifecycle-events).

The OpenTelemetry trace schema remains version 4, with the same span names and attributes.

## Commands and session handles

Sends, answers, and session controls return a `Delivery` directly. Its own `sessionId` and
`deliveryId` are serializable without reading the stream. Iterate it or call `result()` to follow
that exact command's settlement. Replace `MessageResponse` and `MessageResult` imports with
`Delivery` and `DeliveryResult`. `ChannelSource.send()` returns a delivery too; use its `sessionId`
to attach a session handle when you need one.

A refused inactive-session command throws `SessionNotActiveError`; its `retryable` flag distinguishes
an inbox still starting from a session that cannot accept more work. There are no separate
`no_active_turn` or `no_active_session` control results. A cancellation delivery settles when
applied; a reset delivery settles after retirement and address release.

Session readers expose `id` and `stream()`. `sessions.attach(id)` from `eve/server` returns a
`SessionReader` without I/O, not a `ServerSession`. Channel handles carry the existing commands
through `SessionOperations`; a reader does not acquire command authority. The data type previously
exported as `Session` from `eve/context` is now `SessionInfo`, exactly the metadata in `ctx.session`:
`id`, `auth`, `turn`, optional `parent`, `schedule`, and `predecessor`.

Every reader takes `SessionStreamOptions` from `eve/client`, replacing `StreamOptions` and
`GetEventStreamOptions`. `startIndex` addresses durable lines, matching `meta.position.line`,
not individual events. The shared options include `follow` and `signal`; only the client uses
`streamReconnectPolicy`. Client handles (`ClientSession`, `ClientAgentSession`, `ClientSessions`)
are type-only exports. Obtain them from `new Client()` and its methods instead of constructing them.

## Clients and frontends

- **Frontend commands resolve on acceptance.** `EveAgentStore.send()` and the frontend hooks'
`send()`/`respond()` return `{ sessionId, deliveryId }` without waiting for a reply or question.
Observe `view` and lifecycle callbacks for execution. `resume()` resolves after bounded catch-up,
without waiting for an active turn to end. The lower-level `MessageResponse.result()` still reads
its response through completion.
- **Read the native tables.** Frontend snapshots expose `view` (also `data.view`) and
`data.previews`. Replace `data.activeTurnId` with `activeTurn(view)?.turnId`,
`send()`/`respond()` return a `Delivery` without waiting for a reply or question. Observe `view`
and callbacks, or await `delivery.result()`, which reads the store's existing stream rather than
opening a second one. `resume()` resolves after bounded catch-up, without waiting for a turn to end.
- **Read the native tables.** Frontend snapshots expose `view` and `previews`. Replace `data.activeTurnId` with `activeTurn(view)?.turnId`,
`data.turns[turnId].waiting` with `waiting(view, turnId)`, and `data.tasks[taskId].calls`
with calls in `view.calls` whose `taskId` matches. Turns have `status: "running" | "settled"`
and a separate `outcome`, just like the event fold. The reduced `ConversationTurn`,
Expand All @@ -280,8 +303,11 @@ The OpenTelemetry trace schema remains version 4, with the same span names and a
Saved cursors from v26 sessions don't apply to v27 sessions.
- **Identity is `meta.position`.** `meta.id` is gone. Deduplicate on `(line, index)`.
- **Responses end on their delivery.** A message's response ends when its `delivery.settled`
arrives, finishing that line. `result().status` keeps its meaning: `"waiting"` while the session
stays open, `"completed"` once it ended, `"failed"` on failure.
arrives, finishing that line. `result().status` now uses `RequestStatus`: `working`,
`awaiting-input`, `completed`, `failed`, or `cancelled`. **`completed` has changed meaning:**
it means the request completed, not that the session ended. Replace old `completed` checks for
session termination with `sessionEnded`; old `waiting` comparisons no longer typecheck. Read text
from `reply`, not `message`. `requestStatus(view, deliveryId)` from `eve/events` applies the same rule.
- **Interactions.** Every interaction a person is asked, sign-ins included, is in one list:
`view.interactions`, a response's `interactions`, `openInteractions(view)`. Filter
by `request.kind`. An approval or a question about a call shows on its call's tool part
Expand All @@ -302,10 +328,19 @@ The OpenTelemetry trace schema remains version 4, with the same span names and a
client doesn't need its own reducer for session state. `callState(view, callId)` says where a
call stands for a display, `callLabel(row)` how to name it, and `waiting(view, { taskId })` or
`waiting(view, { callId })` who a task's or call's work waits on.
- **Distinguish local and server resets.** Call `newSession()` to clear local frontend state and
its cursor. `reset()` now retires the server-side session and returns its delivery.
- **Follow children with `child.opened`.** `session.agent(opened)` takes a `child.opened` event.

## Evals

- Rename `loadedSkill` to `calledSkill`, `calledSubagent` to `calledAgent`, `noFailedActions` to
`noFailedCalls`, and `parked()` to `awaitingInput()`. `calledTool` and call-count/order assertions stay.
- Replace `await t.target.attachSession(id)` with `const session = t.target.attach(id)` followed by
`await session.nextTurn()`. Attachment is I/O-free. Replace `t.target.watchTurn(id, options)` with
`t.target.attach(id, { startIndex }).watch({ until })`, passing only the options you need. Reads
advance the handle's cursor. Eval session handles use `id`; accepted turns still expose `sessionId`.
- Turn results expose `view`, `reply`, `data`, `sessionEnded`, and the shared request status.
- Match v27 facts in `t.event(...)`, `eventOrder(...)`, and `waitForEvent(...)`. Event matchers can
match a fact's `scope`.
- `turn.waitForToolCall(name, options)` waits for a settled call to a tool, and `toolCallsOf(events)`
Expand All @@ -317,14 +352,20 @@ The OpenTelemetry trace schema remains version 4, with the same span names and a

## MCP and `eve invoke`

- **MCP channel.** An `input_required` invocation lists its questions and approvals in
`interactions`, keyed by `interactionId`; answer every entry in one `agent_update` call with
`responses: [{ interactionId, optionId?, text? }]`. An `authorization_required` invocation lists
its sign-ins in the same `interactions` shape, each with its `signIn` challenge. An id names one
interaction for the session's life, so the ids of an earlier batch never answer a later one.
- **`eve invoke`.** A run that needs a person ends with one `input-required` status whose
`interactions` include sign-ins; `authorization-required` is gone. A sign-in resumes once it
completes, without a prompt.
- **MCP channel.** `input_required` remains the wire spelling for `awaiting-input`; sign-ins now
use it too, instead of `authorization_required`. Questions, approvals, and sign-ins share
`interactions`, keyed by `interactionId`. Answer all non-sign-in entries in one `agent_update`
call with `responses: [{ interactionId, optionId?, text? }]`. Complete sign-ins externally and
poll again. An id names one interaction for the session's life, so an earlier batch's ids never
answer a later one. `sessionEnded` separately reports session termination.
- **`eve remote invoke`.** JSON results use the shared request statuses, `reply`, `data`, and
`sessionEnded`, without a nested `outcome`. Resume coordinates include the original `deliveryId`.
Deployment authentication failures are CLI errors, not session statuses. An `awaiting-input`
sign-in resumes without a prompt once completed.
- **Direct tools.** `invokeTool` returns a call-shaped `outcome`: `completed` with `output` and
`modelOutput`, `failed` with `error: ErrorInfo`, or `rejected` with a `reason`, optional
`connections`, and optional policy `cause`. Invalid input is a failure with `error.code`;
approval and sign-in requirements are rejections, never conversational pauses.

## Sessions and remote agents

Expand Down
19 changes: 10 additions & 9 deletions docs/evals/assertions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,22 @@ Scoped assertions take no explicit value and gate by default. Assertions on `t`
| Assertion | Asserts |
| ----------------------------------------------------- | ----------------------------------------------------------------------------- |
| `t.succeeded()` | The run did not fail and did not park on unanswered HITL input |
| `t.parked()` | The run cleanly parked on HITL input |
| `t.awaitingInput()` | The run cleanly parked on HITL input |
| `t.messageIncludes(token)` | Joined assistant text contains `token` (string or RegExp) |
| `turn.outputEquals(value)` / `.outputMatches(schema)` | Deep equality or Standard Schema validation of turn/session structured output |
| `t.calledTool(name, opts?)` | A matching tool call completed (`input`, `output`, `status`, `count`) |
| `t.loadedSkill(skill, opts?)` | A load of `skill` completed (`output`, `status`, `count`) |
| `t.calledSkill(skill, opts?)` | A load of `skill` completed (`output`, `status`, `count`) |
| `t.notCalledTool(name)` | No request for `name` in any lifecycle state |
| `t.toolOrder([...names])` | Tool requests appear in order |
| `t.usedNoTools()` | No tool calls at all; skill loads don't count |
| `t.maxToolCalls(n)` | At most `n` tool calls; skill loads don't count |
| `t.noFailedActions()` | No tool, subagent, or skill action reported a failure |
| `t.calledSubagent(name, opts?)` | A subagent delegation happened (identity, remote, output, status, count) |
| `t.noFailedCalls()` | No tool, subagent, or skill action reported a failure |
| `t.calledAgent(name, opts?)` | A subagent delegation happened (identity, remote, output, status, count) |
| `t.event(type, opts?)` / `t.notEvent(type, opts?)` | Typed event presence, data, and count matching |
| `t.eventOrder([...matchers])` | Matching event groups occur in order |
| `t.eventsSatisfy(label, predicate)` | Escape hatch: any predicate over the typed event stream |

`succeeded()` accepts both a closed session and a healthy session left open for the next user message; it rejects protocol failures and unanswered HITL. `parked()` requires a clean HITL park. Structured output assertions live on turns and independent sessions, where the output is unambiguous (see the [output schema guide](../guides/client/output-schema)).
`succeeded()` accepts both a closed session and a healthy session left open for the next user message; it rejects protocol failures and unanswered HITL. `awaitingInput()` requires a clean HITL park. Structured output assertions live on turns and independent sessions, where the output is unambiguous (see the [output schema guide](../guides/client/output-schema)).

```ts
await t.send("What is the weather in Brooklyn?");
Expand All @@ -41,7 +41,8 @@ The same vocabulary narrows naturally in multi-turn and externally-created sessi
const first = await t.send("Call get_weather for Brooklyn");
first.calledTool("get_weather", { count: 1 });

const attached = await t.target.attachSession(sessionId);
const attached = t.target.attach(sessionId);
await attached.nextTurn();
attached.succeeded();
attached.messageIncludes("Sunny");
```
Expand Down Expand Up @@ -132,7 +133,7 @@ The same entries land in the [run artifacts](./running#artifacts) under `.eve/ev

## The matcher mini-language

`t.calledTool` and `t.calledSubagent` take matcher objects. Tools accept `{ input, output, status, count }`; subagents accept `{ callId, childSessionId, output, status, count }`. Calls match `status: "completed"` by default. Tool lifecycle checks also accept `"pending"`, `"failed"`, and `"rejected"`. Subagent checks accept `"pending"`, `"completed"`, `"failed"`, `"rejected"`, `"interrupted"`, and `"abandoned"`. A numeric `count` requires an exact number of calls matching every supplied constraint. Use a predicate for ranges or other custom count requirements; it receives the observed number of matching calls.
`t.calledTool` and `t.calledAgent` take matcher objects. Tools accept `{ input, output, status, count }`; subagents accept `{ callId, childSessionId, output, status, count }`. Calls match `status: "completed"` by default. Tool lifecycle checks also accept `"pending"`, `"failed"`, and `"rejected"`. Subagent checks accept `"pending"`, `"completed"`, `"failed"`, `"rejected"`, `"interrupted"`, and `"abandoned"`. A numeric `count` requires an exact number of calls matching every supplied constraint. Use a predicate for ranges or other custom count requirements; it receives the observed number of matching calls.

A subagent call is a task call: it stays `pending` until its `call.settled` arrives, then takes that call's outcome and, when completed, its `output`. A completed call leaves the child session available for later calls with the same `taskId`.

Expand All @@ -150,7 +151,7 @@ t.calledTool("echo", { output: (value) => String(value).includes(marker) });
parked.calledTool("guarded", { status: "pending", count: 1 });
t.calledTool("guarded", { output: /approved/, count: 1 });

t.calledSubagent("weather", {
t.calledAgent("weather", {
remoteUrl: (value) => value === process.env.WEATHER_AGENT_URL,
output: /72F/,
});
Expand Down Expand Up @@ -202,7 +203,7 @@ await t.require(
sessionIds,
satisfies((ids) => ids.length > 0, "dispatch started a session"),
);
await t.target.attachSession(sessionIds[0]!);
await t.target.attach(sessionIds[0]!).nextTurn();
```

Use the matching `require*` lookups when dependent code needs protocol data:
Expand Down
Loading
Loading