diff --git a/.changeset/compaction-strategy-interface.md b/.changeset/compaction-strategy-interface.md new file mode 100644 index 0000000000..aa6d4fb4cf --- /dev/null +++ b/.changeset/compaction-strategy-interface.md @@ -0,0 +1,5 @@ +--- +"eve": minor +--- + +An agent can now choose how its history is compacted. Export a strategy from `agent/compaction.ts` with `defineCompaction` from `eve/compaction`; its `compact` returns the replacement history, and eve keeps the events, memory recall, failure handling and billing. A hook can request a compaction with `ctx.compact()`, and `ctx.view.context.inputTokens` reports the latest model call's input tokens. eve still compacts when a request would fill 90% of the context window, and runs the default `fixedTail()` once if a strategy's result is still over it. The default is unchanged. `compaction.thresholdPercent` is removed. diff --git a/apps/templates/eve-coding-agent-template/agent/agent.ts b/apps/templates/eve-coding-agent-template/agent/agent.ts index 81ff341c93..c269d44b3f 100644 --- a/apps/templates/eve-coding-agent-template/agent/agent.ts +++ b/apps/templates/eve-coding-agent-template/agent/agent.ts @@ -3,6 +3,4 @@ import { defineAgent } from "eve"; export default defineAgent({ model: "anthropic/claude-opus-5.5", reasoning: "medium", - // Coding sessions run long; compact before the context gets crowded. - compaction: { thresholdPercent: 0.7 }, }); diff --git a/docs/agent-config.md b/docs/agent-config.md index 1e86342646..f943e38a7e 100644 --- a/docs/agent-config.md +++ b/docs/agent-config.md @@ -234,17 +234,16 @@ Gateway models; use `providerOptions.gateway.caching` instead. ## Compaction -Compaction summarizes older turns as you approach the context window. It's on by default, so you only tune when it kicks in. eve adds the estimated fixed checkpoint-prompt envelope to the trigger count, so compaction starts sooner than the conversation-only estimate. Lower `thresholdPercent` to compact sooner: +Compaction shortens the history as a request approaches the context window. It's on by default. eve compacts when a request would fill 90% of the model's context window, or when a hook calls `ctx.compact()`. How the history is shortened is a strategy you export from `agent/compaction.ts`; see [Compaction strategies](./concepts/compaction-strategies). `agent.ts` has no compaction settings. To choose the model that writes summaries, pass `model` to `fixedTail()`: -```ts title="agent/agent.ts" -export default defineAgent({ - model: "anthropic/claude-opus-5.5", - compaction: { - thresholdPercent: 0.75, // default 0.9 - }, -}); +```ts title="agent/compaction.ts" +import { fixedTail } from "eve/compaction"; + +export default fixedTail({ model: "anthropic/claude-haiku-4.5" }); ``` +The summary uses the session's model when `model` is omitted. The removed `compaction.model`, `compaction.modelContextWindowTokens`, and `compaction.thresholdPercent` fields in `agent.ts` throw an error that names this replacement. + See [Default harness](./concepts/default-harness#compaction) for how the loop applies it. ## Runtime limits diff --git a/docs/concepts/compaction-strategies.md b/docs/concepts/compaction-strategies.md new file mode 100644 index 0000000000..7ec008ee8d --- /dev/null +++ b/docs/concepts/compaction-strategies.md @@ -0,0 +1,70 @@ +--- +title: "Compaction strategies" +description: "Replace how an agent shortens its history by exporting a strategy from compaction.ts." +--- + +By default eve compacts with the fixed-tail strategy: a summary of everything but the latest conversation, which stays word for word up to 20,000 tokens. See [Default harness](./default-harness). To compact differently, export a strategy from `agent/compaction.ts`. + +```ts title="agent/compaction.ts" +import { defineCompaction } from "eve/compaction"; + +export default defineCompaction({ + async compact(history) { + return history.slice(-6); + }, +}); +``` + +## When it runs + +eve compacts in two cases, and a strategy decides only how: + +- **Overflow guard.** A request that would fill 90% of the model's context window is compacted before the model call. This is fixed. +- **A hook asks.** A [hook](../guides/hooks) calls `ctx.compact()` and the next model call compacts first: the running turn's next call, or the first call of the next turn when no turn is running. Asking twice before it runs compacts once. + +```ts title="agent/hooks/trim.ts" +import { defineHook } from "eve/hooks"; + +export default defineHook({ + events: { + "model.settled"(_event, ctx) { + if ((ctx.view.context.inputTokens ?? 0) > 60_000) ctx.compact(); + }, + }, +}); +``` + +`ctx.view.context.inputTokens` is the input tokens of the latest settled model call. + +If a strategy's result is still over the guard, or drops a tool call or result, the compaction fails: a requested one leaves the history as it was and the turn goes on, and one the guard forced ends the session with `COMPACTION_FAILED`. + +## What eve does and what the strategy does + +eve measures the next request and owns everything around the strategy: the `context.started` and `context.settled` events, [memory](../memory) capture and recall, failure, and billing of the summary call. The strategy owns one decision. + +| Member | Decides | +| ----------------------- | ------------------------------------ | +| `compact(history, ctx)` | The messages that replace `history`. | + +`ctx` carries `inputTokens`, `requestEnvelopeTokens`, `threshold`, `recentWindowSize`, and `forced`, which is `true` when a user or a hook asked and the request was not over the limit. `history` is the conversation without memory records; eve places recalled memory ahead of the result, so a strategy never sees or reorders it. A user message you return needs a `kind`; build one with `createCompactionUserMessage(content)` from `eve/compaction`, for example to mark a checkpoint. + +`ctx` adds `summarize({ messages, system })`, which runs the compaction model and bills the session, `estimateTokens(messages)`, and, when the turn can be cancelled, an `abortSignal`. A strategy that never summarizes makes no model call. + +## Failure + +Throw `CompactionOverLimitError` (exported from `eve/compaction`) when no history the strategy could return fits. Any thrown error fails the compaction the way a failed summary does. + +## Reusing the default + +`fixedTail()` from `eve/compaction` returns the default strategy, so a strategy can wrap it, for example to compact earlier than the threshold. Its options: + +- `model`: the model that writes the summary, a gateway model id or a language model. The session's model is the default. +- `recentTokens`: how many tokens of the latest conversation stay word for word. 20,000 by default, and at most half the compaction limit. + +For example: + +```ts title="agent/compaction.ts" +import { fixedTail } from "eve/compaction"; + +export default fixedTail({ model: "anthropic/claude-haiku-4.5" }); +``` diff --git a/docs/concepts/default-harness.md b/docs/concepts/default-harness.md index ba6bad4cc9..79b8685ac8 100644 --- a/docs/concepts/default-harness.md +++ b/docs/concepts/default-harness.md @@ -7,16 +7,7 @@ The default harness is eve's built-in agent loop. It manages model calls, compac ## Compaction -The harness keeps a long session from overflowing the model's context window. Before comparing the conversation with `thresholdPercent` (`0.9` by default), it adds the estimated fixed envelope of the checkpoint prompt used for compaction. It then summarizes the older turns and keeps going. The prompt asks the compaction model to distinguish completed progress and decisions from remaining work and to retain the constraints, preferences, data, and references needed to continue. When eve compacts again, it passes the previous checkpoint separately and without the transcript's per-message truncation, then replaces it with the updated checkpoint. The summary uses the active turn model unless you override it. After compacting, the harness keeps the most recent messages word for word next to the summary, up to 20,000 tokens (or half of the threshold, if that is smaller), with tool results shortened to 2,000 characters. The current user message is always part of that, so when it alone is larger than the budget the harness keeps only its start and its end, with a note where the middle was cut, and summarizes the cut-out middle with the older turns. If a compaction cannot bring the history under the threshold, the session fails with an error instead of compacting again, and the step is not retried: start a new session or send a smaller message. Tune when and how it kicks in under [`compaction`](../agent-config#compaction) in `agent.ts`: - -```ts title="agent/agent.ts" -export default defineAgent({ - model: "anthropic/claude-opus-5.5", - compaction: { - thresholdPercent: 0.75, - }, -}); -``` +The harness keeps a long session from overflowing the model's context window. Before comparing the conversation with 90% of the context window, it adds the estimated fixed envelope of the checkpoint prompt used for compaction. It then summarizes the older turns and keeps going. The prompt asks the compaction model to distinguish completed progress and decisions from remaining work and to retain the constraints, preferences, data, and references needed to continue. When eve compacts again, it passes the previous checkpoint separately and without the transcript's per-message truncation, then replaces it with the updated checkpoint. The summary uses the session's model unless you pass `model` to `fixedTail()`. After compacting, the harness keeps the most recent messages word for word next to the summary, up to 20,000 tokens (or half of the threshold, if that is smaller), with tool results shortened to 2,000 characters. The current user message is always part of that, so when it alone is larger than the budget the harness keeps only its start and its end, with a note where the middle was cut, and summarizes the cut-out middle with the older turns. If a compaction cannot bring the history under the threshold, the session fails with an error instead of compacting again, and the step is not retried: start a new session or send a smaller message. Set the summary model with `fixedTail({ model })` in [`compaction.ts`](../agent-config#compaction). Before summarizing, eve tries to shorten oversized tool results in older history. It checks whether that reduction is sufficient using the last provider-reported @@ -24,6 +15,8 @@ input token count plus an estimate of new messages. A smaller character estimate alone cannot satisfy compaction triggered by a higher provider count. If trimming cannot free enough space, eve summarizes the older history. +To change how the history is shortened, export your own strategy from `compaction.ts`. See [Compaction strategies](./compaction-strategies). + First-class [memory](../memory) stays outside history, so the summarizer never sees it. eve asks providers to capture before compaction rewrites history, and what earlier turns recalled follows the checkpoint. diff --git a/docs/concepts/meta.json b/docs/concepts/meta.json index a755c0f4d5..52c2ab92a4 100644 --- a/docs/concepts/meta.json +++ b/docs/concepts/meta.json @@ -8,6 +8,7 @@ "upgrade-to-v27-events", "state", "default-harness", + "compaction-strategies", "built-in-tools", "context-control", "security-model" diff --git a/docs/guides/hooks.md b/docs/guides/hooks.md index f49dd4f8f1..64ec13bb38 100644 --- a/docs/guides/hooks.md +++ b/docs/guides/hooks.md @@ -92,6 +92,7 @@ interface HookContext extends SessionContext { readonly occurrence: { readonly idempotencyKey: string; readonly abortSignal: AbortSignal }; readonly memo: Memo; cancel(reason?: string): void; + compact(): void; } ``` @@ -224,6 +225,10 @@ An unhandled error fails the running turn before its next model call or as it wo A hook failure does not trigger a retry. State changes and external side effects made before the exception are not rolled back. If a side effect needs retries or compensation, handle that inside the hook. Runtime failures outside the authored handler, such as failures setting up context or persisting state, still propagate. If a published fact outlives its step's checkpoint, the step's retry does not run the hook again. +## Compact the history from a hook + +Call `ctx.compact()` to compact the history before the next model call: the running turn's next call, or the first call of the next turn when no turn is running. Asking twice before it runs compacts once. How the history is shortened is the agent's [compaction strategy](../concepts/compaction-strategies). `ctx.view.context.inputTokens` is the input tokens of the latest settled model call, for hooks that decide by size. + ## Cancel the running turn from a hook Call `ctx.cancel(reason?)` when a hook finds that the turn cannot proceed. For example, a `turn.started` hook that cannot load the caller's credentials can stop the turn before the model runs, instead of letting every tool call fail: diff --git a/docs/memory/custom-provider.md b/docs/memory/custom-provider.md index 4479e4833b..a33122329f 100644 --- a/docs/memory/custom-provider.md +++ b/docs/memory/custom-provider.md @@ -103,7 +103,7 @@ Phase-specific fields: - `turn.started` and `turn.completed` add `turn` with the turn `id`, `input` messages, and `sequence`. -- `compaction.requested` adds `compaction.modelId` and +- `compaction.requested` adds `compaction.modelId` (the session's model, not the strategy's summary model) and `compaction.usageInputTokens`, the input tokens that crossed the threshold or `null` for a requested compaction. Its `turn` is the turn the compaction runs in, or `null` for a compaction between turns. diff --git a/docs/reference/agent-files.md b/docs/reference/agent-files.md index 55b06c9e5d..1466bb2f68 100644 --- a/docs/reference/agent-files.md +++ b/docs/reference/agent-files.md @@ -57,6 +57,7 @@ Paths below are relative to the agent directory. Root agents can use every path; | `hooks/` | Lifecycle and stream-event subscribers | Yes | Module-backed only; recursive directories are supported. | | `skills/` | On-demand procedures and capability packs | Yes | Flat Markdown, module-backed skills, or packaged skills. | | `lib/` | Shared authored helper code | Yes | Import-only; not copied into the sandbox. Helpers can import data files such as JSON from here. | +| `compaction.ts` | How the history is shortened | Yes | One strategy for the agent. See [Compaction strategies](/docs/concepts/compaction-strategies). | | `memory.ts` or `memory/.ts` | Cross-session memory | Yes | Provider-backed slots. See [Memory](/docs/memory). | | `sandbox.ts` or `sandbox/sandbox.ts` | The agent's sandbox | Yes | The framework default applies when neither is authored. When `sandbox/` exists, only `sandbox/sandbox.ts` is read. | | `sandbox/workspace/**` | Files seeded into the sandbox | Yes | Copied into the prepared environment; each new session sandbox starts with them in the workspace root. | diff --git a/docs/reference/typescript-api.md b/docs/reference/typescript-api.md index 8cb30469b5..67f94118a4 100644 --- a/docs/reference/typescript-api.md +++ b/docs/reference/typescript-api.md @@ -55,32 +55,33 @@ The AI SDK is a required peer and has its own Zod peer requirement. To type the ## The define\* helpers -| Helper | Import from | Authored at | Guide | -| ----------------------------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------ | -| `defineAgent` | `eve` | `agent/agent.ts` | [agent.ts](../agent-config) | -| `defineTool` | `eve/tools` | `agent/tools/.ts` | [Tools](../tools) | -| `defineWorkflowTool` | `eve/tools` | `agent/tools/.ts` | [Workflow tools](../tools/workflows) | -| `defineDynamic` | `eve`, `eve/tools`, `eve/skills`, `eve/instructions`, `eve/connections` | dynamic model or subagent `agent.ts`; `agent/{tools,skills,instructions,connections}/` | [Dynamic capabilities](../guides/dynamic-capabilities) | -| `defineMcpClientConnection` | `eve/connections` | `agent/connections/.ts` | [MCP connections](../connections/mcp) | -| `defineOpenAPIConnection` | `eve/connections` | `agent/connections/.ts` | [OpenAPI connections](../connections/openapi) | -| `defineChannel` | `eve/channels` | `agent/channels/.ts` | [Custom channels](../channels/custom) | -| `eveChannel`, `slackChannel`, and the other platforms | `eve/channels/` | `agent/channels/.ts` | [Channels](../channels/overview) | -| `defineSkill` | `eve/skills` | `agent/skills/.ts` | [Skills](../skills) | -| `defineInstructions` | `eve/instructions` | `agent/instructions.ts` | [Instructions](../instructions) | -| `defineMemory`, `defineMemoryProvider` | `eve/memory` | `agent/memory.ts` or `agent/memory/.ts` | [Memory](../memory) | -| `defineHook` | `eve/hooks` | `agent/hooks/.ts` | [Hooks](../guides/hooks) | -| `defineSchedule` | `eve/schedules` | `agent/schedules/.ts` | [Schedules](../schedules) | -| `defineState` | `eve/context` | tools, hooks, lifecycle | [Session context](../guides/session-context) | -| `defineSandbox` | `eve/sandbox` | `agent/sandbox.ts` or `agent/sandbox/sandbox.ts` | [Sandbox](../sandbox) | -| `defineParentSandbox` | `eve/sandbox` | `agent/subagents//sandbox.ts` | [Sandbox](../sandbox#share-a-parent-sandbox) | -| `defineSandboxProvider` | `eve/sandbox/provider` | custom sandbox provider modules | [Custom providers](../sandbox/custom-providers) | -| `defineInstrumentation` | `eve/instrumentation` | `agent/instrumentation/.ts` | [Instrumentation](../observability/instrumentation) | -| `otel`, `otelIntegration` | `eve/instrumentation/otel` | `agent/instrumentation/.ts` | [OpenTelemetry](../observability/otel) | -| `defineRemoteAgent` | `eve` | `agent/subagents//agent.ts` | [Remote agents](../guides/remote-agents) | -| `defineEval` | `eve/evals` | `evals/*.eval.ts` | [Evals](../evals/overview) | -| `defineEvalConfig` | `eve/evals` | `evals/evals.config.ts` | [Evals](../evals/overview) | -| `mockModel` | `eve/evals` | Deterministic fixture agent models | [Evals](../evals/overview) | -| `useEveAgent` | `eve/react`, `eve/vue`, `eve/svelte` | frontend | [Frontend](../guides/frontend/overview) | +| Helper | Import from | Authored at | Guide | +| ----------------------------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------- | +| `defineAgent` | `eve` | `agent/agent.ts` | [agent.ts](../agent-config) | +| `defineTool` | `eve/tools` | `agent/tools/.ts` | [Tools](../tools) | +| `defineWorkflowTool` | `eve/tools` | `agent/tools/.ts` | [Workflow tools](../tools/workflows) | +| `defineDynamic` | `eve`, `eve/tools`, `eve/skills`, `eve/instructions`, `eve/connections` | dynamic model or subagent `agent.ts`; `agent/{tools,skills,instructions,connections}/` | [Dynamic capabilities](../guides/dynamic-capabilities) | +| `defineMcpClientConnection` | `eve/connections` | `agent/connections/.ts` | [MCP connections](../connections/mcp) | +| `defineOpenAPIConnection` | `eve/connections` | `agent/connections/.ts` | [OpenAPI connections](../connections/openapi) | +| `defineChannel` | `eve/channels` | `agent/channels/.ts` | [Custom channels](../channels/custom) | +| `eveChannel`, `slackChannel`, and the other platforms | `eve/channels/` | `agent/channels/.ts` | [Channels](../channels/overview) | +| `defineSkill` | `eve/skills` | `agent/skills/.ts` | [Skills](../skills) | +| `defineInstructions` | `eve/instructions` | `agent/instructions.ts` | [Instructions](../instructions) | +| `defineCompaction` | `eve/compaction` | `agent/compaction.ts` | [Compaction strategies](../concepts/compaction-strategies) | +| `defineMemory`, `defineMemoryProvider` | `eve/memory` | `agent/memory.ts` or `agent/memory/.ts` | [Memory](../memory) | +| `defineHook` | `eve/hooks` | `agent/hooks/.ts` | [Hooks](../guides/hooks) | +| `defineSchedule` | `eve/schedules` | `agent/schedules/.ts` | [Schedules](../schedules) | +| `defineState` | `eve/context` | tools, hooks, lifecycle | [Session context](../guides/session-context) | +| `defineSandbox` | `eve/sandbox` | `agent/sandbox.ts` or `agent/sandbox/sandbox.ts` | [Sandbox](../sandbox) | +| `defineParentSandbox` | `eve/sandbox` | `agent/subagents//sandbox.ts` | [Sandbox](../sandbox#share-a-parent-sandbox) | +| `defineSandboxProvider` | `eve/sandbox/provider` | custom sandbox provider modules | [Custom providers](../sandbox/custom-providers) | +| `defineInstrumentation` | `eve/instrumentation` | `agent/instrumentation/.ts` | [Instrumentation](../observability/instrumentation) | +| `otel`, `otelIntegration` | `eve/instrumentation/otel` | `agent/instrumentation/.ts` | [OpenTelemetry](../observability/otel) | +| `defineRemoteAgent` | `eve` | `agent/subagents//agent.ts` | [Remote agents](../guides/remote-agents) | +| `defineEval` | `eve/evals` | `evals/*.eval.ts` | [Evals](../evals/overview) | +| `defineEvalConfig` | `eve/evals` | `evals/evals.config.ts` | [Evals](../evals/overview) | +| `mockModel` | `eve/evals` | Deterministic fixture agent models | [Evals](../evals/overview) | +| `useEveAgent` | `eve/react`, `eve/vue`, `eve/svelte` | frontend | [Frontend](../guides/frontend/overview) | Tool-wide authoring helpers and types such as `defineTool`, `defineWorkflowTool`, `defineDynamic`, `disableTool`, and `ToolLabelDefinition` come from `eve/tools`. Capability-specific definitions and helpers use their own subpaths (see [Built-in tools](../concepts/built-in-tools)): reusable definitions such as `bash` and `glob` come from `eve/tools/`, `webSearch` comes from `eve/tools/web_search`, `agentRouter` comes from `eve/tools/agent_router`, `sleep` comes from `eve/tools/sleep`, `noReply` comes from `eve/tools/no_reply`, and approval policies and types come from `eve/tools/approval`. The route verbs `GET`/`HEAD`/`POST`/`PUT`/`PATCH`/`DELETE`/`OPTIONS`/`WS` plus `disableRoute` come from `eve/channels`, and the channel auth helpers `localDev`/`vercelOidc`/`placeholderAuth` come from `eve/channels/auth`. @@ -155,6 +156,7 @@ Tool definitions accept `availableInSubagents: false` to restrict the tool to to | `eve/schedules` | `defineSchedule` | | `eve/skills` | `defineSkill`, `defineDynamic` | | `eve/instructions` | `defineInstructions`, `defineDynamic` | +| `eve/compaction` | `defineCompaction`, `fixedTail`, `CompactionOverLimitError`, strategy types | | `eve/memory` | `defineMemory`, `defineMemoryProvider`, provider and lifecycle types | | `eve/memory/scope` | `byPrincipal` and memory scope helpers | | `eve/memory/file` | `fileMemory`, `inMemory`, and the conditional document backend contract | diff --git a/docs/tools/workflows.mdx b/docs/tools/workflows.mdx index e0f135ffa3..d8d9179f81 100644 --- a/docs/tools/workflows.mdx +++ b/docs/tools/workflows.mdx @@ -600,7 +600,7 @@ const response = await reviewer.send("Review the migration plan.", { ``` The override changes only the model. The child keeps the target agent's instructions, tools, skills, -connections, sandbox behavior, limits, reasoning, and explicit compaction model. Pass a selection +connections, sandbox behavior, limits, reasoning, and compaction strategy. Pass a selection object to override model-specific settings too: ```ts diff --git a/e2e/fixtures/agent-compaction-envelope/agent/agent.ts b/e2e/fixtures/agent-compaction-envelope/agent/agent.ts index 3f2f2fe15b..679b4ff1c8 100644 --- a/e2e/fixtures/agent-compaction-envelope/agent/agent.ts +++ b/e2e/fixtures/agent-compaction-envelope/agent/agent.ts @@ -38,6 +38,6 @@ const model = mockModel({ export default defineAgent({ ...e2eAgentConfig(), model, - modelContextWindowTokens: 50_000, - compaction: { thresholdPercent: 0.2 }, + // eve compacts at 90% of the window: a 10,000-token limit. + modelContextWindowTokens: 11_112, }); diff --git a/e2e/fixtures/agent-compaction-measured/agent/agent.ts b/e2e/fixtures/agent-compaction-measured/agent/agent.ts index be503e49f9..0acf50bbb6 100644 --- a/e2e/fixtures/agent-compaction-measured/agent/agent.ts +++ b/e2e/fixtures/agent-compaction-measured/agent/agent.ts @@ -31,6 +31,6 @@ const model = mockModel({ export default defineAgent({ ...e2eAgentConfig(), model, - modelContextWindowTokens: 100_000, - compaction: { thresholdPercent: 0.2 }, + // eve compacts at 90% of the window: a 20,000-token limit. + modelContextWindowTokens: 22_223, }); diff --git a/e2e/fixtures/agent-compaction-regressions/agent/agent.ts b/e2e/fixtures/agent-compaction-regressions/agent/agent.ts index 4ae8921115..b41c0b05ce 100644 --- a/e2e/fixtures/agent-compaction-regressions/agent/agent.ts +++ b/e2e/fixtures/agent-compaction-regressions/agent/agent.ts @@ -1,4 +1,4 @@ -import { e2eAgentConfig, e2eModel } from "@eve-e2e/config"; +import { e2eAgentConfig } from "@eve-e2e/config"; import { defineAgent } from "eve"; import { mockModel, type MockModelRequest, type MockModelResponder } from "eve/evals"; @@ -15,7 +15,6 @@ import { import { assistantHasReport } from "../report-evidence"; import { HANDOFF_REFERENCE, REVIEW_REFERENCE } from "../release-reports"; -const TEST_CONTEXT_WINDOW_TOKENS = 32_000; // The request envelope includes instructions, the task block, and 10 advertised // tools; nothing is deferred. Keep room for the review/handoff checkpoint while // forcing the raw 2000x2000 file exchange (~5k history tokens) to compact. Its @@ -25,7 +24,7 @@ const TEST_CONTEXT_WINDOW_TOKENS = 32_000; // runs in this fixture. The allowance is below the actual envelope, so the // effective history budget is slightly less than 5,000 tokens. const TEST_REQUEST_ENVELOPE_TOKENS = 2_082; -const TEST_HISTORY_BUDGET_TOKENS = 5_000; +const TEST_HISTORY_BUDGET_TOKENS = 4_500; // Just over the threshold, without claiming a history far bigger than it has. const COMPACTION_PRESSURE_USAGE = { inputTokens: TEST_REQUEST_ENVELOPE_TOKENS + TEST_HISTORY_BUDGET_TOKENS + 1, @@ -225,13 +224,10 @@ export default defineAgent({ // fixture's scripted mock regardless of the matrix model. ...e2eAgentConfig(), model: taskModel, - modelContextWindowTokens: TEST_CONTEXT_WINDOW_TOKENS, - compaction: { - model: e2eModel(), - modelContextWindowTokens: TEST_CONTEXT_WINDOW_TOKENS, - thresholdPercent: - (TEST_REQUEST_ENVELOPE_TOKENS + TEST_HISTORY_BUDGET_TOKENS) / TEST_CONTEXT_WINDOW_TOKENS, - }, + // eve compacts at 90% of the window, so the window is sized for the limit below. + modelContextWindowTokens: Math.ceil( + (TEST_REQUEST_ENVELOPE_TOKENS + TEST_HISTORY_BUDGET_TOKENS) / 0.9, + ), limits: { sessionUsage: ({ window }) => window.inputTokens - window.cacheReadTokens >= 100_000, }, diff --git a/e2e/fixtures/agent-compaction-regressions/agent/compaction.ts b/e2e/fixtures/agent-compaction-regressions/agent/compaction.ts new file mode 100644 index 0000000000..7587c7dd66 --- /dev/null +++ b/e2e/fixtures/agent-compaction-regressions/agent/compaction.ts @@ -0,0 +1,4 @@ +import { e2eModel } from "@eve-e2e/config"; +import { fixedTail } from "eve/compaction"; + +export default fixedTail({ model: e2eModel() }); diff --git a/e2e/fixtures/agent-compaction-strategy/agent/agent.ts b/e2e/fixtures/agent-compaction-strategy/agent/agent.ts new file mode 100644 index 0000000000..93fad28a80 --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/agent/agent.ts @@ -0,0 +1,32 @@ +import { e2eAgentConfig } from "@eve-e2e/config"; +import { defineAgent } from "eve"; +import { mockModel } from "eve/evals"; + +const model = mockModel({ + modelId: "custom-strategy-compaction-model", + respond(request) { + const message = request.lastUserMessage ?? ""; + if (message.includes("verify")) { + const visible = request.messages.map((entry) => entry.text).join("\n"); + const issues = [ + visible.includes("record evidence 0") ? "OLD_HISTORY_KEPT" : undefined, + visible.includes("Summary of our conversation so far:") ? "SUMMARY_WRITTEN" : undefined, + visible.split("PROFILE_VALUE=RECALLED").length === 2 + ? undefined + : "MEMORY_NOT_RECALLED_ONCE", + ].filter((issue) => issue !== undefined); + return issues.length === 0 ? "CUSTOM_STRATEGY_OK" : `CUSTOM_STRATEGY_BAD:${issues.join(",")}`; + } + return { + text: "Evidence recorded.", + usage: { inputTokens: message.includes("report high usage") ? 30_000 : 1, outputTokens: 1 }, + }; + }, +}); + +export default defineAgent({ + ...e2eAgentConfig(), + model, + // Far from the 90% guard: the hook asks for the compaction. + modelContextWindowTokens: 100_000, +}); diff --git a/e2e/fixtures/agent-compaction-strategy/agent/compaction.ts b/e2e/fixtures/agent-compaction-strategy/agent/compaction.ts new file mode 100644 index 0000000000..cdfaa2f361 --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/agent/compaction.ts @@ -0,0 +1,12 @@ +import { defineCompaction } from "eve/compaction"; + +const KEPT_MESSAGES = 2; + +/** Keeps the latest messages, starting at a user message, and writes no summary. */ +export default defineCompaction({ + async compact(history) { + const tail = history.slice(-KEPT_MESSAGES); + const firstUser = tail.findIndex((message) => message.role === "user"); + return firstUser === -1 ? tail : tail.slice(firstUser); + }, +}); diff --git a/e2e/fixtures/agent-compaction-strategy/agent/hooks/compact.ts b/e2e/fixtures/agent-compaction-strategy/agent/hooks/compact.ts new file mode 100644 index 0000000000..e66905a63b --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/agent/hooks/compact.ts @@ -0,0 +1,10 @@ +import { defineHook } from "eve/hooks"; + +/** Asks for a compaction once the last model call's input passes 20,000 tokens. */ +export default defineHook({ + events: { + "model.requested"(_event, ctx) { + if ((ctx.view.context.inputTokens ?? 0) > 20_000) ctx.compact(); + }, + }, +}); diff --git a/e2e/fixtures/agent-compaction-strategy/agent/instructions.md b/e2e/fixtures/agent-compaction-strategy/agent/instructions.md new file mode 100644 index 0000000000..2a13b2f783 --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/agent/instructions.md @@ -0,0 +1 @@ +Record the supplied evidence and confirm the latest request. diff --git a/e2e/fixtures/agent-compaction-strategy/agent/memory/profile.ts b/e2e/fixtures/agent-compaction-strategy/agent/memory/profile.ts new file mode 100644 index 0000000000..3b1391f7d7 --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/agent/memory/profile.ts @@ -0,0 +1,14 @@ +import { defineMemory } from "eve/memory"; +import { byPrincipal } from "eve/memory/scope"; + +export default defineMemory({ + description: "A fixed profile value recalled at the start of every turn.", + provider: { + recall: { + async "turn.started"() { + return { messages: [{ content: "PROFILE_VALUE=RECALLED", id: "profile" }] }; + }, + }, + }, + scope: byPrincipal, +}); diff --git a/e2e/fixtures/agent-compaction-strategy/evals/custom-strategy.eval.ts b/e2e/fixtures/agent-compaction-strategy/evals/custom-strategy.eval.ts new file mode 100644 index 0000000000..1ed98a2f37 --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/evals/custom-strategy.eval.ts @@ -0,0 +1,23 @@ +import { defineEval } from "eve/evals"; + +const CASE = "[case: custom-strategy]"; + +export default defineEval({ + description: + "A hook asks for the compaction and the agent's own compaction.ts decides what stays, with no summary; memory is still recalled.", + async test(t) { + const session = await t.session(); + for (let index = 0; index < 4; index += 1) { + const seed = await session.send(`${CASE} record evidence ${index}.`); + seed.expectOk(); + seed.notEvent("context.settled", { data: { kind: "compaction", outcome: "completed" } }); + } + const pressure = await session.send(`${CASE} report high usage.`); + pressure.expectOk(); + const compacted = await session.send(`${CASE} verify the history.`); + compacted.expectOk(); + compacted.event("context.started", { data: { kind: "compaction" } }); + compacted.event("context.settled", { data: { kind: "compaction", outcome: "completed" } }); + compacted.messageIncludes("CUSTOM_STRATEGY_OK"); + }, +}); diff --git a/e2e/fixtures/agent-compaction-strategy/evals/evals.config.ts b/e2e/fixtures/agent-compaction-strategy/evals/evals.config.ts new file mode 100644 index 0000000000..e2bfb2f36f --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/evals/evals.config.ts @@ -0,0 +1,3 @@ +import { defineEvalConfig } from "eve/evals"; + +export default defineEvalConfig({ timeoutMs: 180_000 }); diff --git a/e2e/fixtures/agent-compaction-strategy/package.json b/e2e/fixtures/agent-compaction-strategy/package.json new file mode 100644 index 0000000000..22ffedf1ee --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/package.json @@ -0,0 +1,22 @@ +{ + "name": "agent-compaction-strategy", + "version": "0.0.0", + "private": true, + "type": "module", + "scripts": { + "build": "eve build", + "dev": "eve dev", + "start": "eve start", + "typecheck": "eve build --skip-sandbox-prewarm && tsc", + "test:e2e": "eve eval --strict" + }, + "dependencies": { + "@eve-e2e/config": "workspace:*", + "@workflow/world-postgres": "catalog:", + "eve": "workspace:*" + }, + "devDependencies": { + "@types/node": "catalog:", + "typescript": "catalog:" + } +} diff --git a/e2e/fixtures/agent-compaction-strategy/tsconfig.json b/e2e/fixtures/agent-compaction-strategy/tsconfig.json new file mode 100644 index 0000000000..85e3eceb8e --- /dev/null +++ b/e2e/fixtures/agent-compaction-strategy/tsconfig.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "esnext", + "moduleResolution": "bundler", + "outDir": "dist", + "rootDir": ".", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "declaration": true, + "noEmit": true, + "types": ["node"] + }, + "include": ["agent/**/*.ts", "evals/**/*.ts"] +} diff --git a/packages/eve/package.json b/packages/eve/package.json index d39ef8db8a..93151e7c80 100644 --- a/packages/eve/package.json +++ b/packages/eve/package.json @@ -118,6 +118,11 @@ "import": "./dist/src/public/tools/approval/index.js", "default": "./dist/src/public/tools/approval/index.js" }, + "./compaction": { + "types": "./dist/src/public/compaction/index.d.ts", + "import": "./dist/src/public/compaction/index.js", + "default": "./dist/src/public/compaction/index.js" + }, "./memory": { "types": "./dist/src/public/memory/index.d.ts", "import": "./dist/src/public/memory/index.js", diff --git a/packages/eve/public-api/eve.json b/packages/eve/public-api/eve.json index 96a31b13ac..6d2caa4fa5 100644 --- a/packages/eve/public-api/eve.json +++ b/packages/eve/public-api/eve.json @@ -1,6 +1,5 @@ [ "AgentBuildDefinition", - "AgentCompactionDefinition", "AgentDefinition", "AgentExperimentalDefinition", "AgentLimitsDefinition", diff --git a/packages/eve/public-api/eve/compaction.json b/packages/eve/public-api/eve/compaction.json new file mode 100644 index 0000000000..046121df5d --- /dev/null +++ b/packages/eve/public-api/eve/compaction.json @@ -0,0 +1,12 @@ +[ + "CompactionContext", + "CompactionMessage", + "CompactionOverLimitError", + "CompactionStrategy", + "DefinedCompaction", + "FixedTailOptions", + "HarnessModelMessage", + "createCompactionUserMessage", + "defineCompaction", + "fixedTail" +] diff --git a/packages/eve/src/compiler/manifest.ts b/packages/eve/src/compiler/manifest.ts index 3f7aa79f83..549e444cfc 100644 --- a/packages/eve/src/compiler/manifest.ts +++ b/packages/eve/src/compiler/manifest.ts @@ -28,7 +28,6 @@ import { ANTHROPIC_PROMPT_CACHE_TTLS, type InternalAgentDefinition, type InternalAgentModelDefinition, - type InternalAgentCompactionDefinition, type AgentBuildDefinition, type ModelRouting, } from "#shared/agent-definition.js"; @@ -140,19 +139,10 @@ export type CompiledDynamicModelDefinition = ModuleSourceRef; */ type CompiledAgentBuildDefinition = AgentBuildDefinition; -/** - * Normalized authored compaction configuration preserved in the compiled - * manifest. - */ -type CompiledAgentCompactionDefinition = Omit & { - model?: CompiledRuntimeModelReference; -}; - /** * Normalized additive agent configuration preserved in the compiled manifest. */ -type CompiledAgentDefinitionBase = Omit & { - compaction?: CompiledAgentCompactionDefinition; +type CompiledAgentDefinitionBase = Omit & { source: ModuleSourceRef; }; @@ -263,6 +253,8 @@ export interface CompiledDynamicToolDefinition extends ModuleSourceRef { readonly extensionNamespace?: string; } +export type CompiledCompactionDefinition = ModuleSourceRef; + export interface CompiledMemoryDefinition extends ModuleSourceRef { readonly description?: string; readonly slot: string; @@ -569,7 +561,6 @@ const compiledRuntimeModelReferenceSchema: z.ZodType = z - .object({ - model: compiledRuntimeModelReferenceSchema.optional(), - thresholdPercent: z.number().finite().min(0).max(1).optional(), - }) - .strict(); - const sessionTimeoutSchema = z.union([z.number().int().positive(), z.literal(false)]); const compiledAgentLimitsDefinitionSchema = z @@ -618,7 +602,6 @@ const compiledAgentLimitsDefinitionSchema = z const compiledAgentConfigBaseFields = { build: compiledAgentBuildDefinitionSchema.optional(), - compaction: compiledAgentCompactionDefinitionSchema.optional(), defaultTools: z.boolean().optional(), description: z.string().optional(), experimental: z @@ -990,6 +973,7 @@ const compiledAgentResourceFields = { extensionMounts: z.array(compiledExtensionMountSchema).default([]), hooks: z.array(compiledHookDefinitionSchema), memories: z.array(compiledMemoryDefinitionSchema).default([]), + compaction: moduleSourceRefSchema.nullable().default(null), sandbox: compiledSandboxDefinitionSchema, sandboxWorkspaces: z.array(compiledSandboxWorkspaceSchema), schedules: z.array(compiledScheduleDefinitionSchema), @@ -1095,6 +1079,7 @@ export const compiledAgentManifestSchema = z dynamicTools: z.array(compiledDynamicToolDefinitionSchema).default([]), hooks: z.array(compiledHookDefinitionSchema), memories: z.array(compiledMemoryDefinitionSchema).default([]), + compaction: moduleSourceRefSchema.nullable().default(null), kind: z.literal(COMPILED_AGENT_MANIFEST_KIND), remoteAgents: z.array(compiledRemoteAgentNodeSchema), sandbox: compiledSandboxDefinitionSchema, @@ -1125,6 +1110,7 @@ export interface CreateCompiledAgentResourcesInput { readonly extensionMounts?: readonly CompiledExtensionMount[]; readonly hooks?: readonly CompiledHookDefinition[]; readonly memories?: readonly CompiledMemoryDefinition[]; + readonly compaction?: CompiledCompactionDefinition | null; readonly remoteAgents?: readonly CompiledRemoteAgentNode[]; readonly sandbox: CompiledSandboxDefinition; readonly sandboxWorkspaces?: readonly CompiledSandboxWorkspace[]; @@ -1164,6 +1150,7 @@ export function createCompiledAgentResources( extensionMounts: [...(input.extensionMounts ?? [])], hooks: [...(input.hooks ?? [])], memories: [...(input.memories ?? [])], + compaction: input.compaction ?? null, instructions: [...(input.instructions ?? [])], remoteAgents: [...(input.remoteAgents ?? [])], sandbox: input.sandbox, @@ -1207,13 +1194,6 @@ function cloneCompiledAgentDefinition(config: CompiledAgentDefinition): Compiled ? undefined : [...config.build.externalDependencies], }, - compaction: { - model: - config.compaction?.model === undefined - ? undefined - : cloneCompiledRuntimeModelReference(config.compaction.model), - thresholdPercent: config.compaction?.thresholdPercent, - }, defaultTools: config.defaultTools, description: config.description, experimental: @@ -1352,9 +1332,6 @@ function cloneCompiledRuntimeModelReference( if (model.source !== undefined) { clone.source = { ...model.source }; } - if (model.sourceField !== undefined) { - clone.sourceField = model.sourceField; - } return clone; } diff --git a/packages/eve/src/compiler/module-lifecycle.ts b/packages/eve/src/compiler/module-lifecycle.ts index aa7d7718a4..120df3502e 100644 --- a/packages/eve/src/compiler/module-lifecycle.ts +++ b/packages/eve/src/compiler/module-lifecycle.ts @@ -56,7 +56,6 @@ export function markConfigRuntimeEntries( const sources = [ config.dynamicModel, config.model?.source, - config.compaction?.model?.source, // The runtime loads `limits.sessionUsage` from the agent config module. config.limits?.sessionUsage === true ? config.source : undefined, ]; diff --git a/packages/eve/src/compiler/normalize-agent-config.ts b/packages/eve/src/compiler/normalize-agent-config.ts index 9f38bd3a99..f009ec1423 100644 --- a/packages/eve/src/compiler/normalize-agent-config.ts +++ b/packages/eve/src/compiler/normalize-agent-config.ts @@ -75,17 +75,8 @@ export async function compileAgentConfig( value: definition.model as PublicAgentStaticModelDefinition, }) : undefined; - const compaction: { - model?: CompiledRuntimeModelReference; - thresholdPercent?: number; - } = {}; - const compiledConfig: { build?: CompiledAgentDefinition["build"]; - compaction: { - model?: CompiledRuntimeModelReference; - thresholdPercent?: number; - }; defaultTools?: boolean; description?: string; experimental?: CompiledAgentDefinition["experimental"]; @@ -95,7 +86,6 @@ export async function compileAgentConfig( tool?: AgentToolExposure; limits?: CompiledAgentDefinition["limits"]; } = { - compaction, name: manifest.agentId, source: { ...configModule }, }; @@ -147,23 +137,6 @@ export async function compileAgentConfig( }; } - if (definition.compaction?.model !== undefined) { - compaction.model = await normalizeAuthoredModelReference({ - modelCatalog: context.modelCatalog, - purpose: "the compaction summary model", - contextWindowTokens: definition.compaction.modelContextWindowTokens, - providerOptions: definition.modelOptions?.providerOptions, - source: configModule, - sourceField: "compaction.model", - sourcePath: configModulePath, - value: definition.compaction.model, - }); - } - - if (definition.compaction?.thresholdPercent !== undefined) { - compaction.thresholdPercent = definition.compaction.thresholdPercent; - } - if (dynamicModel !== undefined) { return { ...compiledConfig, dynamicModel }; } @@ -202,7 +175,6 @@ async function normalizeAuthoredModelReference(input: { readonly promptCache?: AgentPromptCacheDefinition; readonly providerOptions?: Record; readonly source?: ModuleSourceRef; - readonly sourceField?: CompiledRuntimeModelReference["sourceField"]; readonly sourcePath?: string; readonly value: PublicAgentStaticModelDefinition; }): Promise { @@ -265,7 +237,6 @@ async function normalizeAuthoredModelReference(input: { providerOptions: parseProviderOptionsRecord(input.providerOptions), routing, }; - if (input.sourceField !== undefined) sourceBackedModel.sourceField = input.sourceField; if (input.promptCache !== undefined) sourceBackedModel.promptCache = input.promptCache; if (input.contextWindowTokens === undefined) { diff --git a/packages/eve/src/compiler/normalize-compaction.test.ts b/packages/eve/src/compiler/normalize-compaction.test.ts new file mode 100644 index 0000000000..30b89b457e --- /dev/null +++ b/packages/eve/src/compiler/normalize-compaction.test.ts @@ -0,0 +1,68 @@ +import { describe, expect, it } from "vitest"; + +import { compileCompactionDefinition } from "#compiler/normalize-compaction.js"; +import type { AgentModuleBinding } from "#compiler/source-graph.js"; +import { normalizeCompactionDefinition } from "#internal/authored-definition/compaction.js"; +import { defineCompaction } from "#public/compaction/index.js"; + +const message = "Invalid compaction definition."; + +function createStrategy() { + return { + async compact() { + return []; + }, + }; +} + +const source = { + logicalPath: "compaction.ts", + sourceId: "compaction.ts", + sourceKind: "module" as const, +}; +const binding = { logicalPath: "compaction.ts" } as AgentModuleBinding; + +function compile(exported: unknown) { + return compileCompactionDefinition(source, { + binding, + loadNamespace: async () => ({ default: exported }), + }); +} + +describe("normalizeCompactionDefinition", () => { + it("accepts a defineCompaction() strategy", () => { + const definition = defineCompaction(createStrategy()); + + expect(normalizeCompactionDefinition(definition, message)).toBe(definition); + }); + + it("rejects a plain strategy that was not created with defineCompaction()", () => { + expect(() => normalizeCompactionDefinition(createStrategy(), message)).toThrow(message); + }); + + it("rejects a defined export missing a strategy method", () => { + const definition = defineCompaction(createStrategy()); + Reflect.deleteProperty(definition, "compact"); + + expect(() => normalizeCompactionDefinition(definition, message)).toThrow( + `${message} "compact" must be a function.`, + ); + }); +}); + +describe("compileCompactionDefinition", () => { + it("compiles a defined export to its module source ref", async () => { + await expect(compile(defineCompaction(createStrategy()))).resolves.toEqual({ + exportName: undefined, + logicalPath: "compaction.ts", + sourceId: "compaction.ts", + sourceKind: "module", + }); + }); + + it("rejects a non-defined export with a pointer to defineCompaction()", async () => { + await expect(compile(createStrategy())).rejects.toThrow( + 'Expected the compaction export "default" from "compaction.ts" to be created with defineCompaction().', + ); + }); +}); diff --git a/packages/eve/src/compiler/normalize-compaction.ts b/packages/eve/src/compiler/normalize-compaction.ts new file mode 100644 index 0000000000..fd5dc8d82a --- /dev/null +++ b/packages/eve/src/compiler/normalize-compaction.ts @@ -0,0 +1,28 @@ +import type { CompiledCompactionDefinition } from "#compiler/manifest.js"; +import { + loadModuleBackedDefinition, + type ModuleBackedDefinitionLoadOptions, +} from "#compiler/normalize-helpers.js"; +import { normalizeCompactionDefinition } from "#internal/authored-definition/compaction.js"; +import type { ModuleSourceRef } from "#shared/source-ref.js"; + +export async function compileCompactionDefinition( + source: ModuleSourceRef, + options: ModuleBackedDefinitionLoadOptions, +): Promise { + normalizeCompactionDefinition( + await loadModuleBackedDefinition({ + binding: options.binding, + kind: "compaction", + loadNamespace: options.loadNamespace, + source, + }), + `Expected the compaction export "${source.exportName ?? "default"}" from "${source.logicalPath}" to be created with defineCompaction().`, + ); + return { + exportName: source.exportName, + logicalPath: source.logicalPath, + sourceId: source.sourceId, + sourceKind: "module", + }; +} diff --git a/packages/eve/src/compiler/normalize-manifest.test.ts b/packages/eve/src/compiler/normalize-manifest.test.ts index a93899ca95..6a8050791c 100644 --- a/packages/eve/src/compiler/normalize-manifest.test.ts +++ b/packages/eve/src/compiler/normalize-manifest.test.ts @@ -707,7 +707,6 @@ describe("compileAgentManifest source graph", () => { specificationVersion: "v3", }) as never; const turnModel = directModel("turn-model"); - const summaryModel = directModel("summary-model"); const sourceRegistry = registry([ { logicalPath: "agent.ts", @@ -715,7 +714,6 @@ describe("compileAgentManifest source graph", () => { default: defineAgent({ model: turnModel, modelContextWindowTokens: 8_192, - compaction: { model: summaryModel, modelContextWindowTokens: 8_192 }, }), }), }, @@ -737,9 +735,6 @@ describe("compileAgentManifest source graph", () => { await expect(resolveRuntimeModelReference(resolved.config!.model!, scope)).resolves.toBe( turnModel, ); - await expect( - resolveRuntimeModelReference(resolved.config!.compaction!.model!, scope), - ).resolves.toBe(summaryModel); } finally { vi.unstubAllEnvs(); } diff --git a/packages/eve/src/compiler/normalize-manifest.ts b/packages/eve/src/compiler/normalize-manifest.ts index 5112b761fd..d0df210c3e 100644 --- a/packages/eve/src/compiler/normalize-manifest.ts +++ b/packages/eve/src/compiler/normalize-manifest.ts @@ -22,6 +22,7 @@ import { type CompiledDynamicSkillDefinition, type CompiledDynamicToolDefinition, type CompiledInstructionsDefinition, + type CompiledCompactionDefinition, type CompiledMemoryDefinition, type CompiledHookDefinition, type CompiledRemoteAgentNode, @@ -55,6 +56,7 @@ import { resolveWorkspaceSubagentDefinition } from "#compiler/resolve-workspace- import { workspaceSubagentName } from "#public/definitions/workspace-agent.js"; import { compileHookEntry } from "#compiler/normalize-hook.js"; import { compileInstructionsEntry } from "#compiler/normalize-instructions.js"; +import { compileCompactionDefinition } from "#compiler/normalize-compaction.js"; import { compileMemoryDefinition } from "#compiler/normalize-memory.js"; import { createMemoryWrapperCandidates } from "#compiler/memory-wrapper-candidates.js"; import { compileSandboxDefinition } from "#compiler/normalize-sandbox.js"; @@ -482,6 +484,7 @@ class AgentGraphCompiler { const schedules: CompiledScheduleDefinition[] = []; const scheduleCollections: CompiledScheduleCollectionDefinition[] = []; const channels: CompiledChannelDefinition[] = []; + let compaction: CompiledCompactionDefinition | null = null; let sandbox: CompiledSandboxDefinition | undefined; const selectedSourceIds = collectSelectedSourceIds(state.composed); const loadNamespace = state.evaluation.loadNamespace; @@ -562,6 +565,13 @@ class AgentGraphCompiler { ); state.evaluation.requireRuntimeEntry(candidate.sourceId); break; + case "compaction": + compaction = await compileCompactionDefinition(entry.source, { + binding: binding!, + loadNamespace, + }); + state.evaluation.requireRuntimeEntry(candidate.sourceId); + break; case "sandbox": sandbox = await compileSandboxDefinition(input.manifest.agentRoot, entry.source, { binding: binding!, @@ -675,6 +685,7 @@ class AgentGraphCompiler { extensionMounts, hooks, memories, + compaction, instructions, sandbox, sandboxWorkspaces: input.manifest.sandboxWorkspaces.map((workspace) => ({ diff --git a/packages/eve/src/compiler/normalize-subagent.test.ts b/packages/eve/src/compiler/normalize-subagent.test.ts index ac4109e79a..27eb0d6b3b 100644 --- a/packages/eve/src/compiler/normalize-subagent.test.ts +++ b/packages/eve/src/compiler/normalize-subagent.test.ts @@ -1,6 +1,9 @@ import { describe, expect, it } from "vitest"; -import { normalizeSubagentConfig } from "#compiler/normalize-subagent.js"; +import { + assertRemoteAgentDefinitionHasNoLocalPackageEntries, + normalizeSubagentConfig, +} from "#compiler/normalize-subagent.js"; import { defineAgent } from "#public/definitions/agent.js"; import { defineRemoteAgent } from "#public/definitions/remote-agent.js"; @@ -75,3 +78,33 @@ describe("normalizeSubagentConfig", () => { ).toMatchObject({ kind: "remote", tool: false }); }); }); + +describe("assertRemoteAgentDefinitionHasNoLocalPackageEntries", () => { + it("rejects a compaction.ts next to a remote agent definition", () => { + const source = { + logicalPath: "subagents/remote.ts", + manifest: { + compaction: { + exportName: "default", + logicalPath: "subagents/remote/compaction.ts", + sourceId: "subagents/remote/compaction.ts", + sourceKind: "module" as const, + }, + connections: [], + hooks: [], + instructions: [], + lib: [], + sandbox: null, + sandboxWorkspaces: [], + schedules: [], + skills: [], + subagents: [], + tools: [], + }, + }; + + expect(() => assertRemoteAgentDefinitionHasNoLocalPackageEntries(source)).toThrow( + /Remove unsupported entries: compaction\.ts\./, + ); + }); +}); diff --git a/packages/eve/src/compiler/normalize-subagent.ts b/packages/eve/src/compiler/normalize-subagent.ts index 8398baef05..2ac5a0c1ce 100644 --- a/packages/eve/src/compiler/normalize-subagent.ts +++ b/packages/eve/src/compiler/normalize-subagent.ts @@ -64,11 +64,26 @@ export function normalizeSubagentConfig(value: unknown, message: string): Normal return { definition: value, kind: "local" }; } -export function assertRemoteAgentDefinitionHasNoLocalPackageEntries( - source: LocalSubagentSourceRef, -): void { +export function assertRemoteAgentDefinitionHasNoLocalPackageEntries(source: { + readonly logicalPath: string; + readonly manifest: Pick< + LocalSubagentSourceRef["manifest"], + | "compaction" + | "connections" + | "hooks" + | "instructions" + | "lib" + | "sandbox" + | "sandboxWorkspaces" + | "schedules" + | "skills" + | "subagents" + | "tools" + >; +}): void { const manifest = source.manifest; const extraEntries = [ + manifest.compaction !== null ? "compaction.ts" : undefined, manifest.connections.length > 0 ? "connections/" : undefined, manifest.hooks.length > 0 ? "hooks/" : undefined, manifest.instructions.length > 0 ? "instructions" : undefined, diff --git a/packages/eve/src/compiler/project-sources.ts b/packages/eve/src/compiler/project-sources.ts index fd8cde1eb1..f603390378 100644 --- a/packages/eve/src/compiler/project-sources.ts +++ b/packages/eve/src/compiler/project-sources.ts @@ -31,6 +31,7 @@ export type ProjectedModuleSource = readonly candidate: AgentModuleCandidate; readonly kind: | "channel" + | "compaction" | "config" | "extension" | "hook" @@ -67,6 +68,7 @@ const MODULE_KIND_BY_SLOT_ROOT: Partial< > = { agent: "config", channels: "channel", + compaction: "compaction", extensions: "extension", hooks: "hook", instructions: "instructions", @@ -189,6 +191,7 @@ export function listApplicationModulePaths(manifest: AgentSourceManifest): strin ...manifest.connections, ...manifest.hooks, ...manifest.memories, + ...(manifest.compaction === null ? [] : [manifest.compaction]), ...manifest.tools, ...manifest.instructions, ...manifest.schedules, @@ -385,6 +388,7 @@ function projectManifest(input: { for (const source of input.manifest.connections) pushModule(source); for (const source of input.manifest.hooks) pushModule(source); for (const source of input.manifest.memories) pushModule(source); + if (input.manifest.compaction !== null) pushModule(input.manifest.compaction); if (input.manifest.sandbox !== null) pushModule(input.manifest.sandbox); for (const source of input.manifest.tools) pushModule(source); for (const source of input.manifest.instructions) { diff --git a/packages/eve/src/compiler/source-graph-paths.ts b/packages/eve/src/compiler/source-graph-paths.ts index 58ad6fac3b..a610cce243 100644 --- a/packages/eve/src/compiler/source-graph-paths.ts +++ b/packages/eve/src/compiler/source-graph-paths.ts @@ -37,7 +37,7 @@ export function validateProgrammaticLogicalPathInternal( const extensionless = stripLogicalPathExtension(logicalPath); const supported = (segments.length === 1 && - ["agent", "memory", "sandbox", "instrumentation"].includes(extensionless)) || + ["agent", "compaction", "memory", "sandbox", "instrumentation"].includes(extensionless)) || (root === "sandbox" && segments.length === 2 && getSupportedModuleBaseName(fileName) === "sandbox") || diff --git a/packages/eve/src/compiler/validate-artifact.ts b/packages/eve/src/compiler/validate-artifact.ts index 6b4f63151d..f57d664aea 100644 --- a/packages/eve/src/compiler/validate-artifact.ts +++ b/packages/eve/src/compiler/validate-artifact.ts @@ -354,6 +354,7 @@ function collectReferencedModuleSources( for (const value of node.dynamicTools) add(value); for (const value of node.hooks) add(value); for (const value of node.memories) add(value); + if (node.compaction !== null) add(node.compaction); for (const value of node.instructions) if (value.sourceKind === "module") add(value); for (const value of node.skills) if (value.sourceKind === "module") add(value); for (const value of node.schedules) if (value.sourceKind === "module") add(value); diff --git a/packages/eve/src/discover/compaction.integration.test.ts b/packages/eve/src/discover/compaction.integration.test.ts new file mode 100644 index 0000000000..6d70d20a6b --- /dev/null +++ b/packages/eve/src/discover/compaction.integration.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, it } from "vitest"; + +import { discoverCompactionSource } from "#discover/compaction.js"; +import { DISCOVER_MODULE_SLOT_COLLISION, readSortedDirectoryEntries } from "#discover/grammar.js"; +import { buildMemoryAgentProject } from "#internal/testing/memory-agent-source.js"; + +async function discover(agentFiles: Record) { + const project = buildMemoryAgentProject({ agentFiles }); + const rootEntries = await readSortedDirectoryEntries(project.source, project.agentRoot); + return discoverCompactionSource({ rootEntries, rootPath: project.agentRoot }); +} + +describe("discoverCompactionSource", () => { + it("returns null when the agent has no compaction module", async () => { + const result = await discover({ "instructions.md": "" }); + + expect(result.diagnostics).toEqual([]); + expect(result.compaction).toBeNull(); + }); + + it("discovers a flat compaction module at the agent root", async () => { + const result = await discover({ "compaction.ts": "export default {};\n" }); + + expect(result.diagnostics).toEqual([]); + expect(result.compaction).toEqual({ + logicalPath: "compaction.ts", + sourceId: "compaction.ts", + sourceKind: "module", + }); + }); + + it("reports ambiguous compaction modules", async () => { + const result = await discover({ + "compaction.mjs": "export default {};\n", + "compaction.ts": "export default {};\n", + }); + + expect(result.compaction).toBeNull(); + expect(result.diagnostics.map((diagnostic) => diagnostic.code)).toEqual([ + DISCOVER_MODULE_SLOT_COLLISION, + ]); + }); +}); diff --git a/packages/eve/src/discover/compaction.ts b/packages/eve/src/discover/compaction.ts new file mode 100644 index 0000000000..1b62b776da --- /dev/null +++ b/packages/eve/src/discover/compaction.ts @@ -0,0 +1,19 @@ +import type { DiscoverDiagnostic } from "#discover/diagnostics.js"; +import type { CompactionSourceRef } from "#discover/manifest.js"; +import { discoverFlatModuleSource } from "#discover/grammar.js"; +import type { ProjectSourceEntry } from "#discover/project-source.js"; + +export function discoverCompactionSource(input: { + readonly rootEntries: readonly ProjectSourceEntry[]; + readonly rootPath: string; +}): { + readonly compaction: CompactionSourceRef | null; + readonly diagnostics: DiscoverDiagnostic[]; +} { + const flat = discoverFlatModuleSource({ + rootEntries: input.rootEntries, + rootPath: input.rootPath, + slotName: "compaction", + }); + return { compaction: flat.module ?? null, diagnostics: flat.diagnostics }; +} diff --git a/packages/eve/src/discover/discover-agent.ts b/packages/eve/src/discover/discover-agent.ts index b3572c89b2..992795ec3b 100644 --- a/packages/eve/src/discover/discover-agent.ts +++ b/packages/eve/src/discover/discover-agent.ts @@ -8,6 +8,7 @@ import { DISCOVER_EXTENSION_MOUNT_AMBIGUOUS, DISCOVER_EXTENSION_MOUNT_MISSING_DECLARATION, DISCOVER_EXTENSION_INSTRUMENTATION_UNSUPPORTED, + DISCOVER_EXTENSION_COMPACTION_UNSUPPORTED, DISCOVER_EXTENSION_MEMORY_UNSUPPORTED, DISCOVER_EXTENSION_NESTED_MOUNT_UNSUPPORTED, DISCOVER_EXTENSION_SANDBOX_UNSUPPORTED, @@ -31,6 +32,7 @@ import { readSortedDirectoryEntries, } from "#discover/grammar.js"; import { discoverLibSources } from "#discover/lib.js"; +import { discoverCompactionSource } from "#discover/compaction.js"; import { discoverMemorySources } from "#discover/memory.js"; import { type AgentSourceManifest, @@ -167,6 +169,9 @@ export async function discoverAgent(input: DiscoverAgentInput): Promise` module. */ +export type CompactionSourceRef = ModuleSourceRef; + export interface MemorySourceRef extends ModuleSourceRef { readonly slot: string; } @@ -216,6 +219,7 @@ export interface AgentSourceManifest { resolvedExtensions: ResolvedExtensionMount[]; hooks: ModuleSourceRef[]; memories: MemorySourceRef[]; + compaction: CompactionSourceRef | null; lib: LibSourceRef[]; kind: typeof AGENT_SOURCE_MANIFEST_KIND; /** @@ -263,6 +267,7 @@ export interface CreateAgentSourceManifestInput { resolvedExtensions?: readonly ResolvedExtensionMount[]; hooks?: readonly ModuleSourceRef[]; memories?: readonly MemorySourceRef[]; + compaction?: CompactionSourceRef | null; lib?: readonly LibSourceRef[]; /** * Optional package name read from the app root's package.json. @@ -340,6 +345,7 @@ export function createAgentSourceManifest( resolvedExtensions: [...(input.resolvedExtensions ?? [])], hooks: [...(input.hooks ?? [])], memories: [...(input.memories ?? [])], + compaction: input.compaction ?? null, instructions: [...(input.instructions ?? [])], lib: [...(input.lib ?? [])], kind: AGENT_SOURCE_MANIFEST_KIND, diff --git a/packages/eve/src/execution/coordination-dispatch-shared.ts b/packages/eve/src/execution/coordination-dispatch-shared.ts index faa4e1aa35..205486207b 100644 --- a/packages/eve/src/execution/coordination-dispatch-shared.ts +++ b/packages/eve/src/execution/coordination-dispatch-shared.ts @@ -158,9 +158,6 @@ export async function prepareActionDispatch(input: { const bundle = ctx.require(BundleKey); const effectiveAgent = resolveEffectiveAgentRuntime(bundle, ctx); const session = hydrateDurableSession({ - compactionOverrides: { - thresholdPercent: effectiveAgent.thresholdPercent, - }, durable: durableSession, turnAgent: effectiveAgent.turnAgent, }); diff --git a/packages/eve/src/execution/create-session-step.ts b/packages/eve/src/execution/create-session-step.ts index 7c9e4b0f8f..33c9905d14 100644 --- a/packages/eve/src/execution/create-session-step.ts +++ b/packages/eve/src/execution/create-session-step.ts @@ -58,9 +58,6 @@ export async function createSessionStep(input: { ); const session = createSession({ - compactionOverrides: { - thresholdPercent: effectiveAgent.thresholdPercent, - }, continuationToken: input.continuationToken, outputSchema: input.outputSchema, rootSessionId: input.rootSessionId, diff --git a/packages/eve/src/execution/durable-session-store.test.ts b/packages/eve/src/execution/durable-session-store.test.ts index 960be2db9f..c1b556fedf 100644 --- a/packages/eve/src/execution/durable-session-store.test.ts +++ b/packages/eve/src/execution/durable-session-store.test.ts @@ -57,9 +57,6 @@ function buildSession(input: { }): HarnessSession { return { agent: { - compactionModelReference: input.withRefreshableAgent - ? { id: "compaction-model", contextWindowTokens: 200_000 } - : undefined, modelReference: { id: "test-model", contextWindowTokens: 200_000 }, system: "test system", tools: input.withRefreshableAgent diff --git a/packages/eve/src/execution/durable-session-store.ts b/packages/eve/src/execution/durable-session-store.ts index 32e9601062..c6d178f705 100644 --- a/packages/eve/src/execution/durable-session-store.ts +++ b/packages/eve/src/execution/durable-session-store.ts @@ -38,8 +38,7 @@ export interface DurableSessionState { * Durable projection of {@link HarnessSession} embedded in state * snapshots. * - * Omits `agent.modelReference`, `agent.tools`, - * `agent.compactionModelReference`, and the `compaction` thresholds — + * Omits `agent.modelReference`, `agent.tools`, and the `compaction` thresholds — * those are rebuilt every turn from `bundle.turnAgent` by * {@link import("#execution/session.js").hydrateDurableSession}. * Omits `history` too: the session workflow carries it as its own value, so diff --git a/packages/eve/src/execution/effective-agent-config.test.ts b/packages/eve/src/execution/effective-agent-config.test.ts index f4ccfc1c48..dbea141419 100644 --- a/packages/eve/src/execution/effective-agent-config.test.ts +++ b/packages/eve/src/execution/effective-agent-config.test.ts @@ -8,10 +8,6 @@ describe("resolveEffectiveAgentRuntime", () => { it("applies the selected subagent model and runtime settings", () => { const ctx = new ContextContainer(); ctx.set(DynamicSubagentAgentConfigKey, { - compaction: { - model: { id: "anthropic/claude-sonnet-4.5" }, - thresholdPercent: 0.75, - }, description: "Perform deep research.", limits: { sessionTimeoutMs: 120_000 }, model: { id: "anthropic/claude-opus-4.6" }, @@ -23,7 +19,6 @@ describe("resolveEffectiveAgentRuntime", () => { { resolvedAgent: { config: { - compaction: { thresholdPercent: 0.9 }, limits: { sessionTimeoutMs: 60_000 }, }, }, @@ -40,9 +35,7 @@ describe("resolveEffectiveAgentRuntime", () => { expect(effective).toMatchObject({ limits: { sessionTimeoutMs: 120_000 }, - thresholdPercent: 0.75, turnAgent: { - compactionModel: { id: "anthropic/claude-sonnet-4.5" }, model: { id: "anthropic/claude-opus-4.6" }, reasoning: "high", }, @@ -54,10 +47,6 @@ describe("resolveEffectiveAgentRuntime", () => { it("overrides only the selected agent model for a ctx.agent session", () => { const ctx = new ContextContainer(); ctx.set(DynamicSubagentAgentConfigKey, { - compaction: { - model: { id: "anthropic/claude-haiku-4.5" }, - thresholdPercent: 0.75, - }, description: "Perform deep research.", limits: { sessionTimeoutMs: 120_000 }, model: { id: "anthropic/claude-opus-4.6" }, @@ -83,9 +72,7 @@ describe("resolveEffectiveAgentRuntime", () => { expect(effective).toMatchObject({ limits: { sessionTimeoutMs: 120_000 }, - thresholdPercent: 0.75, turnAgent: { - compactionModel: { id: "anthropic/claude-haiku-4.5" }, instructions: ["Research carefully."], model: { id: "openai/gpt-5.5", reasoning: "low" }, reasoning: "high", diff --git a/packages/eve/src/execution/effective-agent-config.ts b/packages/eve/src/execution/effective-agent-config.ts index 0b5ab88517..76c4875eda 100644 --- a/packages/eve/src/execution/effective-agent-config.ts +++ b/packages/eve/src/execution/effective-agent-config.ts @@ -9,7 +9,6 @@ import type { DynamicSubagentAgentConfig } from "#runtime/subagents/dynamic-agen interface EffectiveAgentRuntime { readonly limits?: AgentLimitsDefinition; - readonly thresholdPercent?: number; readonly turnAgent: RuntimeTurnAgent; } @@ -49,12 +48,10 @@ export function resolveEffectiveAgentRuntimeFromConfig( } effective = { limits: bundle.resolvedAgent.config?.limits, - thresholdPercent: bundle.resolvedAgent.config?.compaction?.thresholdPercent, turnAgent: bundle.turnAgent, }; } else { const { - compactionModel: _compiledCompactionModel, configResolver: _configResolver, dynamicModel: _dynamicModel, model: _compiledModel, @@ -62,10 +59,8 @@ export function resolveEffectiveAgentRuntimeFromConfig( } = bundle.turnAgent; effective = { limits: config.limits, - thresholdPercent: config.compaction?.thresholdPercent, turnAgent: { ...turnAgent, - compactionModel: config.compaction?.model, model: config.model, reasoning: config.reasoning, }, diff --git a/packages/eve/src/execution/node-step.ts b/packages/eve/src/execution/node-step.ts index 4995411129..7852a60c36 100644 --- a/packages/eve/src/execution/node-step.ts +++ b/packages/eve/src/execution/node-step.ts @@ -92,6 +92,7 @@ export function createExecutionNodeStep(input: CreateExecutionNodeStepInput): St clearOnly: input.clearOnly, compactOnly: input.compactOnly, controlDelivery: input.controlDelivery, + compactionStrategy: input.node.agent.compaction?.strategy, handleEvent: input.handleEvent, instrumentation: sessionInstrumentation, signInCompletions: input.signInCompletions, diff --git a/packages/eve/src/execution/publish-session-events.ts b/packages/eve/src/execution/publish-session-events.ts index 51d62e3891..1097377468 100644 --- a/packages/eve/src/execution/publish-session-events.ts +++ b/packages/eve/src/execution/publish-session-events.ts @@ -256,7 +256,6 @@ async function publishInSessionScope( }); const effectiveAgent = resolveEffectiveAgentRuntime(ctx.require(BundleKey), ctx); const hydrated = hydrateDurableSession({ - compactionOverrides: { thresholdPercent: effectiveAgent.thresholdPercent }, durable: step.durableSession, turnAgent: effectiveAgent.turnAgent, }); diff --git a/packages/eve/src/execution/session.test.ts b/packages/eve/src/execution/session.test.ts index 48a0fef2c6..07234a1112 100644 --- a/packages/eve/src/execution/session.test.ts +++ b/packages/eve/src/execution/session.test.ts @@ -16,9 +16,6 @@ function createTestTurnAgent(overrides?: Partial): Runti return { id: "test-agent", instructions: ["You are a helpful assistant.", "Be concise."], - compactionModel: { - id: "summary-model", - }, model: { id: "test-model" }, tools: [ { @@ -48,20 +45,6 @@ describe("createCompactionConfig", () => { ).toEqual({ recentWindowSize: 10, threshold: 180_000, - thresholdPercent: 0.9, - }); - }); - - it("uses the authored threshold percent when provided", () => { - expect( - createCompactionConfig({ - contextWindowTokens: 200_000, - thresholdPercent: 0.5, - }), - ).toEqual({ - recentWindowSize: 10, - threshold: 100_000, - thresholdPercent: 0.5, }); }); @@ -69,7 +52,6 @@ describe("createCompactionConfig", () => { expect(createCompactionConfig()).toEqual({ recentWindowSize: 10, threshold: 100_000, - thresholdPercent: 0.9, }); }); }); @@ -84,7 +66,6 @@ describe("createSession", () => { turnAgent: createTestTurnAgent({ reasoning: "high" }), }); - expect(session.agent.compactionModelReference).toEqual({ id: "summary-model" }); expect(session.agent.modelReference).toEqual({ id: "test-model" }); expect(session.agent.reasoning).toBe("high"); expect(session.outputSchema).toEqual(outputSchema); @@ -192,45 +173,6 @@ describe("createSession", () => { expect(session.compaction).toEqual({ recentWindowSize: 10, threshold: 100_000, - thresholdPercent: 0.9, - }); - }); - - it("honors compactionOverrides.thresholdPercent", () => { - const session = createSession({ - compactionOverrides: { thresholdPercent: 0.5 }, - continuationToken: "root-token", - sessionId: "sess-root", - turnAgent: createTestTurnAgent({ - model: { id: "test-model", contextWindowTokens: 200_000 }, - }), - }); - - expect(session.compaction).toEqual({ - recentWindowSize: 10, - threshold: 100_000, - thresholdPercent: 0.5, - }); - }); - - it("copies the compaction model into the refreshed session", () => { - const session = createSession({ - continuationToken: "root-token", - sessionId: "sess-root", - turnAgent: createTestTurnAgent(), - }); - - const refreshed = refreshSessionFromTurnAgent({ - session, - turnAgent: createTestTurnAgent({ - compactionModel: { - id: "updated-summary-model", - }, - }), - }); - - expect(refreshed.agent.compactionModelReference).toEqual({ - id: "updated-summary-model", }); }); @@ -340,9 +282,6 @@ describe("refreshSessionFromTurnAgent", () => { expect(refreshed.history).toEqual([ { content: "previous message", kind: "user", role: "user" }, ]); - expect(refreshed.agent.compactionModelReference).toEqual({ - id: "summary-model", - }); expect(refreshed.agent.modelReference).toEqual({ contextWindowTokens: 200_000, id: "updated-model", @@ -374,9 +313,6 @@ describe("refreshSessionFromTurnAgent", () => { }), }); const refreshed = refreshSessionFromTurnAgent({ - compactionOverrides: { - thresholdPercent: 0.5, - }, session: { ...session, compaction: { @@ -396,8 +332,7 @@ describe("refreshSessionFromTurnAgent", () => { lastKnownPromptMessageCount: 7, requestEnvelopeTokens: 42, recentWindowSize: 10, - threshold: 100_000, - thresholdPercent: 0.5, + threshold: 180_000, }); const restored = hydrateDurableSession({ durable: structuredClone(projectToDurableSession(refreshed)), diff --git a/packages/eve/src/execution/session.ts b/packages/eve/src/execution/session.ts index e237064ee2..42732b2e76 100644 --- a/packages/eve/src/execution/session.ts +++ b/packages/eve/src/execution/session.ts @@ -8,11 +8,10 @@ import type { SessionAgent, SessionToolDefinition, } from "#harness/types.js"; +import { compactionThreshold } from "#harness/compaction/measure.js"; import type { RuntimeTurnAgent } from "#runtime/agent/bootstrap.js"; const DEFAULT_COMPACTION_RECENT_WINDOW_SIZE = 10; -const DEFAULT_COMPACTION_THRESHOLD_PERCENT = 0.9; -const FALLBACK_COMPACTION_THRESHOLD = 100_000; /** * Creates the durable compaction configuration used by one harness session. @@ -21,28 +20,19 @@ export function createCompactionConfig( input: { readonly contextWindowTokens?: number; readonly previous?: CompactionState; - readonly thresholdPercent?: number; } = {}, ): CompactionConfig { - const thresholdPercent = input.thresholdPercent ?? DEFAULT_COMPACTION_THRESHOLD_PERCENT; - const threshold = - input.contextWindowTokens === undefined - ? FALLBACK_COMPACTION_THRESHOLD - : Math.max(1, Math.floor(input.contextWindowTokens * thresholdPercent)); + const threshold = compactionThreshold(input.contextWindowTokens); return { ...input.previous, recentWindowSize: DEFAULT_COMPACTION_RECENT_WINDOW_SIZE, threshold, - thresholdPercent, }; } interface CreateSessionInput { readonly continuationToken: string; - readonly compactionOverrides?: { - readonly thresholdPercent?: number; - }; /** * Optional root session id passed in by the runtime when this * session is a delegated subagent child. `undefined` for top-level @@ -65,7 +55,6 @@ export function createSession(input: CreateSessionInput): HarnessSession { agent: createSessionAgent(turnAgent, createSessionSystemPrompt(turnAgent), tools), compaction: createCompactionConfig({ contextWindowTokens: turnAgent.model?.contextWindowTokens, - thresholdPercent: input.compactionOverrides?.thresholdPercent, }), continuationToken: input.continuationToken, history: [...(turnAgent.initialMessages ?? [])], @@ -88,7 +77,6 @@ function createSessionAgent( tools: readonly SessionToolDefinition[], ): SessionAgent { const base = { - compactionModelReference: turnAgent.compactionModel, reasoning: turnAgent.reasoning, system, tools, @@ -111,9 +99,6 @@ function createSessionAgent( export function refreshSessionFromTurnAgent(input: { readonly session: HarnessSession; readonly turnAgent: RuntimeTurnAgent; - readonly compactionOverrides?: { - readonly thresholdPercent?: number; - }; }): HarnessSession { return { ...input.session, @@ -125,7 +110,6 @@ export function refreshSessionFromTurnAgent(input: { compaction: createCompactionConfig({ contextWindowTokens: input.turnAgent.model?.contextWindowTokens, previous: input.session.compaction, - thresholdPercent: input.compactionOverrides?.thresholdPercent, }), }; } @@ -154,12 +138,7 @@ export function mintSubagentContinuationToken(suffix?: string): string { * informed after rehydration. */ export function projectToDurableSession(session: HarnessSessionBase): DurableSession { - const { - threshold: _threshold, - thresholdPercent: _percent, - recentWindowSize: _size, - ...measured - } = session.compaction; + const { threshold: _threshold, recentWindowSize: _size, ...measured } = session.compaction; return { agent: { system: session.agent.system }, continuationToken: session.continuationToken, @@ -180,9 +159,6 @@ export function projectToDurableSession(session: HarnessSessionBase): DurableSes export function hydrateDurableSession(input: { readonly durable: DurableSession; readonly turnAgent: RuntimeTurnAgent; - readonly compactionOverrides?: { - readonly thresholdPercent?: number; - }; }): HarnessSessionBase { const { durable, turnAgent } = input; const { agent, compaction, ...saved } = durable; @@ -192,7 +168,6 @@ export function hydrateDurableSession(input: { compaction: createCompactionConfig({ contextWindowTokens: turnAgent.model?.contextWindowTokens, previous: compaction, - thresholdPercent: input.compactionOverrides?.thresholdPercent, }), }; } diff --git a/packages/eve/src/execution/session/end-sandbox-step.ts b/packages/eve/src/execution/session/end-sandbox-step.ts index 4de4cfb03a..3faf2b3b1c 100644 --- a/packages/eve/src/execution/session/end-sandbox-step.ts +++ b/packages/eve/src/execution/session/end-sandbox-step.ts @@ -34,7 +34,6 @@ export async function endSessionSandboxStep(input: { const durable = readDurableSession(input.sessionState); const effectiveAgent = resolveEffectiveAgentRuntime(ctx.require(BundleKey), ctx); const session = hydrateDurableSession({ - compactionOverrides: { thresholdPercent: effectiveAgent.thresholdPercent }, durable, turnAgent: effectiveAgent.turnAgent, }); diff --git a/packages/eve/src/execution/session/prewarm-step.ts b/packages/eve/src/execution/session/prewarm-step.ts index d1c784660d..7d32feda79 100644 --- a/packages/eve/src/execution/session/prewarm-step.ts +++ b/packages/eve/src/execution/session/prewarm-step.ts @@ -40,7 +40,6 @@ async function prewarmSession(input: SessionPrewarmInput): Promise<{ const ctx = await deserializeContext(input.serializedContext); const effectiveAgent = resolveEffectiveAgentRuntime(ctx.require(BundleKey), ctx); const session = hydrateDurableSession({ - compactionOverrides: { thresholdPercent: effectiveAgent.thresholdPercent }, durable: readDurableSession(input.sessionState), turnAgent: effectiveAgent.turnAgent, }); diff --git a/packages/eve/src/execution/session/turn-step.ts b/packages/eve/src/execution/session/turn-step.ts index af288c7966..b3f7022f33 100644 --- a/packages/eve/src/execution/session/turn-step.ts +++ b/packages/eve/src/execution/session/turn-step.ts @@ -239,9 +239,6 @@ async function runSessionStepBody( if (requester !== undefined) ctx.set(AuthKey, requester); let initialSession: HarnessSession = { ...hydrateDurableSession({ - compactionOverrides: { - thresholdPercent: effectiveAgent.thresholdPercent, - }, durable: durableSession, turnAgent: effectiveAgent.turnAgent, }), @@ -610,9 +607,6 @@ async function runSessionStepBody( signInCompletions: readonly AuthorizationChallenge[] | undefined, ): Promise => { const refreshedSession = refreshSessionFromTurnAgent({ - compactionOverrides: { - thresholdPercent: effectiveAgent.thresholdPercent, - }, session: lifecycleSession, turnAgent: effectiveAgent.turnAgent, }); diff --git a/packages/eve/src/harness/compaction/clone-messages.ts b/packages/eve/src/harness/compaction/clone-messages.ts new file mode 100644 index 0000000000..fa41f196fe --- /dev/null +++ b/packages/eve/src/harness/compaction/clone-messages.ts @@ -0,0 +1,19 @@ +/** + * Deep-copies messages for a strategy to edit freely. Plain objects, arrays and binary data are + * copied; a `URL` is immutable and kept as it is, which `structuredClone` cannot do. + */ +export function cloneMessages(value: T): T { + if (Array.isArray(value)) return value.map(cloneMessages) as T; + if (value instanceof Uint8Array) return value.slice() as T; + if (value instanceof ArrayBuffer) return value.slice(0) as T; + if ( + typeof value === "object" && + value !== null && + Object.getPrototypeOf(value) === Object.prototype + ) { + return Object.fromEntries( + Object.entries(value).map(([key, entry]) => [key, cloneMessages(entry)]), + ) as T; + } + return value; +} diff --git a/packages/eve/src/harness/compaction/measure.ts b/packages/eve/src/harness/compaction/measure.ts new file mode 100644 index 0000000000..5d5252a638 --- /dev/null +++ b/packages/eve/src/harness/compaction/measure.ts @@ -0,0 +1,93 @@ +import type { ModelMessage } from "ai"; + +import { estimateTokens } from "#harness/token-estimate.js"; +import type { CompactionConfig } from "#harness/types.js"; + +/** + * Best available input-token count: the model-reported count from the last + * step, plus a rough character-based estimate of whatever messages have been + * appended since. + */ +export function getInputTokenCount( + messages: readonly ModelMessage[], + config: CompactionConfig, + requestEnvelopeTokens = 0, + previousEnvelopeTokens = 0, +): number { + const prior = config.lastKnownInputTokens; + const priorCount = config.lastKnownPromptMessageCount; + + if ( + prior === undefined || + priorCount === undefined || + !Number.isInteger(priorCount) || + priorCount < 0 || + priorCount > messages.length + ) { + return estimateTokens(messages) + requestEnvelopeTokens; + } + + const appended = messages.slice(priorCount); + return ( + prior + + (appended.length === 0 ? 0 : estimateTokens(appended)) + + Math.max(0, requestEnvelopeTokens - previousEnvelopeTokens) + ); +} + +/** The settings without the last measured prompt, which a rewritten history no longer matches. */ +export function withoutPromptMeasurement(config: CompactionConfig): CompactionConfig { + const { lastKnownInputTokens: _input, lastKnownPromptMessageCount: _count, ...rest } = config; + return rest; +} + +/** + * The settings with a model-reported prompt measurement and the envelope around it. Without a + * usable measurement, the next count is an estimate, and no envelope baseline applies. + */ +export function withPromptMeasurement( + config: CompactionConfig, + measured: { + readonly inputTokens?: number; + readonly promptMessageCount?: number; + readonly requestEnvelopeTokens?: number; + }, +): CompactionConfig { + const { requestEnvelopeTokens: _envelope, ...next } = withoutPromptMeasurement(config); + if (measured.inputTokens === undefined || measured.promptMessageCount === undefined) return next; + return { + ...next, + lastKnownInputTokens: measured.inputTokens, + lastKnownPromptMessageCount: measured.promptMessageCount, + ...(measured.requestEnvelopeTokens !== undefined && { + requestEnvelopeTokens: measured.requestEnvelopeTokens, + }), + }; +} + +/** + * The overflow guard: true when the next request's input tokens pass the limit. The kernel + * runs it before every model call, and no strategy can configure or turn it off. + */ +export function exceedsLimit(measured: { + readonly inputTokens: number; + readonly threshold: number; +}): boolean { + return measured.inputTokens > measured.threshold; +} + +/** The share of the context window a request may fill: the core's fixed overflow guard. */ +const OVERFLOW_GUARD_RATIO = 0.9; + +export function overflowThreshold(contextWindowTokens: number): number { + return Math.max(1, Math.floor(contextWindowTokens * OVERFLOW_GUARD_RATIO)); +} + +const FALLBACK_COMPACTION_THRESHOLD = 100_000; + +/** The input tokens a request may carry before the history compacts, for a model's window. */ +export function compactionThreshold(contextWindowTokens?: number): number { + return contextWindowTokens === undefined + ? FALLBACK_COMPACTION_THRESHOLD + : overflowThreshold(contextWindowTokens); +} diff --git a/packages/eve/src/harness/compaction/message.ts b/packages/eve/src/harness/compaction/message.ts new file mode 100644 index 0000000000..57fc235b4a --- /dev/null +++ b/packages/eve/src/harness/compaction/message.ts @@ -0,0 +1,8 @@ +import type { UserContent } from "ai"; + +import { createFrameworkUserMessage } from "#harness/messages.js"; + +/** Builds the user-role message a strategy returns, such as a checkpoint marker. */ +export function createCompactionUserMessage(content: UserContent) { + return createFrameworkUserMessage("context.compaction", content); +} diff --git a/packages/eve/src/harness/compaction/over-limit-error.ts b/packages/eve/src/harness/compaction/over-limit-error.ts index 9cbe4f66cc..272cd8a52a 100644 --- a/packages/eve/src/harness/compaction/over-limit-error.ts +++ b/packages/eve/src/harness/compaction/over-limit-error.ts @@ -5,3 +5,11 @@ export class CompactionOverLimitError extends Error { override readonly name = "CompactionOverLimitError"; } + +/** + * A strategy returned a history the harness cannot use: it dropped a tool call or result, or does + * not start with a user message. A retry would run the strategy on the same history again. + */ +export class CompactionInvalidResultError extends Error { + override readonly name = "CompactionInvalidResultError"; +} diff --git a/packages/eve/src/harness/compaction/step.integration.test.ts b/packages/eve/src/harness/compaction/step.integration.test.ts index 7a8d3ff109..085a5cc1fc 100644 --- a/packages/eve/src/harness/compaction/step.integration.test.ts +++ b/packages/eve/src/harness/compaction/step.integration.test.ts @@ -3,7 +3,7 @@ import { MockLanguageModelV3 } from "ai/test"; import { afterEach, describe, expect, it, vi } from "vitest"; import { ContextContainer, contextStorage } from "#context/container.js"; -import * as compaction from "#harness/compaction/engine.js"; +import * as compaction from "#harness/compaction/strategies/fixed-tail/engine.js"; import { CompactionOverLimitError } from "#harness/compaction/over-limit-error.js"; import type { HarnessToolDefinition } from "#harness/execute-tool.js"; import { createToolLoopHarness } from "#harness/tool-loop.js"; diff --git a/packages/eve/src/harness/compaction/step.ts b/packages/eve/src/harness/compaction/step.ts index 35cdb3cbd6..4edc142a5d 100644 --- a/packages/eve/src/harness/compaction/step.ts +++ b/packages/eve/src/harness/compaction/step.ts @@ -1,12 +1,22 @@ -import { withoutPromptMeasurement } from "#harness/compaction/engine.js"; +import { + exceedsLimit, + getInputTokenCount, + withoutPromptMeasurement, +} from "#harness/compaction/measure.js"; +import type { CompactionContext, CompactionStrategy } from "#harness/compaction/strategy.js"; +import { resolveMockAuthoredRuntimeModel } from "#runtime/agent/mock-model-adapter.js"; +import { cloneMessages } from "#harness/compaction/clone-messages.js"; +import { fixedTail } from "#harness/compaction/strategies/fixed-tail/index.js"; import { type LanguageModel, type ModelMessage, streamText, type TelemetryOptions } from "ai"; -import { CompactionOverLimitError } from "#harness/compaction/over-limit-error.js"; +import { + CompactionInvalidResultError, + CompactionOverLimitError, +} from "#harness/compaction/over-limit-error.js"; import { estimateTokens } from "#harness/token-estimate.js"; import { createLogger, logError } from "#internal/logging.js"; import { AuthKey, HistoryStateKey } from "#context/keys.js"; import { - type CompactionConfig, type HarnessSession, requireSessionModelReference, type StepResult, @@ -26,12 +36,6 @@ import { resolveEffectiveRuntimeModel, withGatewayTraceContext, } from "#harness/model-call/model.js"; -import { - compactMessages, - type CompactionSummarizer, - getInputTokenCount, - shouldCompact, -} from "#harness/compaction/engine.js"; import { contextStorage } from "#context/container.js"; import { takeCompactionCaptureFailure } from "#reactions/kinds/memory.js"; import type { SessionEvent } from "#protocol/session-event.js"; @@ -112,16 +116,12 @@ export async function compactHistory(step: Step): Promise { const runId = nextRunId(step.view().projection); let outcome: "applied" | "failed" = "applied"; try { - await step.apply([runRequested({ changeId }, runId)], messages); const resolvedModel = await resolveEffectiveRuntimeModel({ config, ctx: step.ctx, session: step.session, }); step.session = resolvedModel.session; - const summaryModel = - step.session.agent.compactionModelReference ?? requireSessionModelReference(step.session); - await step.publish(runStarted({ attempt: 0, modelId: summaryModel.id, runId })); const compacted = await maybeCompact({ abortSignal: config.abortSignal, auth: step.ctx?.get(AuthKey) ?? null, @@ -140,6 +140,7 @@ export async function compactHistory(step: Step): Promise { session: step.session, prepareTelemetry: (runId) => step.instrumentation?.prepareAttempt({ runId, attempt: 0 }).telemetry, + strategy: config.compactionStrategy, traceContextHeaders: step.instrumentation?.gatewayTraceContextHeaders, }); step.session = compacted.session; @@ -243,7 +244,7 @@ async function compactOnce(input: { readonly betweenTurns?: boolean; /** * The change a compaction makes, and the run that writes its summary. `announced` when the - * caller already published the change's start and the run's request and start. + * caller already published the change's start. The summary run opens with its first call. */ readonly change: { readonly changeId: string; @@ -266,68 +267,99 @@ async function compactOnce(input: { readonly resolveModel: ToolLoopHarnessConfig["resolveModel"]; readonly runtimeIdentity?: ToolLoopHarnessConfig["runtimeIdentity"]; readonly session: HarnessSession; + readonly strategy?: CompactionStrategy; readonly telemetry?: TelemetryOptions; readonly prepareTelemetry?: (runId: string) => TelemetryOptions | undefined; readonly traceContextHeaders?: () => Record | undefined; }): Promise<{ readonly compacted: boolean; + /** The request was over the limit, so a failed compaction cannot let the model call go on. */ + readonly overLimit?: boolean; /** The summary failed; `session` still counts the summary calls that finished. */ readonly failure?: { readonly error: unknown }; readonly messages: HarnessModelMessage[]; readonly session: HarnessSession; }> { const { emissionState, publish } = input; + const strategy = input.strategy ?? fixedTail(); let messages = input.messages; let session = input.session; const promptMessages = input.promptMessages ?? messages; const projectedPromptMessages = validateHarnessModelMessages(promptMessages); - const needsSummary = - input.force === true || - shouldCompact( + const requestEnvelopeTokens = input.requestEnvelopeTokens ?? 0; + const ordinary = validateHarnessModelMessages(messages); + // Whatever the model sees besides the history (memory, client context) is kept whole, so it takes room like the envelope. + const protectedTokens = Math.max( + 0, + estimateTokens(projectedPromptMessages) - estimateTokens(ordinary), + ); + const measured = { + inputTokens: getInputTokenCount( projectedPromptMessages, session.compaction, input.requestEnvelopeTokens, session.compaction.requestEnvelopeTokens, - ); + ), + recentWindowSize: session.compaction.recentWindowSize, + requestEnvelopeTokens: requestEnvelopeTokens + protectedTokens, + threshold: session.compaction.threshold, + }; + const overLimit = projectedPromptMessages.length > 0 && exceedsLimit(measured); + const needsSummary = input.force === true || overLimit; if (!needsSummary) { return { compacted: false, messages, session }; } - const modelReference = requireSessionModelReference(session); - const compactionModelReference = session.agent.compactionModelReference ?? modelReference; - const model = - compactionModelReference === modelReference - ? input.model - : await input.resolveModel(compactionModelReference); - const profile = resolveModelProfile(model); - const providerOptions = resolveCallProviderOptions({ - auth: input.auth, - profile, - providerOptions: compactionModelReference.providerOptions, - session, - }) as Parameters[0]["providerOptions"]; - const call = { - abortSignal: input.abortSignal, - headers: buildGatewayAttributionHeaders(profile, input.runtimeIdentity), - model: withGatewayTraceContext(model, profile, input.traceContextHeaders), - providerOptions, - telemetry: input.telemetry && { ...input.telemetry, functionId: "eve.compaction" }, + // Resolved on the first summary, so a strategy that never summarizes needs no compaction model. + const summaryCalls = new Map< + string | LanguageModel | undefined, + Promise + >(); + const resolveSummaryCall = (choice: string | LanguageModel | undefined) => { + let call = summaryCalls.get(choice); + if (call === undefined) { + call = (async () => { + const model = + choice === undefined + ? input.model + : typeof choice === "string" + ? await input.resolveModel({ id: choice }) + : (resolveMockAuthoredRuntimeModel({ id: choice.modelId }) ?? choice); + const profile = resolveModelProfile(model); + const providerOptions = resolveCallProviderOptions({ + auth: input.auth, + profile, + providerOptions: + choice === undefined + ? requireSessionModelReference(session).providerOptions + : undefined, + session, + }) as Parameters[0]["providerOptions"]; + return { + abortSignal: input.abortSignal, + headers: buildGatewayAttributionHeaders(profile, input.runtimeIdentity), + model: withGatewayTraceContext(model, profile, input.traceContextHeaders), + modelId: gatewayModelId(model) ?? "unknown", + providerOptions, + telemetry: input.telemetry && { ...input.telemetry, functionId: "eve.compaction" }, + }; + })(); + summaryCalls.set(choice, call); + } + return call; }; let summaryTelemetry: TelemetryOptions | undefined; const summaryUsage: (TokenUsageDelta | undefined)[] = []; const { change } = input; // The summary run starts with its first call: `compactMessages` can keep the history without one. - let summaryRunOpen = change.announced === true; - const summarize = async (prompt: CompactionSummaryPrompt) => { + let summaryRunOpen = false; + const summarize = async ({ model: choice, ...prompt }: CompactionSummaryPrompt) => { + const { modelId, ...call } = await resolveSummaryCall(choice); if (!summaryRunOpen) { summaryRunOpen = true; await publish([ runRequested({ changeId: change.changeId }, change.summaryRunId), - runStarted({ - attempt: 0, - modelId: gatewayModelId(model) ?? "unknown", - runId: change.summaryRunId, - }), + runStarted({ attempt: 0, modelId, runId: change.summaryRunId }), ]); } const summary = await runModelCallWithRetries( @@ -349,49 +381,22 @@ async function compactOnce(input: { return summary.text; }; - { - if (change.announced !== true) { - const inputTokens = getInputTokenCount( - projectedPromptMessages, - session.compaction, - input.requestEnvelopeTokens, - session.compaction.requestEnvelopeTokens, - ); - await publish( - contextStarted({ - cause: change.cause, - changeId: change.changeId, - kind: "compaction", - trigger: input.force === true || inputTokens === null ? undefined : { inputTokens }, - turnId: change.turnId, - }), - projectedPromptMessages, - ); - } + if (change.announced !== true) { + await publish( + contextStarted({ + cause: change.cause, + changeId: change.changeId, + kind: "compaction", + trigger: input.force === true ? undefined : { inputTokens: measured.inputTokens }, + turnId: change.turnId, + }), + projectedPromptMessages, + ); } if (needsSummary) summaryTelemetry = input.prepareTelemetry?.(change.summaryRunId); - const ordinary = validateHarnessModelMessages(messages); - const requestEnvelopeTokens = input.requestEnvelopeTokens ?? 0; - // Whatever the model sees besides the history (memory, client context) is kept whole, so it takes room like the envelope. - const protectedTokens = Math.max( - 0, - estimateTokens(projectedPromptMessages) - estimateTokens(ordinary), - ); - const historyCompaction: CompactionConfig = { - ...session.compaction, - threshold: Math.max(1, session.compaction.threshold - requestEnvelopeTokens - protectedTokens), - lastKnownInputTokens: - session.compaction.lastKnownInputTokens === undefined - ? undefined - : Math.max( - 0, - session.compaction.lastKnownInputTokens - - Math.min(session.compaction.requestEnvelopeTokens ?? 0, requestEnvelopeTokens), - ), - }; - let compactedOrdinary: ModelMessage[] = [...ordinary]; + let compactedOrdinary: readonly HarnessModelMessage[] = ordinary; // A memory capture before compaction ran in the commit that started it. When it failed, the // compaction fails before it rewrites the history the capture couldn't keep. const captureFailure = takeCompactionCaptureFailure(input.ctx); @@ -404,25 +409,31 @@ async function compactOnce(input: { requestEnvelopeTokens, threshold: session.compaction.threshold, }); - compactedOrdinary = await compactMessages( - [...ordinary], - historyCompaction, + const context: CompactionContext = { + ...measured, + forced: input.force === true, + abortSignal: input.abortSignal, + estimateTokens, summarize, - input.force === true, - input.requestEnvelopeTokens === undefined - ? undefined - : Math.max( - 0, - getInputTokenCount( - projectedPromptMessages, - session.compaction, - requestEnvelopeTokens, - session.compaction.requestEnvelopeTokens, - ) - - requestEnvelopeTokens - - protectedTokens, - ), - ); + }; + compactedOrdinary = await strategy.compact(cloneMessages(ordinary), context); + validateCompactionResult(ordinary, compactedOrdinary); + // Judged on the estimate alone: the provider's count can't say which messages its excess + // belongs to. A compaction that was needed because the history is over its limit must + // shrink it; one that doesn't would only run again on the next step. + const originalEstimate = estimateTokens(ordinary); + const resultEstimate = estimateTokens(compactedOrdinary); + if (overLimit && resultEstimate >= originalEstimate) { + throw new CompactionOverLimitError( + `The compacted history is no smaller than the history (about ${resultEstimate} tokens), which is over the limit.`, + ); + } + const resultTokens = resultEstimate + context.requestEnvelopeTokens; + if (exceedsLimit({ inputTokens: resultTokens, threshold: context.threshold })) { + throw new CompactionOverLimitError( + `The compacted history is still over the limit (about ${resultTokens} of ${context.threshold} tokens).`, + ); + } } } catch (error) { failure = { error }; @@ -441,7 +452,7 @@ async function compactOnce(input: { ...(usage !== undefined && { spent: { runId, usage } }), }), ); - return { compacted: false, failure, messages: input.messages, session }; + return { compacted: false, failure, messages: input.messages, overLimit, session }; } messages = validateHarnessModelMessages(compactedOrdinary); @@ -474,7 +485,11 @@ export function compactedTurnInput( return messages.findLast((message) => isUserModelMessage(message) && message.kind === "user"); } -type CompactionSummaryPrompt = Parameters[0]; +type CompactionSummaryPrompt = Parameters[0]; +type SummaryCall = Pick< + Parameters[0], + "abortSignal" | "headers" | "model" | "providerOptions" | "telemetry" +>; /** * Calls the compaction model the way a step calls its model: streamed, with stream errors @@ -508,3 +523,58 @@ async function streamCompactionSummary( usage: extractTokenUsageDelta({ costUsd: extractGatewayCostUsd(providerMetadata), usage }), }; } + +/** + * A strategy's result must keep every tool call beside its result, keep the calls the turn is + * waiting on, and start with a user message, or the next provider call rejects the history. + */ +function validateCompactionResult( + original: readonly HarnessModelMessage[], + result: readonly HarnessModelMessage[], +): void { + const calls = (messages: readonly HarnessModelMessage[]) => toolPartIds(messages, "tool-call"); + const results = (messages: readonly HarnessModelMessage[]) => + toolPartIds(messages, "tool-result"); + const keptCalls = calls(result); + const keptResults = results(result); + for (const id of keptCalls) { + if (!keptResults.has(id) && results(original).has(id)) { + throw new CompactionInvalidResultError( + `The compacted history dropped the result of tool call ${id}.`, + ); + } + } + for (const id of keptResults) { + if (!keptCalls.has(id)) { + throw new CompactionInvalidResultError( + `The compacted history kept the result of tool call ${id} without its call.`, + ); + } + } + const waiting = [...calls(original)].filter((id) => !results(original).has(id)); + for (const id of waiting) { + if (!keptCalls.has(id)) { + throw new CompactionInvalidResultError( + `The compacted history dropped tool call ${id}, which is still waiting.`, + ); + } + } + const first = result[0]; + if (first !== undefined && first.role !== "user") { + throw new CompactionInvalidResultError("The compacted history must start with a user message."); + } +} + +function toolPartIds( + messages: readonly HarnessModelMessage[], + type: "tool-call" | "tool-result", +): Set { + const ids = new Set(); + for (const message of messages) { + if (typeof message.content === "string") continue; + for (const part of message.content) { + if (part.type === type && "toolCallId" in part) ids.add(part.toolCallId); + } + } + return ids; +} diff --git a/packages/eve/src/harness/compaction/engine.test.ts b/packages/eve/src/harness/compaction/strategies/fixed-tail/engine.test.ts similarity index 98% rename from packages/eve/src/harness/compaction/engine.test.ts rename to packages/eve/src/harness/compaction/strategies/fixed-tail/engine.test.ts index acee3a2968..535c7918c9 100644 --- a/packages/eve/src/harness/compaction/engine.test.ts +++ b/packages/eve/src/harness/compaction/strategies/fixed-tail/engine.test.ts @@ -1,14 +1,55 @@ import type { ModelMessage } from "ai"; import { describe, expect, it, vi } from "vitest"; -import { COMPACTION_PROMPT_ENVELOPE } from "#harness/compaction/prompt.js"; -import { compactMessages, getInputTokenCount, shouldCompact } from "#harness/compaction/engine.js"; +import { COMPACTION_PROMPT_ENVELOPE } from "#harness/compaction/strategies/fixed-tail/prompt.js"; +import { + exceedsLimit as exceedsThreshold, + getInputTokenCount, +} from "#harness/compaction/measure.js"; +import { compactMessages as compactWithMeasure } from "#harness/compaction/strategies/fixed-tail/engine.js"; import { createFrameworkUserMessage } from "#harness/messages.js"; import { estimateTokens } from "#harness/token-estimate.js"; import type { CompactionConfig } from "#harness/types.js"; import { encodeSandboxRef } from "#internal/attachments/sandbox-refs.js"; import { pngBytes } from "#internal/testing/media-fixtures.js"; +/** The strategy's trigger over a history: the core measures, `exceedsThreshold` decides. */ +function shouldCompact( + messages: readonly ModelMessage[], + config: CompactionConfig, + requestEnvelopeTokens = 0, + previousEnvelopeTokens = 0, +): boolean { + return ( + messages.length > 0 && + exceedsThreshold({ + inputTokens: getInputTokenCount( + messages, + config, + requestEnvelopeTokens, + previousEnvelopeTokens, + ), + threshold: config.threshold, + }) + ); +} + +/** Compacts with the history's own token count, as the strategy does when nothing was measured. */ +function compactMessages( + messages: ModelMessage[], + config: CompactionConfig, + summarize: Parameters[2], + forceSummary = false, +) { + return compactWithMeasure( + messages, + config, + summarize, + forceSummary, + getInputTokenCount(messages, config), + ); +} + const config: CompactionConfig = { recentWindowSize: 2, threshold: 100, @@ -262,34 +303,6 @@ describe("shouldCompact", () => { expect(shouldCompact(messages, config)).toBe(true); }); - it("uses the fixed prompt envelope in threshold accounting", () => { - const messages: ModelMessage[] = [{ content: "Continue the investigation.", role: "user" }]; - const compaction: CompactionConfig = { - lastKnownInputTokens: 200, - lastKnownPromptMessageCount: messages.length, - recentWindowSize: 2, - threshold: 1_000, - }; - const activeInputTokens = getInputTokenCount(messages, compaction); - const promptEnvelopeTokens = estimateTokens([ - { content: COMPACTION_PROMPT_ENVELOPE.system, role: "system" }, - createFrameworkUserMessage("context.compaction", COMPACTION_PROMPT_ENVELOPE.prompt), - ] satisfies ModelMessage[]); - - expect( - shouldCompact(messages, { - ...compaction, - threshold: activeInputTokens + promptEnvelopeTokens, - }), - ).toBe(false); - expect( - shouldCompact(messages, { - ...compaction, - threshold: activeInputTokens + promptEnvelopeTokens - 1, - }), - ).toBe(true); - }); - it("does not compact an empty history based on prompt overhead alone", () => { expect(shouldCompact([], { ...config, threshold: 0 })).toBe(false); }); diff --git a/packages/eve/src/harness/compaction/engine.ts b/packages/eve/src/harness/compaction/strategies/fixed-tail/engine.ts similarity index 88% rename from packages/eve/src/harness/compaction/engine.ts rename to packages/eve/src/harness/compaction/strategies/fixed-tail/engine.ts index dcda057c09..2120e26cef 100644 --- a/packages/eve/src/harness/compaction/engine.ts +++ b/packages/eve/src/harness/compaction/strategies/fixed-tail/engine.ts @@ -10,7 +10,7 @@ import { sliceUtf16Safe, stubContentOutputFileParts, TRANSCRIPT_PAYLOAD_LIMIT, -} from "#harness/compaction/prompt.js"; +} from "#harness/compaction/strategies/fixed-tail/prompt.js"; import { createFrameworkUserMessage, isFrameworkUserMessage } from "#harness/messages.js"; import { CompactionOverLimitError } from "#harness/compaction/over-limit-error.js"; import { estimateTokens } from "#harness/token-estimate.js"; @@ -25,6 +25,11 @@ import type { CompactionConfig } from "#harness/types.js"; */ const KEPT_TAIL_TOKENS = 20_000; +/** The session's compaction config, plus how many tokens of the latest conversation to keep. */ +export interface FixedTailConfig extends CompactionConfig { + readonly recentTokens?: number; +} + /** * Element type of a non-string `ModelMessage.content` array. */ @@ -38,86 +43,6 @@ const COMPACTION_PROMPT_OVERHEAD_TOKENS = estimateTokens([ createFrameworkUserMessage("context.compaction", COMPACTION_PROMPT_ENVELOPE.prompt), ] satisfies ModelMessage[]); -/** - * Best available input-token count: the model-reported count from the last - * step, plus a rough character-based estimate of whatever messages have been - * appended since. - */ -export function getInputTokenCount( - messages: readonly ModelMessage[], - config: CompactionConfig, - requestEnvelopeTokens = 0, - previousEnvelopeTokens = 0, -): number { - const prior = config.lastKnownInputTokens; - const priorCount = config.lastKnownPromptMessageCount; - - if ( - prior === undefined || - priorCount === undefined || - !Number.isInteger(priorCount) || - priorCount < 0 || - priorCount > messages.length - ) { - return estimateTokens(messages) + requestEnvelopeTokens; - } - - const appended = messages.slice(priorCount); - return ( - prior + - (appended.length === 0 ? 0 : estimateTokens(appended)) + - Math.max(0, requestEnvelopeTokens - previousEnvelopeTokens) - ); -} - -/** The settings without the last measured prompt, which a rewritten history no longer matches. */ -export function withoutPromptMeasurement(config: CompactionConfig): CompactionConfig { - const { lastKnownInputTokens: _input, lastKnownPromptMessageCount: _count, ...rest } = config; - return rest; -} - -/** - * The settings with a model-reported prompt measurement and the envelope around it. Without a - * usable measurement, the next count is an estimate, and no envelope baseline applies. - */ -export function withPromptMeasurement( - config: CompactionConfig, - measured: { - readonly inputTokens?: number; - readonly promptMessageCount?: number; - readonly requestEnvelopeTokens?: number; - }, -): CompactionConfig { - const { requestEnvelopeTokens: _envelope, ...next } = withoutPromptMeasurement(config); - if (measured.inputTokens === undefined || measured.promptMessageCount === undefined) return next; - return { - ...next, - lastKnownInputTokens: measured.inputTokens, - lastKnownPromptMessageCount: measured.promptMessageCount, - ...(measured.requestEnvelopeTokens !== undefined && { - requestEnvelopeTokens: measured.requestEnvelopeTokens, - }), - }; -} - -/** - * Returns true when the message history and fixed compaction-prompt envelope - * exceed the compaction threshold. - */ -export function shouldCompact( - messages: readonly ModelMessage[], - config: CompactionConfig, - requestEnvelopeTokens = 0, - previousEnvelopeTokens = 0, -): boolean { - return ( - messages.length > 0 && - getInputTokenCount(messages, config, requestEnvelopeTokens, previousEnvelopeTokens) + - COMPACTION_PROMPT_OVERHEAD_TOKENS > - config.threshold - ); -} - /** Summarizes one compaction prompt with the compaction model, returning the summary text. */ export type CompactionSummarizer = (prompt: { readonly messages: ModelMessage[]; @@ -216,10 +141,10 @@ function evaluateThreshold( */ export async function compactMessages( messages: ModelMessage[], - config: CompactionConfig, + config: FixedTailConfig, summarize: CompactionSummarizer, - forceSummary = false, - historyInputTokenCount?: number, + forceSummary: boolean, + historyInputTokenCount: number, ): Promise { const { conversation, previousCheckpoint } = extractPreviousCheckpoint(messages); const budget = keptTailBudget(config); @@ -254,10 +179,7 @@ export async function compactMessages( // Capping preserves most of the measured prompt. Retain any known // underestimate while crediting only the estimated tokens it removes. // A new summary replaces that prompt, so it uses its own estimate below. - const tokenEstimateAdjustment = Math.max( - 0, - (historyInputTokenCount ?? getInputTokenCount(messages, config)) - estimateTokens(messages), - ); + const tokenEstimateAdjustment = Math.max(0, historyInputTokenCount - estimateTokens(messages)); for (const heuristic of COMPACTION_HEURISTICS) { const outcome = heuristic({ config, @@ -711,8 +633,8 @@ function assistantMessageText(message: ModelMessage): string { .trim(); } -function keptTailBudget(config: CompactionConfig): number { - return Math.min(KEPT_TAIL_TOKENS, Math.floor(config.threshold / 2)); +function keptTailBudget(config: FixedTailConfig): number { + return Math.min(config.recentTokens ?? KEPT_TAIL_TOKENS, Math.floor(config.threshold / 2)); } function selectRecentWindowSize( diff --git a/packages/eve/src/harness/compaction/strategies/fixed-tail/index.test.ts b/packages/eve/src/harness/compaction/strategies/fixed-tail/index.test.ts new file mode 100644 index 0000000000..f787c8172e --- /dev/null +++ b/packages/eve/src/harness/compaction/strategies/fixed-tail/index.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it } from "vitest"; + +import type { CompactionContext } from "#harness/compaction/strategy.js"; +import { fixedTail } from "#harness/compaction/strategies/fixed-tail/index.js"; +import type { HarnessModelMessage } from "#harness/messages.js"; +import { estimateTokens } from "#harness/token-estimate.js"; + +const chunk = "x".repeat(4_000); + +function history(): HarnessModelMessage[] { + return Array.from({ length: 8 }, (_, index) => + index % 2 === 0 + ? ({ content: `${index} ${chunk}`, kind: "user", role: "user" } as const) + : ({ content: `${index} ${chunk}`, role: "assistant" } as const), + ) as HarnessModelMessage[]; +} + +function contextOf(threshold: number): CompactionContext { + return { + estimateTokens, + forced: false, + inputTokens: threshold + 1_000, + recentWindowSize: 20, + requestEnvelopeTokens: 0, + summarize: async () => "summary", + threshold, + }; +} + +async function keptVerbatim(options: Parameters[0], threshold: number) { + const result = await fixedTail(options).compact(history(), contextOf(threshold)); + return result.filter((message) => JSON.stringify(message.content).includes("xxxx")).length; +} + +describe("fixedTail options", () => { + it("keeps up to recentTokens of the latest conversation, 20,000 by default", async () => { + const byDefault = await keptVerbatim({}, 100_000); + const small = await keptVerbatim({ recentTokens: 1_100 }, 100_000); + const none = await keptVerbatim({ recentTokens: 1 }, 100_000); + expect(small).toBeGreaterThan(none); + expect(byDefault).toBeGreaterThan(small); + }); + + it.each([{ recentTokens: 0 }, { recentTokens: 1.5 }])("rejects %o", (options) => { + expect(() => fixedTail(options)).toThrow(/must be a positive integer/); + }); +}); diff --git a/packages/eve/src/harness/compaction/strategies/fixed-tail/index.ts b/packages/eve/src/harness/compaction/strategies/fixed-tail/index.ts new file mode 100644 index 0000000000..6394843f07 --- /dev/null +++ b/packages/eve/src/harness/compaction/strategies/fixed-tail/index.ts @@ -0,0 +1,46 @@ +import type { LanguageModel } from "ai"; + +import type { CompactionStrategy } from "#harness/compaction/strategy.js"; +import { compactMessages } from "#harness/compaction/strategies/fixed-tail/engine.js"; +import { COMPACTION_DEFINITION_BRAND } from "#shared/compaction-definition.js"; +import { validateHarnessModelMessages } from "#harness/messages.js"; + +export interface FixedTailOptions { + /** The model that writes the summary: a gateway model id or a language model. The session's model by default. */ + readonly model?: string | LanguageModel; + /** How many tokens of the latest conversation stay word for word. 20,000 by default, and at most half the compaction limit. */ + readonly recentTokens?: number; +} + +function expectPositiveInteger(name: string, value: number | undefined): void { + if (value !== undefined && (!Number.isInteger(value) || value <= 0)) { + throw new Error(`fixedTail: "${name}" must be a positive integer, got ${value}.`); + } +} + +/** + * The default strategy: when the history nears the context window, a summary + * of everything but the latest conversation (`recentTokens`, 20,000 by default, + * kept word for word) replaces it. + */ +export function fixedTail(options: FixedTailOptions = {}): CompactionStrategy { + const { model, recentTokens } = options; + expectPositiveInteger("recentTokens", recentTokens); + const strategy: CompactionStrategy = { + async compact(history, context) { + const messages = await compactMessages( + [...history], + { + recentWindowSize: context.recentWindowSize, + ...(recentTokens !== undefined && { recentTokens }), + threshold: Math.max(1, context.threshold - context.requestEnvelopeTokens), + }, + (prompt) => context.summarize({ ...prompt, model }), + context.forced, + Math.max(0, context.inputTokens - context.requestEnvelopeTokens), + ); + return validateHarnessModelMessages(messages); + }, + }; + return Object.assign(strategy, { [COMPACTION_DEFINITION_BRAND]: true }); +} diff --git a/packages/eve/src/harness/compaction/prompt.test.ts b/packages/eve/src/harness/compaction/strategies/fixed-tail/prompt.test.ts similarity index 99% rename from packages/eve/src/harness/compaction/prompt.test.ts rename to packages/eve/src/harness/compaction/strategies/fixed-tail/prompt.test.ts index 8b085ba8d9..641b2068a8 100644 --- a/packages/eve/src/harness/compaction/prompt.test.ts +++ b/packages/eve/src/harness/compaction/strategies/fixed-tail/prompt.test.ts @@ -7,7 +7,7 @@ import { createCompactionPrompt, sliceUtf16Safe, TRANSCRIPT_PAYLOAD_LIMIT, -} from "#harness/compaction/prompt.js"; +} from "#harness/compaction/strategies/fixed-tail/prompt.js"; describe("createCompactionPrompt", () => { it("preserves the previous checkpoint without applying transcript truncation", () => { diff --git a/packages/eve/src/harness/compaction/prompt.ts b/packages/eve/src/harness/compaction/strategies/fixed-tail/prompt.ts similarity index 100% rename from packages/eve/src/harness/compaction/prompt.ts rename to packages/eve/src/harness/compaction/strategies/fixed-tail/prompt.ts diff --git a/packages/eve/src/harness/compaction/strategy.integration.test.ts b/packages/eve/src/harness/compaction/strategy.integration.test.ts new file mode 100644 index 0000000000..de81319af4 --- /dev/null +++ b/packages/eve/src/harness/compaction/strategy.integration.test.ts @@ -0,0 +1,401 @@ +import { MockLanguageModelV3 } from "ai/test"; +import { describe, expect, it, vi } from "vitest"; + +import { ContextContainer, contextStorage } from "#context/container.js"; +import type { CompactionStrategy } from "#harness/compaction/strategy.js"; +import { eventsOf } from "#harness/publication.js"; +import { createToolLoopHarness } from "#harness/tool-loop.js"; +import type { HarnessModelMessage } from "#harness/messages.js"; +import type { HarnessSession } from "#harness/types.js"; +import { foldingHandler } from "#internal/testing/session-machine.js"; +import type { SessionEvent } from "#protocol/session-event.js"; +import { readReactionsState, writeReactionsState } from "#reactions/state.js"; + +type StreamResult = Awaited>; +type StreamPart = StreamResult["stream"] extends ReadableStream ? Part : never; + +const textStream = (text: string, inputTokens: number): StreamResult => ({ + stream: new ReadableStream({ + start(controller) { + for (const part of [ + { type: "stream-start", warnings: [] }, + { id: "t", type: "text-start" }, + { delta: text, id: "t", type: "text-delta" }, + { id: "t", type: "text-end" }, + { + finishReason: { raw: undefined, unified: "stop" }, + type: "finish", + usage: { + inputTokens: { cacheRead: 0, cacheWrite: 0, noCache: inputTokens, total: inputTokens }, + outputTokens: { reasoning: 0, text: 4, total: 4 }, + }, + }, + ] as StreamPart[]) + controller.enqueue(part); + controller.close(); + }, + }), +}); + +const isSummaryCall = (prompt: readonly { role: string; content: unknown }[]): boolean => + !String(prompt[0]?.content).includes("You help Alice."); + +function session(history: HarnessSession["history"], threshold: number): HarnessSession { + return { + agent: { modelReference: { id: "turn-model" }, system: "You help Alice.", tools: [] }, + compaction: { recentWindowSize: 1, threshold }, + continuationToken: "http:strategy", + history, + sessionId: "strategy", + }; +} + +async function run(input: { + readonly ctx?: ContextContainer; + readonly message: string; + readonly model: MockLanguageModelV3; + readonly session: HarnessSession; + readonly strategy?: CompactionStrategy; +}): Promise { + const events: SessionEvent[] = []; + const handle = foldingHandler(); + await contextStorage.run(input.ctx ?? new ContextContainer(), async () => { + let stepped = await createToolLoopHarness({ + compactionStrategy: input.strategy, + handleEvent: async (publication) => { + events.push(...eventsOf(publication)); + await handle(publication); + }, + resolveModel: async () => input.model, + tools: new Map(), + })(input.session, { message: input.message }); + while (typeof stepped.next === "function") stepped = await stepped.next(stepped.session); + }); + return events; +} + +const contextStarts = (events: readonly SessionEvent[]) => + events.filter((event) => event.type === "context.started"); + +describe("compaction triggers", () => { + it("compacts once for a request that alone exceeds the limit, then runs on", async () => { + let summaries = 0; + const model = new MockLanguageModelV3({ + doStream: async ({ prompt }) => { + if (isSummaryCall(prompt)) { + summaries++; + return textStream("Alice sent a very long document.", 120); + } + // The provider counts about 1.6x what the estimate does, close under the limit. + return textStream("Done.", 19_000); + }, + }); + + const events = await run({ + message: "Summarize this. " + "word ".repeat(80_000), + model, + session: session([], 20_000), + }); + + expect(contextStarts(events)).toHaveLength(1); + expect(summaries).toBe(1); + expect(events.map(({ type }) => type)).toContain("turn.settled"); + expect(events.some((event) => event.type === "session.ended")).toBe(false); + }); + + it("fails the step when the strategy's result is still over the limit", async () => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const doStream = vi.fn(async () => textStream("Done.", 500)); + const events = await run({ + message: "Please continue.", + model: new MockLanguageModelV3({ doStream }), + session: session( + [{ content: "Earlier notes. ".repeat(2_000), kind: "user", role: "user" }], + 2_000, + ), + strategy: { compact: async (history) => history }, + }); + + expect(doStream).not.toHaveBeenCalled(); + expect(events.at(-1)).toMatchObject({ + type: "session.ended", + data: { outcome: "failed", error: { code: "COMPACTION_FAILED" } }, + }); + }); + + it("fails when a strategy leaves a history the provider measured over the limit", async () => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const doStream = vi.fn(async () => textStream("Done.", 500)); + const history: HarnessSession["history"] = [ + { content: "Earlier notes.", kind: "user", role: "user" }, + ]; + const base = session(history, 10_000); + const events = await run({ + message: "Please continue.", + model: new MockLanguageModelV3({ doStream }), + session: { + ...base, + compaction: { + ...base.compaction, + lastKnownInputTokens: 11_000, + lastKnownPromptMessageCount: 1, + }, + }, + strategy: { compact: async (messages) => messages }, + }); + + expect(doStream).not.toHaveBeenCalled(); + expect(events.at(-1)).toMatchObject({ + type: "session.ended", + data: { outcome: "failed", error: { code: "COMPACTION_FAILED" } }, + }); + }); + + it("leaves the session's messages as they were when a strategy edits them and throws", async () => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const ctx = new ContextContainer(); + writeReactionsState(ctx, { + ...readReactionsState(ctx), + intents: [{ occurrence: "o", reactionId: "hooks/trim.ts", type: "compact" }], + }); + const history: HarnessSession["history"] = [ + { content: "Original.", kind: "user", role: "user" }, + ]; + await run({ + ctx, + message: "Thanks.", + model: new MockLanguageModelV3({ doStream: async () => textStream("Done.", 500) }), + session: session(history, 100_000), + strategy: { + compact: async (messages) => { + (messages[0] as { content: string }).content = "Edited."; + throw new Error("strategy broke"); + }, + }, + }); + + expect(history[0]).toMatchObject({ content: "Original." }); + }); + + it("ends a compaction as COMPACTION_FAILED, once, when the strategy returns an invalid history", async () => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const doStream = vi.fn(async () => textStream("Done.", 500)); + const compact = vi.fn(async () => [ + { content: "No user message first.", role: "assistant" }, + ]); + const events = await run({ + message: "Please continue.", + model: new MockLanguageModelV3({ doStream }), + session: session( + [{ content: "Earlier notes. ".repeat(2_000), kind: "user", role: "user" }], + 2_000, + ), + strategy: { compact }, + }); + + expect(compact).toHaveBeenCalledOnce(); + expect(doStream).not.toHaveBeenCalled(); + expect(events.at(-1)).toMatchObject({ + type: "session.ended", + data: { outcome: "failed", error: { code: "COMPACTION_FAILED" } }, + }); + }); + + it("fails an unchanged history even when the strategy rebuilds its messages", async () => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const doStream = vi.fn(async () => textStream("Done.", 500)); + const history: HarnessSession["history"] = [ + { content: "Earlier notes.", kind: "user", role: "user" }, + ]; + const base = session(history, 10_000); + const events = await run({ + message: "Please continue.", + model: new MockLanguageModelV3({ doStream }), + session: { + ...base, + compaction: { + ...base.compaction, + lastKnownInputTokens: 11_000, + lastKnownPromptMessageCount: 1, + }, + }, + strategy: { + compact: async (messages) => + messages.map((message) => Object.fromEntries(Object.entries(message).reverse())) as never, + }, + }); + + expect(doStream).not.toHaveBeenCalled(); + expect(events.at(-1)).toMatchObject({ data: { error: { code: "COMPACTION_FAILED" } } }); + }); + + it("counts a result that dropped messages by its estimate, whatever the provider reported", async () => { + const doStream = vi.fn(async () => textStream("Done.", 500)); + const history: HarnessSession["history"] = Array.from({ length: 6 }, (_, index) => ({ + content: `Note ${index}.`, + kind: "user" as const, + role: "user" as const, + })); + const base = session(history, 10_000); + const events = await run({ + message: "Please continue.", + model: new MockLanguageModelV3({ doStream }), + session: { + ...base, + compaction: { + ...base.compaction, + lastKnownInputTokens: 30_000, + lastKnownPromptMessageCount: 6, + }, + }, + strategy: { compact: async (messages) => messages.slice(-1) }, + }); + + expect(doStream).toHaveBeenCalled(); + expect(events.at(-1)).not.toMatchObject({ data: { error: { code: "COMPACTION_FAILED" } } }); + }); + + it("compacts when a strategy keeps every message and stubs the large one, under a large provider count", async () => { + const doStream = vi.fn(async () => textStream("Done.", 500)); + const history: HarnessSession["history"] = [ + { content: "Earlier notes.", kind: "user", role: "user" }, + { content: "x".repeat(40_000), role: "assistant" }, + { content: "More notes.", kind: "user", role: "user" }, + ]; + const base = session(history, 10_000); + const events = await run({ + message: "Please continue.", + model: new MockLanguageModelV3({ doStream }), + session: { + ...base, + compaction: { + ...base.compaction, + lastKnownInputTokens: 30_000, + lastKnownPromptMessageCount: 3, + }, + }, + strategy: { + compact: async (messages) => + messages.map((message) => + typeof message.content === "string" && message.content.length > 1_000 + ? { ...message, content: "[moved to a file]" } + : message, + ) as never, + }, + }); + + expect(doStream).toHaveBeenCalled(); + expect(events.at(-1)).not.toMatchObject({ data: { error: { code: "COMPACTION_FAILED" } } }); + }); + + it("gives a strategy a history with a URL file part, and compacts", async () => { + const compact = vi.fn(async (messages) => messages.slice(-1)); + const ctx = new ContextContainer(); + writeReactionsState(ctx, { + ...readReactionsState(ctx), + intents: [{ occurrence: "o", reactionId: "hooks/trim.ts", type: "compact" }], + }); + const events = await run({ + ctx, + message: "Thanks.", + model: new MockLanguageModelV3({ doStream: async () => textStream("Done.", 500) }), + session: session( + [ + { + content: [ + { type: "file", data: new URL("https://example.com/a.png"), mediaType: "image/png" }, + ], + kind: "user", + role: "user", + }, + ], + 100_000, + ), + strategy: { compact }, + }); + + expect(compact).toHaveBeenCalledOnce(); + expect(events.filter((event) => event.type === "context.settled")).toMatchObject([ + { data: { outcome: "completed" } }, + ]); + }); + + it.each([ + ["throws", async (): Promise => Promise.reject(new Error("strategy broke"))], + [ + "drops a tool result", + async (history: readonly HarnessModelMessage[]) => + history.filter(({ role }) => role !== "tool"), + ], + ])( + "settles a requested compaction as failed when the strategy %s, and the turn goes on", + async (_name, compact) => { + vi.spyOn(console, "error").mockImplementation(() => {}); + const ctx = new ContextContainer(); + writeReactionsState(ctx, { + ...readReactionsState(ctx), + intents: [{ occurrence: "o", reactionId: "hooks/trim.ts", type: "compact" }], + }); + const history: HarnessSession["history"] = [ + { content: "Look it up.", kind: "user", role: "user" }, + { + content: [{ input: {}, toolCallId: "c1", toolName: "lookup", type: "tool-call" }], + role: "assistant", + }, + { + content: [ + { + output: { type: "text", value: "Two cases." }, + toolCallId: "c1", + toolName: "lookup", + type: "tool-result", + }, + ], + role: "tool", + }, + ]; + const events = await run({ + ctx, + message: "Thanks.", + model: new MockLanguageModelV3({ doStream: async () => textStream("Done.", 500) }), + session: session(history, 100_000), + strategy: { compact }, + }); + + expect(events.filter((event) => event.type === "context.settled")).toMatchObject([ + { data: { outcome: "failed" } }, + ]); + expect(events.map(({ type }) => type)).toContain("turn.settled"); + expect(events.some((event) => event.type === "session.ended")).toBe(false); + }, + ); + + it("compacts before the next model call when a hook asked, and only once", async () => { + const model = new MockLanguageModelV3({ + doStream: async () => textStream("Done.", 500), + }); + const compact = vi.fn(async (history) => history.slice(-1)); + const ctx = new ContextContainer(); + writeReactionsState(ctx, { + ...readReactionsState(ctx), + intents: [{ occurrence: "o", reactionId: "hooks/trim.ts", type: "compact" }], + }); + + const history = [ + { content: "First question.", kind: "user" as const, role: "user" as const }, + { content: "First answer.", role: "assistant" as const }, + ]; + const events = await run({ + ctx, + message: "Second question.", + model, + session: session(history, 100_000), + strategy: { compact }, + }); + + expect(compact).toHaveBeenCalledOnce(); + expect(compact.mock.calls[0]?.[1]).toMatchObject({ forced: true }); + expect(contextStarts(events)).toMatchObject([{ data: { cause: { hook: "hooks/trim.ts" } } }]); + expect(readReactionsState(ctx).intents ?? []).toEqual([]); + }); +}); diff --git a/packages/eve/src/harness/compaction/strategy.ts b/packages/eve/src/harness/compaction/strategy.ts new file mode 100644 index 0000000000..278cb83853 --- /dev/null +++ b/packages/eve/src/harness/compaction/strategy.ts @@ -0,0 +1,49 @@ +import type { LanguageModel, ModelMessage } from "ai"; + +import type { HarnessModelMessage } from "#harness/messages.js"; + +/** + * A message a strategy returns. A user message needs a `kind`; build one with + * `createCompactionUserMessage`. + */ +export type CompactionMessage = HarnessModelMessage; + +export interface CompactionContext { + /** Best count of the next request's input tokens, including the request envelope. */ + readonly inputTokens: number; + /** The share of `inputTokens` that is not history: instructions, tool schemas, memory, client context. */ + readonly requestEnvelopeTokens: number; + /** The most input tokens a request may carry before the history must be compacted. */ + readonly threshold: number; + /** How many of the latest messages the session suggests keeping word for word. */ + readonly recentWindowSize: number; + /** A user or a hook asked for this compaction; the context was not over its limit. */ + readonly forced: boolean; + readonly abortSignal?: AbortSignal; + /** Estimates tokens the way the core measures the history. */ + estimateTokens(messages: readonly ModelMessage[]): number; + /** Runs the compaction model over `messages`, billed to the session. */ + summarize(input: { + readonly messages: ModelMessage[]; + readonly system: string; + /** A gateway model id or a language model for this call; the session's model when omitted. */ + readonly model?: string | LanguageModel; + }): Promise; +} + +/** + * How a session's history is shortened. The core owns when compaction can + * run, the events around it, memory records, failure and billing; a strategy + * owns whether to compact and what the compacted history looks like. + */ +export interface CompactionStrategy { + /** + * Returns the replacement for `history` (the conversation, without memory + * records, which eve places ahead of the result). Throws + * `CompactionOverLimitError` when no history it could return fits. + */ + compact( + history: readonly CompactionMessage[], + context: CompactionContext, + ): Promise; +} diff --git a/packages/eve/src/harness/model-call/call.ts b/packages/eve/src/harness/model-call/call.ts index dad3f98bbd..1ac27bae24 100644 --- a/packages/eve/src/harness/model-call/call.ts +++ b/packages/eve/src/harness/model-call/call.ts @@ -1,4 +1,4 @@ -import { withoutPromptMeasurement } from "#harness/compaction/engine.js"; +import { withoutPromptMeasurement } from "#harness/compaction/measure.js"; import type { ApprovalMemory } from "#approval/evaluation.js"; import { isStepCount, @@ -52,7 +52,12 @@ import { estimateTokens } from "#harness/token-estimate.js"; import { throwIfTurnAborted } from "#harness/turn-cancellation.js"; import { requireSessionModelReference, type StepResult } from "#harness/types.js"; import { createLogger, logError } from "#internal/logging.js"; -import { CompactionOverLimitError } from "#harness/compaction/over-limit-error.js"; +import { + CompactionInvalidResultError, + CompactionOverLimitError, +} from "#harness/compaction/over-limit-error.js"; +import type { ContextContainer } from "#context/container.js"; +import { takeCompactRequest } from "#reactions/intents.js"; import { compactedTurnInput, maybeCompact } from "#harness/compaction/step.js"; import { nextChangeId, nextRunId } from "#protocol/session-projection.js"; import { reportCompactionFailure } from "./failure.js"; @@ -223,7 +228,10 @@ export class ModelCaller { async failIfCompactionFailed(): Promise { if (this.compactionFailure === undefined) return undefined; const { error } = this.compactionFailure; - if (error instanceof CompactionOverLimitError) { + if ( + error instanceof CompactionOverLimitError || + error instanceof CompactionInvalidResultError + ) { return reportCompactionFailure(this.step, error); } throw error; @@ -299,19 +307,21 @@ export class ModelCaller { const { step, prompt } = this; const { config } = step; let compaction: Awaited>; + const requested = takeCompactRequest(step.ctx as ContextContainer | undefined); try { const { projection } = step.view(); compaction = await maybeCompact({ abortSignal: config.abortSignal, auth: step.ctx?.get(AuthKey) ?? null, change: { + cause: requested, changeId: nextChangeId(projection), summaryRunId: nextRunId(projection), turnId: step.position().turnId, }, ctx: step.ctx, emissionState: step.position(), - // A hook's compact intent compacts before this call, once. + force: requested !== undefined, messages: [...prompt.messages], model: this.input.model, promptMessages: withClientContext(prompt), @@ -327,10 +337,18 @@ export class ModelCaller { attempt: 0, turnId: activeTurnId(step.position()), }).telemetry, + strategy: config.compactionStrategy, traceContextHeaders: step.instrumentation?.gatewayTraceContextHeaders, }); step.session = compaction.session; - if (compaction.failure !== undefined) throw compaction.failure.error; + // A requested compaction that failed settled as failed and left the history as it was; the + // call goes on unless the request is over the limit. + if ( + compaction.failure !== undefined && + (compaction.overLimit === true || requested === undefined) + ) { + throw compaction.failure.error; + } } catch (error) { this.compactionFailure = { error }; throw error; diff --git a/packages/eve/src/harness/model-call/failure.ts b/packages/eve/src/harness/model-call/failure.ts index 6d56c55f67..d72cc3785e 100644 --- a/packages/eve/src/harness/model-call/failure.ts +++ b/packages/eve/src/harness/model-call/failure.ts @@ -4,7 +4,10 @@ import { extractModelCallErrorDetails, extractUpstreamRejectionMessage, } from "#harness/model-call/errors.js"; -import type { CompactionOverLimitError } from "#harness/compaction/over-limit-error.js"; +import type { + CompactionInvalidResultError, + CompactionOverLimitError, +} from "#harness/compaction/over-limit-error.js"; import { createErrorId, createLogger, formatError } from "#internal/logging.js"; import { toErrorMessage } from "#shared/errors.js"; import type { Step } from "#harness/step/context.js"; @@ -111,7 +114,7 @@ export async function reportModelCallFailure(step: Step, error: unknown): Promis */ export async function reportCompactionFailure( step: Step, - error: CompactionOverLimitError, + error: CompactionOverLimitError | CompactionInvalidResultError, ): Promise { const { sessionId } = step.session; log.error(error.message, { sessionId, turnId: step.position().turnId }); diff --git a/packages/eve/src/harness/model-call/model.ts b/packages/eve/src/harness/model-call/model.ts index 8182b2b144..917443f319 100644 --- a/packages/eve/src/harness/model-call/model.ts +++ b/packages/eve/src/harness/model-call/model.ts @@ -1,3 +1,4 @@ +import { overflowThreshold } from "#harness/compaction/measure.js"; import { gateway, type LanguageModel, type LanguageModelMiddleware, wrapLanguageModel } from "ai"; import type { CompactionConfig, HarnessSession, ToolLoopHarnessConfig } from "#harness/types.js"; @@ -113,11 +114,6 @@ function updateCompactionThresholdForModelReference(input: { return { ...input.compaction, - threshold: Math.max( - 1, - Math.floor( - input.modelReference.contextWindowTokens * (input.compaction.thresholdPercent ?? 0.9), - ), - ), + threshold: overflowThreshold(input.modelReference.contextWindowTokens), }; } diff --git a/packages/eve/src/harness/step/after-model.ts b/packages/eve/src/harness/step/after-model.ts index c20e90e4c8..aae5fd1764 100644 --- a/packages/eve/src/harness/step/after-model.ts +++ b/packages/eve/src/harness/step/after-model.ts @@ -37,7 +37,7 @@ import { isWorkflowTool, workingTaskIds } from "#execution/tasks/model-step.js"; import { normalizeProviderToolHistory } from "#harness/provider-tool-history.js"; import { renderFinalOutputWhileWorkingError } from "#execution/tasks/render.js"; import { resolveInlineAuthorizationInterrupt } from "#harness/inline-tool-authorization.js"; -import { withPromptMeasurement } from "#harness/compaction/engine.js"; +import { withPromptMeasurement } from "#harness/compaction/measure.js"; import { createLogger } from "#internal/logging.js"; const log = createLogger("harness.tool-loop"); diff --git a/packages/eve/src/harness/types.ts b/packages/eve/src/harness/types.ts index 679505deb5..5b90e2bad5 100644 --- a/packages/eve/src/harness/types.ts +++ b/packages/eve/src/harness/types.ts @@ -1,5 +1,6 @@ import type { SessionEvent } from "#protocol/session-event.js"; import type { ConsumedDelivery } from "#harness/session-machine/transitions.js"; +import type { CompactionStrategy } from "#harness/compaction/strategy.js"; import type { AuthorizationChallenge } from "#harness/authorization.js"; import type { LanguageModel } from "ai"; import type { MessageContent } from "#shared/message-content.js"; @@ -43,19 +44,12 @@ export interface CompactionState { export interface CompactionConfig extends CompactionState { readonly recentWindowSize: number; readonly threshold: number; - readonly thresholdPercent?: number; } /** * Serializable agent configuration stored on the session. */ interface SessionAgentBase { - /** - * Optional model used only for compaction summaries. - * - * When omitted, the harness uses the active turn model for compaction. - */ - readonly compactionModelReference?: RuntimeModelReference; readonly reasoning?: AgentReasoningDefinition; readonly system: string; readonly tools: readonly SessionToolDefinition[]; @@ -316,6 +310,8 @@ export interface ToolLoopHarnessConfig { readonly compactOnly?: boolean; /** The delivery a clear or compact control arrived as; its facts name it. */ readonly controlDelivery?: ControlDelivery; + /** How the history is compacted. The built-in fixed-tail strategy when omitted. */ + readonly compactionStrategy?: CompactionStrategy; readonly handleEvent?: HandleEventFn; /** * Internal lifecycle hooks injected into each actual model attempt. diff --git a/packages/eve/src/internal/authored-definition/compaction.ts b/packages/eve/src/internal/authored-definition/compaction.ts new file mode 100644 index 0000000000..f08692c7af --- /dev/null +++ b/packages/eve/src/internal/authored-definition/compaction.ts @@ -0,0 +1,11 @@ +import type { CompactionStrategy } from "#harness/compaction/strategy.js"; +import { isCompactionDefinition } from "#shared/compaction-definition.js"; + +export function normalizeCompactionDefinition(value: unknown, message: string): CompactionStrategy { + if (!isCompactionDefinition(value)) throw new Error(message); + const record = value as Record; + if (typeof record.compact !== "function") { + throw new Error(`${message} "compact" must be a function.`); + } + return value as CompactionStrategy; +} diff --git a/packages/eve/src/internal/authored-definition/core.test.ts b/packages/eve/src/internal/authored-definition/core.test.ts index eacfab8c68..5ac9fb6c2e 100644 --- a/packages/eve/src/internal/authored-definition/core.test.ts +++ b/packages/eve/src/internal/authored-definition/core.test.ts @@ -74,18 +74,31 @@ describe("normalizeAgentDefinition", () => { ).toThrow(/takes only a function; unknown key\(s\): reasoning/); }); - it("rejects a dynamic compaction model", () => { + it("names the removed compaction.model and points at fixedTail", () => { expect(() => normalizeAgentDefinition( - { - compaction: { - model: defineDynamic(() => "openai/gpt-5.5-mini"), - }, - model: "openai/gpt-5.5", - }, + { compaction: { model: "openai/gpt-5.5-mini" }, model: "openai/gpt-5.5" }, + FAILURE_MESSAGE, + ), + ).toThrow('"compaction.model" was removed: pass `model` to `fixedTail({ model })`'); + }); + + it("names the removed compaction.modelContextWindowTokens and points at the top-level setting", () => { + expect(() => + normalizeAgentDefinition( + { compaction: { modelContextWindowTokens: 8_192 }, model: "openai/gpt-5.5" }, + FAILURE_MESSAGE, + ), + ).toThrow("never changed the compaction limit"); + }); + + it("names the removed compaction.thresholdPercent", () => { + expect(() => + normalizeAgentDefinition( + { compaction: { thresholdPercent: 0.7 }, model: "openai/gpt-5.5" }, FAILURE_MESSAGE, ), - ).toThrow('"compaction.model" does not support defineDynamic'); + ).toThrow('"compaction.thresholdPercent" was removed'); }); it("accepts Anthropic prompt cache options", () => { diff --git a/packages/eve/src/internal/authored-definition/core.ts b/packages/eve/src/internal/authored-definition/core.ts index dcd05116e5..42e779fc33 100644 --- a/packages/eve/src/internal/authored-definition/core.ts +++ b/packages/eve/src/internal/authored-definition/core.ts @@ -22,7 +22,6 @@ import { ANTHROPIC_PROMPT_CACHE_TTLS, type AgentPromptCacheDefinition, type AnthropicPromptCacheTtl, - type PublicAgentStaticModelDefinition, type SessionUsageLimit, } from "#shared/agent-definition.js"; import { assertDynamicSentinelKeys, isDynamicSentinel } from "#dynamic/definition.js"; @@ -93,7 +92,7 @@ export function normalizeAgentDefinition( } if (record.compaction !== undefined) { - definition.compaction = normalizeAgentCompactionDefinition(record.compaction, message); + rejectRemovedAgentCompactionFields(record.compaction, message); } if (record.build !== undefined) { @@ -316,46 +315,24 @@ export function normalizeAgentPromptCache( return { anthropic: { ttl: ttl as AnthropicPromptCacheTtl } }; } -function normalizeAgentCompactionDefinition( - value: unknown, - message: string, -): NonNullable { +function rejectRemovedAgentCompactionFields(value: unknown, message: string): void { const record = expectObjectRecord(value, message); - expectOnlyKnownKeys(record, ["model", "modelContextWindowTokens", "thresholdPercent"], message); - const normalizedDefinition: Mutable> = {}; - + if (record.thresholdPercent !== undefined) { + throw new Error( + `${message} "compaction.thresholdPercent" was removed: eve compacts at 90% of the context window, and a hook can call ctx.compact() to compact sooner. See docs/concepts/compaction-strategies.`, + ); + } if (record.model !== undefined) { - if (isDynamicSentinel(record.model)) { - throw new Error( - `${message} "compaction.model" does not support defineDynamic — provide a static model.`, - ); - } - normalizedDefinition.model = record.model as PublicAgentStaticModelDefinition; + throw new Error( + `${message} "compaction.model" was removed: pass \`model\` to \`fixedTail({ model })\` in compaction.ts instead. See docs/concepts/compaction-strategies.`, + ); } - if (record.modelContextWindowTokens !== undefined) { - normalizedDefinition.modelContextWindowTokens = expectPositiveInteger( - record.modelContextWindowTokens, - message, + throw new Error( + `${message} "compaction.modelContextWindowTokens" was removed: it never changed the compaction limit, which comes from the model's context window. Set top-level \`modelContextWindowTokens\` in agent.ts to change that window.`, ); } - - if (record.thresholdPercent !== undefined) { - const thresholdPercent = record.thresholdPercent; - - if ( - typeof thresholdPercent !== "number" || - !Number.isFinite(thresholdPercent) || - thresholdPercent < 0 || - thresholdPercent > 1 - ) { - throw new Error(message); - } - - normalizedDefinition.thresholdPercent = thresholdPercent; - } - - return normalizedDefinition; + expectOnlyKnownKeys(record, [], message); } /** diff --git a/packages/eve/src/internal/testing/durable-session-workflow.ts b/packages/eve/src/internal/testing/durable-session-workflow.ts index 338cef4953..811fa08ba6 100644 --- a/packages/eve/src/internal/testing/durable-session-workflow.ts +++ b/packages/eve/src/internal/testing/durable-session-workflow.ts @@ -30,14 +30,12 @@ function buildSyntheticSession(input: { return { agent: { - compactionModelReference: { id: "test", contextWindowTokens: 1_000_000 }, modelReference: { id: "test", contextWindowTokens: 1_000_000 }, system: input.marker, tools: [], }, compaction: { recentWindowSize: 0, - thresholdPercent: 0.9, threshold: 0.7, }, continuationToken: `test:${input.sessionId}`, diff --git a/packages/eve/src/protocol/session-projection/fold.test.ts b/packages/eve/src/protocol/session-projection/fold.test.ts index ebe8d6dde1..1652deedac 100644 --- a/packages/eve/src/protocol/session-projection/fold.test.ts +++ b/packages/eve/src/protocol/session-projection/fold.test.ts @@ -75,6 +75,15 @@ function folded(lines: readonly StoredLine[], retention?: "operational") { } describe("the shared fold", () => { + it("keeps the last model call's input tokens after its run settles and into the next turn", () => { + const lines = [...turnLines(0, { opensSession: true }), ...turnLines(1)]; + expect(folded(lines.slice(0, 3)).lastInputTokens).toBe(spent.inputTokens); + expect(folded(lines, "operational").lastInputTokens).toBe(spent.inputTokens); + // The next turn's first model call has not reported usage yet. + const next = folded(lines.slice(0, 4 + 1)); + expect(next.lastInputTokens).toBe(spent.inputTokens); + }); + it("keeps the first introduction, valid status update, and terminal for a retained delivery", () => { const view = folded([ { diff --git a/packages/eve/src/protocol/session-projection/fold.ts b/packages/eve/src/protocol/session-projection/fold.ts index b8d16f2856..3b264dfb25 100644 --- a/packages/eve/src/protocol/session-projection/fold.ts +++ b/packages/eve/src/protocol/session-projection/fold.ts @@ -290,7 +290,10 @@ function foldFact( if (!isObject(owner)) return; if ("runId" in owner) { const run = view.runs[owner.runId]; - if (run !== undefined) view.runs[run.runId] = { ...run, usage: addUsage(run.usage, usage) }; + if (run !== undefined) { + view.runs[run.runId] = { ...run, usage: addUsage(run.usage, usage) }; + if (run.step !== undefined) view.lastInputTokens = usage.inputTokens; + } } else if ("callId" in owner) { const call = view.calls[owner.callId]; if (call !== undefined) { diff --git a/packages/eve/src/protocol/session-projection/public-view.test.ts b/packages/eve/src/protocol/session-projection/public-view.test.ts index ca639a6ba7..ab604f3516 100644 --- a/packages/eve/src/protocol/session-projection/public-view.test.ts +++ b/packages/eve/src/protocol/session-projection/public-view.test.ts @@ -128,3 +128,9 @@ it("projects interactions without exposing callback URLs through waiting or acti ]), ).not.toContain("private.example"); }); + +it("keeps a snapshot an opaque handle with no own keys", () => { + const view = events.emptySessionView(); + expect(Reflect.ownKeys(view)).toEqual([]); + expect(view.context).toEqual({ inputTokens: undefined, limit: undefined }); +}); diff --git a/packages/eve/src/protocol/session-projection/session-view.ts b/packages/eve/src/protocol/session-projection/session-view.ts index 73f93a9931..6894dd18fe 100644 --- a/packages/eve/src/protocol/session-projection/session-view.ts +++ b/packages/eve/src/protocol/session-projection/session-view.ts @@ -7,6 +7,13 @@ declare const sessionViewBrand: unique symbol; /** An immutable observation snapshot. Persist events and cursors, not its backing representation. */ export interface SessionView { readonly [sessionViewBrand]: true; + /** What the model last saw. */ + readonly context: { + /** The input tokens of the latest settled model call; absent before the first one. */ + readonly inputTokens?: number; + /** The input tokens a request may carry before the history is compacted. */ + readonly limit?: number; + }; } interface Snapshot { @@ -20,6 +27,16 @@ interface Snapshot { const KEY = Symbol.for("eve.session-view-registry.v1"); type Shared = typeof globalThis & { [KEY]?: WeakMap }; const snapshots = ((globalThis as Shared)[KEY] ??= new WeakMap()); +const contexts = new WeakMap(); +// `context` lives on the prototype so a snapshot has no own keys: it stays an opaque handle. +const snapshotPrototype = Object.freeze( + Object.defineProperty({}, "context", { + enumerable: true, + get(this: object) { + return contexts.get(this); + }, + }), +); const immutableRows = new WeakMap(); function immutable(value: T): T { @@ -56,8 +73,10 @@ export function snapshotOf( view: Tables, previews: SessionPreviews = emptyPreviews(), pending: readonly ReceivedEvent[] = [], + limit?: number, ): SessionView { - const snapshot = Object.freeze({}) as SessionView; + const snapshot = Object.freeze(Object.create(snapshotPrototype)) as SessionView; + contexts.set(snapshot, Object.freeze({ inputTokens: view.lastInputTokens, limit })); snapshots.set(snapshot, { tables: Object.freeze({ position: view.position, @@ -74,6 +93,7 @@ export function snapshotOf( turns: table(view.turns), usage: immutable(view.usage), selection: immutable(view.selection), + lastInputTokens: view.lastInputTokens, }), previews: Object.freeze({ calls: table(previews.calls), parts: table(previews.parts) }), pending: immutable(pending), diff --git a/packages/eve/src/protocol/session-projection/tables.ts b/packages/eve/src/protocol/session-projection/tables.ts index 004514f184..1b6a5c4916 100644 --- a/packages/eve/src/protocol/session-projection/tables.ts +++ b/packages/eve/src/protocol/session-projection/tables.ts @@ -222,6 +222,8 @@ export interface SessionView { readonly children: Table; readonly changes: Table; readonly usage: UsageTotals; + /** The input tokens of the latest model call a turn made. */ + readonly lastInputTokens?: number; /** * The conversation's selected turn, when a context change chose one: `null` after a clear. * Absent means the newest turn. diff --git a/packages/eve/src/public/compaction/index.ts b/packages/eve/src/public/compaction/index.ts new file mode 100644 index 0000000000..ac11dafc2f --- /dev/null +++ b/packages/eve/src/public/compaction/index.ts @@ -0,0 +1,27 @@ +import type { ExactDefinition } from "#public/definitions/exact.js"; +import type { CompactionStrategy } from "#harness/compaction/strategy.js"; +import { COMPACTION_DEFINITION_BRAND } from "#shared/compaction-definition.js"; + +export type { + CompactionMessage, + CompactionContext, + CompactionStrategy, +} from "#harness/compaction/strategy.js"; +export type { HarnessModelMessage } from "#harness/messages.js"; +export { CompactionOverLimitError } from "#harness/compaction/over-limit-error.js"; +export { createCompactionUserMessage } from "#harness/compaction/message.js"; +export { fixedTail } from "#harness/compaction/strategies/fixed-tail/index.js"; +export type { FixedTailOptions } from "#harness/compaction/strategies/fixed-tail/index.js"; + +export type DefinedCompaction = T & { + readonly [COMPACTION_DEFINITION_BRAND]: true; +}; + +/** Declares the agent's compaction strategy in `compaction.ts`. */ +export function defineCompaction( + strategy: ExactDefinition, +): DefinedCompaction; +export function defineCompaction(strategy: CompactionStrategy): DefinedCompaction { + Object.assign(strategy, { [COMPACTION_DEFINITION_BRAND]: true }); + return strategy as DefinedCompaction; +} diff --git a/packages/eve/src/public/definitions/agent.ts b/packages/eve/src/public/definitions/agent.ts index 4e7e6e9d1c..d3898da5a3 100644 --- a/packages/eve/src/public/definitions/agent.ts +++ b/packages/eve/src/public/definitions/agent.ts @@ -21,7 +21,6 @@ export type { AgentWorkflowWorldDefinition, PublicAgentModelDefinition as AgentModelDefinition, PublicAgentStaticModelDefinition as AgentStaticModelDefinition, - PublicAgentCompactionDefinition as AgentCompactionDefinition, } from "#shared/agent-definition.js"; /** diff --git a/packages/eve/src/public/definitions/hook.ts b/packages/eve/src/public/definitions/hook.ts index 8d974ab53e..e9320d6bcf 100644 --- a/packages/eve/src/public/definitions/hook.ts +++ b/packages/eve/src/public/definitions/hook.ts @@ -58,6 +58,12 @@ export interface HookContext extends SessionContext, Omit queued.type === "compact")) continue; + intents.push({ occurrence, reactionId, type: "compact" }); + changed = true; + continue; + } if (turnId === undefined || state.cancelledTurnId === turnId) continue; if (intents.some((queued) => queued.type === "cancel" && queued.turnId === turnId)) continue; intents.push({ @@ -104,3 +110,15 @@ export function takeTurnFailure(ctx: ContextContainer | undefined): string | und ); return failure?.reason; } + +/** + * Takes the compaction hooks asked for, so the next model call compacts once. Returns the hook + * that asked first, or undefined when none did. + */ +export function takeCompactRequest( + ctx: ContextContainer | undefined, +): { readonly hook: string } | undefined { + if (ctx === undefined) return undefined; + const [first] = takeIntents(ctx, (intent) => intent.type === "compact"); + return first === undefined ? undefined : { hook: first.reactionId }; +} diff --git a/packages/eve/src/reactions/kinds/hook.ts b/packages/eve/src/reactions/kinds/hook.ts index aacb3d8e76..469edc12f1 100644 --- a/packages/eve/src/reactions/kinds/hook.ts +++ b/packages/eve/src/reactions/kinds/hook.ts @@ -76,21 +76,21 @@ async function runHandler(call: { sessionId, }); const requested: Command[] = []; - const command = - (type: Command["type"]) => - (reason?: string): void => { - if (settled) { - throw new Error( - `Hook "${label}" called ctx.${type}() after its handler settled. Await the work that calls it before the handler returns.`, - ); - } - requested.push(reason === undefined ? { type } : { reason, type }); - }; + const request = (command: Command): void => { + if (settled) { + throw new Error( + `Hook "${label}" called ctx.${command.type}() after its handler settled. Await the work that calls it before the handler returns.`, + ); + } + requested.push(command); + }; const hookContext: HookContext = { ...base, ...frame, agent: agentOf(ctx), - cancel: command("cancel"), + cancel: (reason) => + request(reason === undefined ? { type: "cancel" } : { reason, type: "cancel" }), + compact: () => request({ type: "compact" }), memo: memos.memo, occurrence: { abortSignal, idempotencyKey }, position: call.position, @@ -107,9 +107,21 @@ async function runHandler(call: { ? call.input.cancelBeforeTurn : undefined; if (beforeTurn === undefined) queueCommands(ctx, id, idempotencyKey, requested); - else - for (const { reason } of requested) - beforeTurn({ hook: label, ...(reason !== undefined && { reason }) }); + else { + queueCommands( + ctx, + id, + idempotencyKey, + requested.filter((command) => command.type === "compact"), + ); + for (const command of requested) { + if (command.type !== "cancel") continue; + beforeTurn({ + hook: label, + ...(command.reason !== undefined && { reason: command.reason }), + }); + } + } } catch (error) { mergeMemos(ctx, memoScope, memos.used()); reportReactionFailure(ctx, call.reaction, error, idempotencyKey); diff --git a/packages/eve/src/reactions/kinds/memory.ts b/packages/eve/src/reactions/kinds/memory.ts index 9e5c21fdaa..aa6a39318c 100644 --- a/packages/eve/src/reactions/kinds/memory.ts +++ b/packages/eve/src/reactions/kinds/memory.ts @@ -255,10 +255,7 @@ function compactionCapture(memory: ResolvedMemoryDefinition): CommitReaction | u ...buildCallbackContext(), abortSignal: ctx.abortSignal, compaction: { - modelId: - ctx.ctx.get(BundleKey)?.turnAgent.compactionModel?.id ?? - effectiveModelId(ctx.ctx) ?? - "unknown", + modelId: effectiveModelId(ctx.ctx) ?? "unknown", usageInputTokens: started.data.trigger?.inputTokens ?? null, }, memory: { scope, slot: memory.slot }, diff --git a/packages/eve/src/reactions/runner.test.ts b/packages/eve/src/reactions/runner.test.ts index b2bed21f83..cb5471fc3e 100644 --- a/packages/eve/src/reactions/runner.test.ts +++ b/packages/eve/src/reactions/runner.test.ts @@ -2,7 +2,12 @@ import type { ModelMessage } from "ai"; import { afterEach, describe, expect, it } from "vitest"; import { ContextContainer, contextStorage } from "#context/container.js"; -import { ChannelInstrumentationKey, SessionIdKey, SessionKey } from "#context/keys.js"; +import { + ChannelInstrumentationKey, + SessionIdKey, + SessionKey, + StaticModelReferenceKey, +} from "#context/keys.js"; import { resolveApprovalPolicy, type ApprovalContext } from "#approval/definition.js"; import { evaluateApproval, type ApprovalMemory } from "#approval/evaluation.js"; import type { ApprovalDecision } from "#approval/status.js"; @@ -22,7 +27,7 @@ import { defineInstructions } from "#public/definitions/instructions.js"; import { BundleKey, ChannelKey } from "#runtime/sessions/runtime-context-keys.js"; import { defineTool } from "#tools/definition.js"; import { runApprovedCalls } from "#harness/hitl/approved-calls.js"; -import { takeTurnFailure } from "./intents.js"; +import { takeCompactRequest, takeTurnFailure } from "./intents.js"; import { takeUserInstructionMessages } from "./kinds/instructions.js"; import { dynamicTools } from "./kinds/tool.js"; import { evaluateValues, isReactionFailedError, restoreReactions, runReactions } from "./runner.js"; @@ -30,7 +35,13 @@ import { MEMOS_CAP, forgetSessionLive, memoBytes, readReactionsState, slotsOf } const user = (content: string): ModelMessage => ({ content, role: "user" }); -function written(line: number, type = "turn.started", data: object = {}, progress = false) { +function written( + line: number, + type = "turn.started", + data: object = {}, + progress = false, + view: object = initialSessionProjection().view, +) { return [ { event: { @@ -40,7 +51,7 @@ function written(line: number, type = "turn.started", data: object = {}, progres }, position: { index: 0, line }, progress, - view: initialSessionProjection().view, + view, }, ] as never; } @@ -565,6 +576,65 @@ describe("runReactions", () => { expect(readReactionsState(ctx).intents ?? []).toEqual([]); }); + it("keeps a compact request made between turns for the next model call, once", async () => { + const ctx = session(noTools, [hook("trim", { "*": (_event, ctx) => ctx.compact() })]); + await commit(ctx, { written: written(1, "session.started") }); + await commit(ctx, { written: written(2, "session.started") }); + + expect((readReactionsState(ctx).intents ?? []).map(({ type }) => type)).toEqual(["compact"]); + expect(takeCompactRequest(ctx)).toEqual({ hook: expect.any(String) }); + expect(takeCompactRequest(ctx)).toBeUndefined(); + }); + + it("lets a hook read the latest call's input tokens and ask for a compaction", async () => { + const ctx = session(noTools, [ + hook("trim", { + "*": (_event, ctx) => { + if ((ctx.view.context.inputTokens ?? 0) > 60_000) ctx.compact(); + }, + }), + ]); + const small = written(1, "session.started"); + await commit(ctx, { written: small }); + expect(takeCompactRequest(ctx)).toBeUndefined(); + + const view = { ...initialSessionProjection().view, lastInputTokens: 61_000 }; + await commit(ctx, { written: written(2, "session.started", {}, false, view) }); + expect(takeCompactRequest(ctx)).toEqual({ hook: expect.any(String) }); + }); + + it("lets a hook compare the input tokens with the compaction limit", async () => { + const seen: unknown[] = []; + const ctx = session(noTools, [ + hook("limit", { + "*": (_event, ctx) => { + seen.push(ctx.view.context.limit); + }, + }), + ]); + ctx.set(StaticModelReferenceKey, { id: "m", contextWindowTokens: 100_000 }); + await commit(ctx, { written: written(1, "session.started") }); + expect(seen).toEqual([90_000]); + }); + + it("keeps a compact request made during a turn, and a cancel does not take it", async () => { + const ctx = session(noTools, [ + hook("trim", { + "*": (_event, ctx) => { + ctx.compact(); + ctx.cancel("Stop."); + }, + }), + ]); + turnRunning(ctx); + const cancels: unknown[] = []; + await commit(ctx, { written: written(1) }); + await commit(ctx, { cancelTurn: (cancel) => cancels.push(cancel), written: written(2) }); + + expect(cancels).toHaveLength(1); + expect(takeCompactRequest(ctx)).toEqual({ hook: expect.any(String) }); + }); + it("throws for a command after the handler settled", async () => { let later: HookContext | undefined; const ctx = session(noTools, [ diff --git a/packages/eve/src/reactions/state.ts b/packages/eve/src/reactions/state.ts index e8fa663a5f..2d043f0fd1 100644 --- a/packages/eve/src/reactions/state.ts +++ b/packages/eve/src/reactions/state.ts @@ -54,7 +54,7 @@ export interface PendingIntent { /** The occurrence it acts for, for diagnostics. */ readonly occurrence: string; /** `fail` is eve's own: a hook failed during the turn, which fails at its next boundary. */ - readonly type: "cancel" | "fail"; + readonly type: "cancel" | "compact" | "fail"; readonly reason?: string; /** For a cancel or failure, the turn running when the hook ran: never a later one. */ readonly turnId?: string; diff --git a/packages/eve/src/runtime/agent/bootstrap.ts b/packages/eve/src/runtime/agent/bootstrap.ts index f879e13366..0a7a6c90f5 100644 --- a/packages/eve/src/runtime/agent/bootstrap.ts +++ b/packages/eve/src/runtime/agent/bootstrap.ts @@ -40,12 +40,6 @@ interface RuntimeTurnAgentBase { readonly id: string; readonly instructions: readonly string[]; readonly initialMessages?: readonly HarnessModelMessage[]; - /** - * Optional model used only for compaction summaries. - * - * When omitted, the harness uses the active turn model for compaction. - */ - readonly compactionModel?: RuntimeModelReference; readonly nodeId?: string; readonly reasoning?: ResolvedAgentDefinition["reasoning"]; readonly tools: readonly PreparedRuntimeTool[]; @@ -106,7 +100,6 @@ export function createResolvedRuntimeTurnAgent(input: { toolsAvailable: input.tools.length > 0, workspaceSpec: agent.workspaceSpec, }), - compactionModel: config?.compaction?.model, nodeId: input.nodeId, reasoning: config?.reasoning, tools: [...input.tools], diff --git a/packages/eve/src/runtime/agent/resolve-model.ts b/packages/eve/src/runtime/agent/resolve-model.ts index 33b8d9951e..ae841105f5 100644 --- a/packages/eve/src/runtime/agent/resolve-model.ts +++ b/packages/eve/src/runtime/agent/resolve-model.ts @@ -107,14 +107,11 @@ async function loadSourceBackedRuntimeModelReference( definition, `Expected the authored agent config export "${reference.source.exportName ?? "default"}" from "${reference.source.logicalPath}" to match the public eve shape.`, ); - const model = - reference.sourceField === "compaction.model" - ? normalizedDefinition.compaction?.model - : normalizedDefinition.model; + const model = normalizedDefinition.model; if (model === undefined) { throw new Error( - `Expected the authored agent config export "${reference.source.exportName ?? "default"}" from "${reference.source.logicalPath}" to provide a runtime model at "${reference.sourceField ?? "model"}".`, + `Expected the authored agent config export "${reference.source.exportName ?? "default"}" from "${reference.source.logicalPath}" to provide a runtime model at "model".`, ); } diff --git a/packages/eve/src/runtime/resolve-agent-graph.ts b/packages/eve/src/runtime/resolve-agent-graph.ts index 7cf6efda62..323e8a647b 100644 --- a/packages/eve/src/runtime/resolve-agent-graph.ts +++ b/packages/eve/src/runtime/resolve-agent-graph.ts @@ -96,6 +96,7 @@ export async function resolveRuntimeAgentGraph( manifest: input.manifest, nodesByNodeId, }); + attachInheritedCompaction({ manifest: input.manifest, nodesByNodeId }); return { nodesByNodeId, @@ -387,6 +388,29 @@ function resolveRemoteAgentHeaders(value: unknown): HeadersValue | undefined { return headers; } +/** A subagent without its own `compaction.ts` compacts with the nearest ancestor's strategy. */ +function attachInheritedCompaction(input: { + readonly manifest: CompiledAgentManifest; + readonly nodesByNodeId: ReadonlyMap; +}): void { + const parentNodeIdByChildNodeId = new Map( + input.manifest.subagents.map((subagent) => [ + toRuntimeNodeId(subagent.nodeId), + toRuntimeNodeId(subagent.parentNodeId), + ]), + ); + for (const [nodeId, node] of input.nodesByNodeId) { + let ancestorId = parentNodeIdByChildNodeId.get(nodeId); + while (node.agent.compaction === undefined && ancestorId !== undefined) { + const ancestor = input.nodesByNodeId.get(ancestorId); + if (ancestor?.agent.compaction !== undefined) { + (node.agent as { compaction?: unknown }).compaction = ancestor.agent.compaction; + } + ancestorId = parentNodeIdByChildNodeId.get(ancestorId); + } + } +} + function attachInheritedSandboxWorkspaceResources(input: { readonly manifest: CompiledAgentManifest; readonly nodesByNodeId: ReadonlyMap; diff --git a/packages/eve/src/runtime/resolve-agent.ts b/packages/eve/src/runtime/resolve-agent.ts index 792e8d9fc9..fdf6dcbaf8 100644 --- a/packages/eve/src/runtime/resolve-agent.ts +++ b/packages/eve/src/runtime/resolve-agent.ts @@ -13,6 +13,7 @@ import { resolveHookDefinition } from "#runtime/resolve-hook.js"; import { createResolvedModuleSourceRef } from "#runtime/resolve-helpers.js"; import { resolveSandboxDefinition } from "#runtime/resolve-sandbox.js"; import { resolveToolDefinition } from "#runtime/resolve-tool.js"; +import { resolveCompactionDefinition } from "#runtime/resolve-compaction.js"; import { resolveMemoryDefinition } from "#runtime/resolve-memory.js"; import { normalizeAgentDefinition } from "#internal/authored-definition/core.js"; import { loadResolvedModuleExport } from "#runtime/resolve-helpers.js"; @@ -101,6 +102,10 @@ export async function resolveAgent(input: ResolveAgentInput): Promise { const config: { - compaction?: NonNullable["compaction"]; defaultTools?: boolean; description?: string; experimental?: NonNullable["experimental"]; @@ -226,43 +231,6 @@ function createResolvedAgentConfig( config.description = manifest.config.description; } - if (manifest.config.compaction !== undefined) { - const compaction: { - model?: NonNullable["compaction"]>["model"]; - thresholdPercent?: number; - } = {}; - - if (manifest.config.compaction.model !== undefined) { - compaction.model = - manifest.config.compaction.model.source === undefined - ? { - contextWindowTokens: manifest.config.compaction.model.contextWindowTokens, - id: manifest.config.compaction.model.id, - maxOutputTokens: manifest.config.compaction.model.maxOutputTokens, - providerOptions: manifest.config.compaction.model.providerOptions, - } - : { - contextWindowTokens: manifest.config.compaction.model.contextWindowTokens, - id: manifest.config.compaction.model.id, - maxOutputTokens: manifest.config.compaction.model.maxOutputTokens, - providerOptions: manifest.config.compaction.model.providerOptions, - source: { - exportName: manifest.config.compaction.model.source.exportName, - sourceKind: "module" as const, - logicalPath: manifest.config.compaction.model.source.logicalPath, - sourceId: manifest.config.compaction.model.source.sourceId, - }, - sourceField: manifest.config.compaction.model.sourceField, - }; - } - - if (manifest.config.compaction.thresholdPercent !== undefined) { - compaction.thresholdPercent = manifest.config.compaction.thresholdPercent; - } - - config.compaction = compaction; - } - if (manifest.config.experimental !== undefined) { config.experimental = { workflow: diff --git a/packages/eve/src/runtime/resolve-compaction.test.ts b/packages/eve/src/runtime/resolve-compaction.test.ts new file mode 100644 index 0000000000..a1ae1dada8 --- /dev/null +++ b/packages/eve/src/runtime/resolve-compaction.test.ts @@ -0,0 +1,51 @@ +import { describe, expect, it } from "vitest"; + +import { + ROOT_COMPILED_AGENT_NODE_ID, + type CompiledCompactionDefinition, +} from "../compiler/manifest.js"; +import type { CompiledModuleMap } from "../compiler/module-map.js"; +import { defineCompaction } from "../public/compaction/index.js"; +import { resolveCompactionDefinition } from "./resolve-compaction.js"; + +const compiled: CompiledCompactionDefinition = { + exportName: undefined, + logicalPath: "compaction.ts", + sourceId: "compaction.ts", + sourceKind: "module", +}; + +function buildModuleMap(moduleNamespace: unknown): CompiledModuleMap { + return { + nodes: { [ROOT_COMPILED_AGENT_NODE_ID]: { modules: { [compiled.sourceId]: moduleNamespace } } }, + } as CompiledModuleMap; +} + +describe("resolveCompactionDefinition", () => { + it("resolves the authored strategy with its source ref", async () => { + const strategy = defineCompaction({ + async compact() { + return []; + }, + }); + + const resolved = await resolveCompactionDefinition( + compiled, + buildModuleMap({ default: strategy }), + undefined, + ); + + expect(resolved.strategy).toBe(strategy); + expect(resolved).toMatchObject({ logicalPath: "compaction.ts", sourceKind: "module" }); + }); + + it("rejects an export not created with defineCompaction()", async () => { + await expect( + resolveCompactionDefinition( + compiled, + buildModuleMap({ default: { compact() {} } }), + undefined, + ), + ).rejects.toThrow("to be created with defineCompaction()"); + }); +}); diff --git a/packages/eve/src/runtime/resolve-compaction.ts b/packages/eve/src/runtime/resolve-compaction.ts new file mode 100644 index 0000000000..9b002e73d6 --- /dev/null +++ b/packages/eve/src/runtime/resolve-compaction.ts @@ -0,0 +1,38 @@ +import type { CompiledCompactionDefinition } from "#compiler/manifest.js"; +import type { CompiledModuleMap } from "#compiler/module-map.js"; +import { normalizeCompactionDefinition } from "#internal/authored-definition/compaction.js"; +import { loadResolvedModuleExport, ResolveAgentError } from "#runtime/resolve-helpers.js"; +import type { ResolvedCompactionDefinition } from "#runtime/types.js"; +import { toErrorMessage } from "#shared/errors.js"; + +export async function resolveCompactionDefinition( + compiled: CompiledCompactionDefinition, + moduleMap: CompiledModuleMap, + nodeId: string | undefined, +): Promise { + try { + const value = await loadResolvedModuleExport({ + definition: compiled, + kindLabel: "compaction", + moduleMap, + nodeId, + }); + const strategy = normalizeCompactionDefinition( + value, + `Expected the compaction export "${compiled.exportName ?? "default"}" from "${compiled.logicalPath}" to be created with defineCompaction().`, + ); + return { + exportName: compiled.exportName, + logicalPath: compiled.logicalPath, + sourceId: compiled.sourceId, + sourceKind: "module", + strategy, + }; + } catch (error) { + if (error instanceof ResolveAgentError) throw error; + throw new ResolveAgentError( + `Failed to resolve compaction from "${compiled.logicalPath}": ${toErrorMessage(error)}`, + { logicalPath: compiled.logicalPath, sourceId: compiled.sourceId }, + ); + } +} diff --git a/packages/eve/src/runtime/subagents/dynamic-agent-config.ts b/packages/eve/src/runtime/subagents/dynamic-agent-config.ts index 0b415d3eb9..2202449dbc 100644 --- a/packages/eve/src/runtime/subagents/dynamic-agent-config.ts +++ b/packages/eve/src/runtime/subagents/dynamic-agent-config.ts @@ -12,10 +12,6 @@ import { } from "#shared/agent-definition.js"; export interface DynamicSubagentAgentConfig { - readonly compaction?: { - readonly model?: DynamicSubagentModelReference; - readonly thresholdPercent?: number; - }; readonly description: string; /** Serialized into workflow input, so it can't carry the `sessionUsage` function. */ readonly limits?: Pick; @@ -59,7 +55,6 @@ export async function normalizeDynamicSubagentAgentConfig(input: { } const config: { - compaction?: DynamicSubagentAgentConfig["compaction"]; description: string; limits?: DynamicSubagentAgentConfig["limits"]; model: DynamicSubagentModelReference; @@ -79,27 +74,6 @@ export async function normalizeDynamicSubagentAgentConfig(input: { }), }; - if (definition.compaction !== undefined) { - const compaction: { - model?: DynamicSubagentModelReference; - thresholdPercent?: number; - } = {}; - if (definition.compaction.model !== undefined) { - compaction.model = await normalizeDurableModelSelection({ - catalog: input.catalog, - selection: { - model: definition.compaction.model, - modelContextWindowTokens: definition.compaction.modelContextWindowTokens, - modelOptions: definition.modelOptions, - }, - state: input.state, - }); - } - if (definition.compaction.thresholdPercent !== undefined) { - compaction.thresholdPercent = definition.compaction.thresholdPercent; - } - config.compaction = compaction; - } if (definition.limits !== undefined) { config.limits = definition.limits; } diff --git a/packages/eve/src/runtime/types.ts b/packages/eve/src/runtime/types.ts index d9696f0efe..a593db5e35 100644 --- a/packages/eve/src/runtime/types.ts +++ b/packages/eve/src/runtime/types.ts @@ -44,6 +44,7 @@ import type { CompiledToolBehavior } from "#tools/behavior.js"; import type { SandboxEnvironmentIdentity, SandboxSelector } from "#shared/sandbox-environment.js"; import type { ToolSchema } from "#tools/schema.js"; import type { AgentSourceOwner } from "#compiler/source-graph.js"; +import type { CompactionStrategy } from "#harness/compaction/strategy.js"; import type { MemoryDefinition } from "#public/memory/index.js"; /** @@ -385,6 +386,10 @@ export interface ResolvedDynamicConnectionResolver extends ResolvedReactionSourc readonly extensionNamespace?: string; } +export type ResolvedCompactionDefinition = Readonly< + { strategy: CompactionStrategy } & ModuleSourceRef +>; + export type ResolvedMemoryDefinition = Readonly< MemoryDefinition & ModuleSourceRef & { @@ -444,6 +449,8 @@ export interface ResolvedAgent { readonly workspaceResourceRoot: CompiledWorkspaceResourceRoot; readonly hooks: readonly ResolvedHookDefinition[]; readonly memories: readonly ResolvedMemoryDefinition[]; + /** Authored compaction strategy from `compaction.ts`, or `undefined` for the default. */ + readonly compaction?: ResolvedCompactionDefinition | undefined; readonly skills: readonly ResolvedSkillDefinition[]; readonly tools: readonly ResolvedToolDefinition[]; readonly workspaceSpec: WorkspaceRuntimeSpec; diff --git a/packages/eve/src/shared/agent-definition.ts b/packages/eve/src/shared/agent-definition.ts index 2ed3283c29..a5159c70d4 100644 --- a/packages/eve/src/shared/agent-definition.ts +++ b/packages/eve/src/shared/agent-definition.ts @@ -84,8 +84,6 @@ export type InternalAgentModelDefinition = { contextWindowTokens?: number; maxOutputTokens?: number; source?: ModuleSourceRef; - /** Field of the `source` agent config export that holds this model; absent means `model`. */ - sourceField?: "compaction.model"; providerOptions?: Record; promptCache?: AgentPromptCacheDefinition; }; @@ -133,50 +131,6 @@ export type PublicAgentModelDefinition = | PublicAgentStaticModelDefinition | PublicAgentDynamicModelDefinition; -export interface InternalAgentCompactionDefinition { - /** - * Optional model used only for generating compaction summaries. - * - * When omitted, eve uses the active turn model for the summary call. - */ - model?: InternalAgentModelDefinition; - /** - * Fraction of the primary model context window that triggers compaction. - * - * eve defaults to `0.9` when this is omitted. - */ - thresholdPercent?: number; -} - -/** - * Configures conversation compaction: when the model context window fills past - * `thresholdPercent`, eve summarizes earlier turns to reclaim space. Every - * field is optional; omit the block to use eve's defaults. - */ -export interface PublicAgentCompactionDefinition { - /** - * Optional override for the compaction summary model's context window size, - * in tokens. - * - * Same escape hatch as the agent-level `modelContextWindowTokens`. When set, - * eve uses this value verbatim and skips the AI Gateway lookup for the - * compaction summary model. - */ - readonly modelContextWindowTokens?: number; - /** - * Optional model used only for generating compaction summaries. - * - * When omitted, eve uses the active turn model for the summary call. - */ - readonly model?: PublicAgentStaticModelDefinition; - /** - * Fraction of the primary model context window that triggers compaction. - * - * eve defaults to `0.9` when this is omitted. - */ - readonly thresholdPercent?: number; -} - /** * Configures framework-owned runtime limits for this agent's runs. */ @@ -339,7 +293,6 @@ export type InternalAgentDefinition = { name: string; description?: string; build?: AgentBuildDefinition; - compaction?: InternalAgentCompactionDefinition; defaultTools?: boolean; experimental?: AgentExperimentalDefinition; model: InternalAgentModelDefinition; @@ -373,7 +326,6 @@ type PublicAgentDefinitionBase = { */ readonly description?: string; readonly build?: AgentBuildDefinition; - readonly compaction?: PublicAgentCompactionDefinition; /** * Whether eve automatically adds its optional default tools. Defaults to `true`. * Required connection tooling and tools authored under `agent/tools/` remain diff --git a/packages/eve/src/shared/compaction-definition.ts b/packages/eve/src/shared/compaction-definition.ts new file mode 100644 index 0000000000..794d8f3f3c --- /dev/null +++ b/packages/eve/src/shared/compaction-definition.ts @@ -0,0 +1,9 @@ +export const COMPACTION_DEFINITION_BRAND = Symbol.for("eve:compaction-definition"); + +export function isCompactionDefinition(value: unknown): boolean { + return ( + typeof value === "object" && + value !== null && + Reflect.get(value, COMPACTION_DEFINITION_BRAND) === true + ); +} diff --git a/packages/eve/test/discover-manifest.test.ts b/packages/eve/test/discover-manifest.test.ts index 0544e5f5e2..edab0f28eb 100644 --- a/packages/eve/test/discover-manifest.test.ts +++ b/packages/eve/test/discover-manifest.test.ts @@ -27,6 +27,7 @@ describe("agent source manifest", () => { agentRoot, appRoot, channels: [], + compaction: null, connections: [], diagnosticsSummary: { errors: 0, diff --git a/packages/eve/test/scenarios/compile-agent.scenario.test.ts b/packages/eve/test/scenarios/compile-agent.scenario.test.ts index f6f35c521c..b2ed3c81fb 100644 --- a/packages/eve/test/scenarios/compile-agent.scenario.test.ts +++ b/packages/eve/test/scenarios/compile-agent.scenario.test.ts @@ -134,7 +134,6 @@ describe("compiler artifacts", () => { const withoutConfig = await compileAgent({ startPath: appRoot }); expect(withoutConfig.manifest.config).toMatchObject({ - compaction: {}, model: { contextWindowTokens: expect.any(Number), id: DEFAULT_AGENT_MODEL_ID, @@ -318,7 +317,6 @@ describe("compiler artifacts", () => { agentRoot: "/agent", appRoot: "", config: { - compaction: {}, model: { contextWindowTokens: expect.any(Number), id: "openai/gpt-5.4", diff --git a/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts b/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts index c3e034b1e5..c7349992ff 100644 --- a/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts +++ b/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts @@ -79,7 +79,6 @@ describe("compiler model catalog", () => { ); expect(result.manifest.config).toMatchObject({ - compaction: {}, model: { contextWindowTokens: 123_456, id: "example/default-compaction-model", @@ -106,7 +105,7 @@ describe("compiler model catalog", () => { expect(globalThis.fetch).toHaveBeenCalledTimes(1); }); - it("hydrates compiled compaction limits from AI Gateway and persists the cache", async () => { + it("hydrates compiled model limits from AI Gateway and persists the cache", async () => { const { agentRoot, appRoot } = await createAppRoot( "eve-model-catalog-fetch-", APP_ROOT_OPTIONS, @@ -114,15 +113,7 @@ describe("compiler model catalog", () => { await writeFile( join(agentRoot, "agent.ts"), - [ - "export default {", - ' model: "example/primary-model",', - " compaction: {", - ' model: "example/summary-model",', - " },", - "};", - "", - ].join("\n"), + ["export default {", ' model: "example/primary-model",', "};", ""].join("\n"), ); vi.spyOn(globalThis, "fetch").mockImplementation(async () => @@ -163,12 +154,6 @@ describe("compiler model catalog", () => { contextWindowTokens: 123_456, id: "example/primary-model", }); - expect(result.manifest.config.compaction).toMatchObject({ - model: { - contextWindowTokens: 65_536, - id: "example/summary-model", - }, - }); expect(cache).toMatchObject({ kind: "eve-model-catalog-cache", version: 2, @@ -243,7 +228,7 @@ describe("compiler model catalog", () => { expect(globalThis.fetch).toHaveBeenCalledTimes(2); }); - it("hydrates compaction limits when model IDs end in -thinking", async () => { + it("hydrates model limits when model IDs end in -thinking", async () => { const { agentRoot, appRoot } = await createAppRoot( "eve-model-catalog-thinking-", APP_ROOT_OPTIONS, @@ -251,15 +236,7 @@ describe("compiler model catalog", () => { await writeFile( join(agentRoot, "agent.ts"), - [ - "export default {", - ' model: "example/primary-model-thinking",', - " compaction: {", - ' model: "example/summary-model-thinking",', - " },", - "};", - "", - ].join("\n"), + ["export default {", ' model: "example/primary-model-thinking",', "};", ""].join("\n"), ); vi.spyOn(globalThis, "fetch").mockImplementation(async () => @@ -297,12 +274,6 @@ describe("compiler model catalog", () => { contextWindowTokens: 123_456, id: "example/primary-model-thinking", }); - expect(result.manifest.config.compaction).toMatchObject({ - model: { - contextWindowTokens: 65_536, - id: "example/summary-model-thinking", - }, - }); expect(globalThis.fetch).toHaveBeenCalledTimes(1); }); @@ -314,15 +285,7 @@ describe("compiler model catalog", () => { await writeFile( join(agentRoot, "agent.ts"), - [ - "export default {", - ' model: "example/stale-primary-model",', - " compaction: {", - ' model: "example/stale-summary-model",', - " },", - "};", - "", - ].join("\n"), + ["export default {", ' model: "example/stale-primary-model",', "};", ""].join("\n"), ); await mkdir(join(appRoot, ".eve", "cache"), { recursive: true, @@ -376,12 +339,6 @@ describe("compiler model catalog", () => { contextWindowTokens: 111_111, id: "example/stale-primary-model", }); - expect(result.manifest.config.compaction).toMatchObject({ - model: { - contextWindowTokens: 22_222, - id: "example/stale-summary-model", - }, - }); expect(globalThis.fetch).toHaveBeenCalledTimes(1); }); @@ -582,48 +539,6 @@ describe("compiler model catalog", () => { expect(fetchSpy).not.toHaveBeenCalled(); }); - it("uses authored compaction modelContextWindowTokens and skips the AI Gateway lookup", async () => { - const { agentRoot, appRoot } = await createAppRoot( - "eve-model-catalog-authored-compaction-window-", - APP_ROOT_OPTIONS, - ); - - await writeFile( - join(agentRoot, "agent.ts"), - [ - "export default {", - ' model: "example/unlisted-primary-model",', - " modelContextWindowTokens: 128_000,", - " compaction: {", - ' model: "example/unlisted-summary-model",', - " modelContextWindowTokens: 64_000,", - " },", - "};", - "", - ].join("\n"), - ); - - const fetchSpy = vi - .spyOn(globalThis, "fetch") - .mockRejectedValue(new Error("fetch should not be called")); - - const result = await compileAgent({ - startPath: appRoot, - }); - - expect(result.manifest.config.model).toMatchObject({ - contextWindowTokens: 128_000, - id: "example/unlisted-primary-model", - }); - expect(result.manifest.config.compaction).toMatchObject({ - model: { - contextWindowTokens: 64_000, - id: "example/unlisted-summary-model", - }, - }); - expect(fetchSpy).not.toHaveBeenCalled(); - }); - it("compiles the one-line mockModel fixture without an authored context window", async () => { const { agentRoot, appRoot } = await createAppRoot( "eve-model-catalog-mock-model-", diff --git a/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts b/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts index 273a0303c0..fff5b3f51f 100644 --- a/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts +++ b/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts @@ -283,53 +283,6 @@ async function writeRuntimeLoaderTsconfigAliasFixture(input: { } describe("runtime compiled artifact loaders", () => { - it("loads resolved compaction config from compiled authored artifacts", async () => { - const { agentRoot, appRoot } = await createAppRoot( - "eve-runtime-loaders-compaction-", - APP_ROOT_OPTIONS, - ); - - await writeFile( - join(agentRoot, "agent.mjs"), - [ - "export default {", - ' model: "openai/gpt-5.4",', - " compaction: {", - ' model: "openai/gpt-5.4-mini",', - " thresholdPercent: 0.75,", - " },", - "};", - "", - ].join("\n"), - ); - await writeFile( - join(agentRoot, "instructions.md"), - "You are a precise runtime loader test agent.\n", - ); - - await compileAgent({ - startPath: appRoot, - }); - - const resolved = await loadResolvedCompiledAgent({ - compiledArtifactsSource: createDiskRuntimeCompiledArtifactsSource(appRoot), - }); - - expect(resolved.config).toMatchObject({ - compaction: { - model: { - contextWindowTokens: expect.any(Number), - id: "openai/gpt-5.4-mini", - }, - thresholdPercent: 0.75, - }, - model: { - contextWindowTokens: expect.any(Number), - id: "openai/gpt-5.4", - }, - }); - }); - it("loads the compiled manifest and module map from disk-backed compiler artifacts", async () => { const { agentRoot, appRoot } = await createAppRoot("eve-runtime-loaders-", APP_ROOT_OPTIONS); await writeRuntimeLoaderFixture(agentRoot); @@ -356,7 +309,6 @@ describe("runtime compiled artifact loaders", () => { const resolvedChannel = resolvedAgent.channels.find((channel) => channel.name === "slack"); expect(manifest.config).toEqual({ - compaction: {}, model: { contextWindowTokens: expect.any(Number), id: "openai/gpt-5.4", diff --git a/packages/eve/test/scenarios/subagent-compaction.scenario.test.ts b/packages/eve/test/scenarios/subagent-compaction.scenario.test.ts new file mode 100644 index 0000000000..761e354a3f --- /dev/null +++ b/packages/eve/test/scenarios/subagent-compaction.scenario.test.ts @@ -0,0 +1,56 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { describe, expect, it } from "vitest"; + +import { compileAgent } from "#compiler/compile-agent.js"; +import { createDiskRuntimeCompiledArtifactsSource } from "#runtime/compiled-artifacts-source.js"; +import { loadCompiledManifest } from "#runtime/loaders/manifest.js"; +import { loadCompiledModuleMap } from "#runtime/loaders/module-map.js"; +import { resolveRuntimeAgentGraph } from "#runtime/resolve-agent-graph.js"; +import { useTemporaryAppRoots } from "#internal/testing/use-temporary-app-roots.js"; + +const createAppRoot = useTemporaryAppRoots(); + +const strategy = (label: string) => `export default { + [Symbol.for("eve:compaction-definition")]: true, + label: ${JSON.stringify(label)}, + async compact(history) { + return history; + }, +}; +`; + +const subagent = `export default { model: "openai/gpt-5.4", description: "Investigate one task." };\n`; + +describe("subagent compaction", () => { + it("uses the root strategy unless the subagent authors its own", async () => { + const { agentRoot, appRoot } = await createAppRoot("eve-compaction-subagent-", { + packageName: "compaction-subagent-agent", + }); + await mkdir(join(agentRoot, "subagents", "inherits"), { recursive: true }); + await mkdir(join(agentRoot, "subagents", "owns"), { recursive: true }); + await writeFile(join(agentRoot, "agent.mjs"), `export default { model: "openai/gpt-5.4" };\n`); + await writeFile(join(agentRoot, "instructions.md"), "You are a precise agent.\n"); + await writeFile(join(agentRoot, "compaction.mjs"), strategy("root")); + for (const name of ["inherits", "owns"]) { + await writeFile(join(agentRoot, "subagents", name, "agent.mjs"), subagent); + await writeFile(join(agentRoot, "subagents", name, "instructions.md"), "Investigate.\n"); + } + await writeFile(join(agentRoot, "subagents", "owns", "compaction.mjs"), strategy("own")); + + await compileAgent({ startPath: appRoot }); + const compiledArtifactsSource = createDiskRuntimeCompiledArtifactsSource(appRoot); + const [manifest, moduleMap] = await Promise.all([ + loadCompiledManifest({ compiledArtifactsSource }), + loadCompiledModuleMap({ compiledArtifactsSource }), + ]); + const graph = await resolveRuntimeAgentGraph({ manifest, moduleMap }); + + const labelOf = (nodeId: string) => + (graph.nodesByNodeId.get(nodeId)?.agent.compaction?.strategy as { label?: string })?.label; + expect(labelOf("__root__")).toBe("root"); + expect(labelOf("subagents/inherits")).toBe("root"); + expect(labelOf("subagents/owns")).toBe("own"); + }); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 978602bb68..3d64e4ebd9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1000,6 +1000,25 @@ importers: specifier: 'catalog:' version: 7.0.2 + e2e/fixtures/agent-compaction-strategy: + dependencies: + '@eve-e2e/config': + specifier: workspace:* + version: link:../e2e-config + '@workflow/world-postgres': + specifier: 'catalog:' + version: 5.0.5(@opentelemetry/api@1.9.1)(@types/pg@8.23.1)(@upstash/redis@1.38.4)(sql.js@1.14.2)(supports-color@10.2.2)(typescript@7.0.2) + eve: + specifier: workspace:* + version: link:../../../packages/eve + devDependencies: + '@types/node': + specifier: 'catalog:' + version: 26.6.5 + typescript: + specifier: 'catalog:' + version: 7.0.2 + e2e/fixtures/agent-decide: dependencies: '@eve-e2e/config': diff --git a/scripts/guard-invariants.mjs b/scripts/guard-invariants.mjs index 2e0bb19c2f..1a16c22644 100644 --- a/scripts/guard-invariants.mjs +++ b/scripts/guard-invariants.mjs @@ -159,6 +159,10 @@ * `eve/events`; the client) never import a schema module by * value: schemas serve tests and development only. * + * rule 56 — Compaction strategies are closed behind their entry. Outside + * `harness/compaction/strategies/`, a strategy is reached only + * through its `index.js`; its engine, prompt and other internals + * stay private, so a strategy can change without core edits. * rule 55 — Public named exports match checker-generated per-entry lists. * * Baselines for rules with pre-existing violations live in @@ -167,7 +171,7 @@ */ import { glob, readFile, readdir, lstat } from "node:fs/promises"; import { createRequire } from "node:module"; -import { dirname, join, relative, resolve, sep } from "node:path"; +import { dirname, join, posix as pathPosix, relative, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; import { load as loadYaml } from "js-yaml"; import { checkPublicExports } from "./public-exports.mjs"; @@ -282,6 +286,7 @@ function isTsLike(relPath) { * rule51: Violation[]; * rule52: Violation[]; * rule53: Violation[]; + * rule56: Violation[]; * symlinks: string[]; * }} state */ @@ -364,6 +369,7 @@ async function scanRepo(state) { }); } checkRule53(posix, lines, state.rule53); + checkRule56(posix, lines, state.rule56); } } @@ -429,6 +435,41 @@ function checkLegacySessionImports(posix, lines, violations) { }); } +// ---------- Rule 56: compaction strategies are reached through their index ---------- + +const COMPACTION_STRATEGIES_DIR = "packages/eve/src/harness/compaction/strategies/"; +const COMPACTION_STRATEGY_SPECIFIER_RE = /(?:from\s+|import\s*\()["']([^"']+)["']/; + +/** The import's target inside the strategies directory, from an alias or relative specifier. */ +function strategyInternalTarget(posix, line) { + const specifier = COMPACTION_STRATEGY_SPECIFIER_RE.exec(line)?.[1]; + if (specifier === undefined) return undefined; + const resolved = specifier.startsWith("#harness/") + ? `packages/eve/src/harness/${specifier.slice("#harness/".length)}` + : specifier.startsWith(".") + ? pathPosix.normalize(pathPosix.join(pathPosix.dirname(posix), specifier)) + : undefined; + if (resolved === undefined || !resolved.startsWith(COMPACTION_STRATEGIES_DIR)) return undefined; + const inside = resolved.slice(COMPACTION_STRATEGIES_DIR.length).split("/"); + return inside.length >= 2 && inside.at(-1) !== "index.js" ? resolved : undefined; +} + +/** @param {string} posix @param {string[]} lines @param {Violation[]} violations */ +function checkRule56(posix, lines, violations) { + if (posix.startsWith(COMPACTION_STRATEGIES_DIR)) return; + if (/\.(?:test|integration\.test|scenario\.test)\.ts$/.test(posix)) return; + lines.forEach((line, index) => { + if (strategyInternalTarget(posix, line) === undefined) return; + violations.push({ + rule: 56, + file: posix, + line: index + 1, + message: + "Compaction strategies are reached only through their index.js; their engine, prompt and helpers are private so a strategy can change without core edits.", + }); + }); +} + // ---------- Rule 44: sandbox providers use the public provider contract ---------- const SANDBOX_PROVIDER_DIRS = [ @@ -1830,6 +1871,7 @@ async function main() { rule51: /** @type {Violation[]} */ ([]), rule52: /** @type {Violation[]} */ ([]), rule53: /** @type {Violation[]} */ ([]), + rule56: /** @type {Violation[]} */ ([]), symlinks: /** @type {string[]} */ ([]), }; @@ -1940,6 +1982,7 @@ async function main() { violations.push(...state.rule51); violations.push(...state.rule52); violations.push(...state.rule53); + violations.push(...state.rule56); for (const issue of await checkPublicExports()) { violations.push({ rule: 55, ...issue });