Skip to content

Compaction: strategy interface and compaction.ts - #5332

Closed
ruiconti wants to merge 2 commits into
mainfrom
infra/compaction-strategy
Closed

ruiconti wants to merge 2 commits into
mainfrom
infra/compaction-strategy

Conversation

@ruiconti

Copy link
Copy Markdown
Contributor

Makes compaction a strategy an agent can replace. Stacked on #5290; retarget to main after it merges.

  • CompactionStrategy (shouldCompact(facts), compact(history, ctx) returning {checkpoint, kept}) in harness/compaction/strategy.ts. The core keeps measuring, the requested/completed events, memory capture and recall, failure and billing.
  • The existing fixed-tail compaction (engine and prompt) moves, unchanged, to compaction/strategies/fixed-tail/ and is the default; also exported as fixedTail().
  • agent/compaction.ts with defineCompaction from eve/compaction, discovered, compiled and resolved like memory.ts.
  • Guard rule 54: only a strategy's index.js may be imported from outside compaction/strategies/.
  • Memory records keep their place ahead of the checkpoint, as before.

Checks: 883 harness tests, discover/compiler/runtime unchanged apart from 8 failures that also fail on #5290's branch; the three compaction evals pass unchanged; new e2e fixture agent-compaction-strategy (own strategy, no summary, memory recalled once); real 408 KB run: 183,525 then 56.4k flat vs 56.1k on #5290 (+0.6%).

@vercel

vercel Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Ready Ready Preview, v0 Oct 26, 2026 2:59pm UTC
eve-pkg Ready Ready Preview, v0 Oct 26, 2026 2:59pm UTC

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs infra/compaction-progress (6cd2fd8).

Delta vs infra/compaction-progress (6cd2fd8)

Area Metric Baseline Current Delta
Package Packed tarball 7.78 MB 7.79 MB +4.7 kB ⚠️
Package Unpacked publish size 27.33 MB 27.35 MB +17.1 kB ⚠️
Package Installed footprint 75.65 MB 75.66 MB +17.1 kB ⚠️
Package Published files 4228 4249 +21
Package Installed files 8221 8242 +21
Package Installed package instances 34 34 0
Package Distinct installed package names 32 32 0
Package Installed dependency edges 51 51 0
Package Installed optional peer edges 9 9 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 20.92 MB 20.93 MB +3.8 kB ⚠️
Runtime Public routes 18 18 0
Changed function payloads vs infra/compaction-progress (6cd2fd8) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 10.46 MB 10.46 MB +1.9 kB ⚠️ none
functions/.well-known/workflow/v1/flow.func changed 10.46 MB 10.46 MB +1.9 kB ⚠️ none

eve init install

Metric Baseline Current Delta
Installed footprint 114.03 MB 114.05 MB +17.1 kB ⚠️
Installed packages 97 97 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 45.73 MB 45.75 MB +17.1 kB ⚠️
devDependency package bytes 5.11 MB 5.11 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260903-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-10-26T14:57:34.427Z
  • Route aliases: 18 public, 1 internal (19 total aliases)
  • Vercel routes in config: 21
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.83.2
  • Package directory: packages/eve
  • Tarball: 7.79 MB (eve-0.83.2.tgz)
  • Unpacked payload: 27.35 MB across 4249 published files
  • Installed footprint: 75.66 MB across 8242 installed files
  • Installed root package: 27.35 MB
  • Installed dependencies: 48.32 MB
  • Installed package instances: 34
  • Distinct installed package names: 32
  • Installed dependency edges: 51
  • Installed optional peer edges: 9
  • Runtime dependencies: 2
  • Peer dependencies: 7 (6 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

  • eve: 27.35 MB (36.1%)
  • @rolldown/binding-linux-x64-gnu: 19.89 MB (26.3%)
  • ai: 8.37 MB (11.1%)
  • zod: 6.15 MB (8.1%)
  • undici: 3.58 MB (4.7%)
Publish payload breakdown
Published file size
🟠 dist/src/compiled/shadcn-registry/index.js       [#####...................] 3.85 MB 14.1%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.29 MB 8.4%
🟠 dist/src/compiled/run/index.js                   [#.......................] 1.00 MB 3.7%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 604.9 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-D-I... [#.......................] 521.9 kB 1.9%
🔴 Other published files                            [########################] 19.08 MB 69.8%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 27.35 MB 36.1%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.89 MB 26.3%
🔴 ai                              [#######.................] 8.37 MB 11.1%
🔴 zod                             [#####...................] 6.15 MB 8.1%
🟠 undici                          [###.....................] 3.58 MB 4.7%
🟠 nitro                           [##......................] 1.89 MB 2.5%
🔴 Other installed packages        [#######.................] 8.43 MB 11.1%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260903-beta
undici 8.10.2
Peer dependencies (7)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
chat ^4.42.0 optional peer
dd-trace ^6.13.0 optional peer
just-bash ^3.1.0 optional peer
next ^16.0.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 114.05 MB across 10159 installed files
  • Installed packages: 97 total (91 transitive-only)
  • dependencies: 4 direct packages totaling 45.75 MB
  • devDependencies: 2 direct packages totaling 5.11 MB
  • Other transitive package files: 63.18 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • @typescript/typescript-linux-x64: 27.95 MB (24.5%)
  • eve: 27.35 MB (24.0%)
  • @rolldown/binding-linux-x64-gnu: 19.89 MB (17.4%)
  • zod: 9.76 MB (8.6%)
  • ai: 8.37 MB (7.3%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 24.5%
🔴 eve                              [#######################.] 27.35 MB 24.0%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.89 MB 17.4%
🔴 zod                              [########................] 9.76 MB 8.6%
🔴 ai                               [#######.................] 8.37 MB 7.3%
🟠 undici                           [###.....................] 3.58 MB 3.1%
🔴 Other installed packages         [###############.........] 17.16 MB 15.0%
dependencies (4)
Package Range Installed size Share
@vercel/connect 2.5.2 276.3 kB 0.2%
ai ^7.0.137 8.37 MB 7.3%
eve file:eve-0.83.2.tgz 27.35 MB 24.0%
zod 4.5.4 9.76 MB 8.6%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.61 MB 2.3%
typescript 7.0.2 2.50 MB 2.2%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 10.46 MB 50.0%
🔴 functions/__server.func                         [########################] 10.46 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 10.46 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 10.46 MB
Function files 10.46 MB across 121 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 2.49 MB (23.8%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [#############...........] 2.49 MB 23.8%
🟡 _libs/undici.mjs                               [#####...................] 994.5 kB 9.5%
🟡 _chunks/sandbox2.mjs                           [####....................] 815.0 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 732.3 kB 7.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.5 kB 6.7%
🔴 Other bundled files                            [########################] 4.72 MB 45.1%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 17 public routes, 1 internal alias • 10.46 MB
Metric Value
Public routes /
/.well-known/workflow/v1/webhook/[token]
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[attemptId]/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
/eve/v1/session/[sessionId]/stubs
/eve/v1/task-input/[token]
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 10.46 MB
Function files 10.46 MB across 121 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 2.49 MB (23.8%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [#############...........] 2.49 MB 23.8%
🟡 _libs/undici.mjs                               [#####...................] 994.5 kB 9.5%
🟡 _chunks/sandbox2.mjs                           [####....................] 815.0 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 732.3 kB 7.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.5 kB 6.7%
🔴 Other bundled files                            [########################] 4.72 MB 45.1%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 2.50 s -> 2.65 s (+146.0 ms) vs infra/compaction-progress (6cd2fd8).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `infra/compaction-progress (6cd2fd8)`
Phase Baseline Current Delta
extension.check 8.0 ms 0.3 ms -7.7 ms
project.resolve 0.3 ms 0.2 ms -0.1 ms
workspace.create 0.5 ms 0.4 ms -0.1 ms
host.prepare 353.3 ms 389.7 ms +36.4 ms
vercel.service-prefix.resolve 1.0 ms 1.0 ms 0.0 ms
nitro.create 422.6 ms 440.3 ms +17.7 ms
sandbox.prewarm 349.6 ms 414.6 ms +65.0 ms
nitro.cache.prepare 0.2 ms 0.2 ms 0.0 ms
nitro.prepare 0.5 ms 0.5 ms 0.0 ms
nitro.public-assets 0.6 ms 0.6 ms 0.0 ms
nitro.prerender 0.4 ms 0.3 ms -0.1 ms
nitro.bundle 1.22 s 1.22 s +1.9 ms
nitro.cache.write 0.2 ms 0.2 ms 0.0 ms
vercel.workflow-function.materialize 82.5 ms 119.4 ms +36.9 ms
agent-summary.emit 0.4 ms 0.5 ms +0.1 ms
connect-manifest.emit 0.2 ms 0.2 ms 0.0 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 2.4 ms 2.3 ms -0.1 ms
workspace.remove 1.5 ms 1.6 ms +0.1 ms

@ruiconti

Copy link
Copy Markdown
Contributor Author

The split is right: the core keeps measurement, events, memory, failure and billing, the strategy decides whether to compact and how, and compaction.ts follows the memory.ts discovery path. Five things to fix before this is the public contract:

  1. { checkpoint, kept } has no consumer. step.ts:285-289 concatenates the two parts, and fixedTail re-parses its own checkpoint marker (strategies/fixed-tail/index.ts:35-44) only to produce the split. compact should return the replacement messages; drop CompactedHistory and splitCheckpoint.
  2. A third-party strategy can't write a user-role message. { role: "user", content: "..." } is valid under the published ModelMessage type, but step.ts:285 rejects it ("Expected every user-role model message to have a kind."). The built-in strategy gets around this with the internal createFrameworkUserMessage. Export a constructor for the compaction message from eve/compaction and type the return so a kind-less user message fails to compile. Add a test with a custom strategy that writes a user checkpoint.
  3. A remote subagent's compaction.ts is silently ignored. assertRemoteAgentDefinitionHasNoLocalPackageEntries (normalize-subagent.ts:88) doesn't list it, and remote nodes skip resource compilation. Add it to the list, with a rejection test.
  4. The summary model is resolved before the strategy runs (step.ts:202-207), so a strategy that never summarizes still fails when a separate compaction model can't resolve. Resolve it inside summarize.
  5. Leftovers. The engine's optional historyInputTokenCount and its getInputTokenCount fallback (fixed-tail/engine.ts:152,189) are only reached by tests now; make the parameter required. input.abortSignal ?? new AbortController().signal (step.ts:267) makes up a signal that never fires; make abortSignal optional on the context instead.

Checked: 890 unit tests, the discover and step integration tests, guard:invariants and tsc all pass. The engine moved with only its trigger signature changed.

@ruiconti

Copy link
Copy Markdown
Contributor Author

All five points are addressed in a3ea1d3:

  • compact returns the messages;
  • createCompactionUserMessage is exported, and a kind-less user message fails to compile (checked with a scratch strategy);
  • a remote subagent's compaction.ts is rejected;
  • the summary model resolves on the first summarize;
  • the leftovers are gone.

The compaction events now carry the configured compaction model id. For AI SDK model objects that's the same formatLanguageModelGatewayId value as before.

Unit tests across harness, compiler, discover and runtime pass, except the network-bound model-catalog test, which fails on the base branch too. The integration tests and the invariant guard pass. Looks good.

Comment on lines 285 to 289
return { compacted: false, failure, messages: input.messages, session };
}
messages = validateHarnessModelMessages([...canonical.memory, ...compactedOrdinary]);
messages = validateHarnessModelMessages([...canonical.memory, ...compacted]);

{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A custom strategy's output reaches this validation unchecked, and the validation runs outside the try, so a bad result isn't handled as a compaction failure.

  • CompactedHistory is typed as plain ModelMessage[], so checkpoint: [{ role: "user", content: "Earlier context: ..." }] type-checks. Here it throws TypeError: Expected every user-role model message to have a kind. (harness/messages.ts:113-114). eve/compaction exports no way to build a kinded user message, so an author can't write a user-role checkpoint at all, even though strategy.ts:33 suggests "a marker".
  • The throw happens after summaryUsage has been folded into the local session. On the manual path (compactHistory, ~L91-110), step.session = compacted.session never runs, so a summary call that already succeeded is dropped from billing. That contradicts the docs: "Any thrown error fails the compaction the way a failed summary does." The same goes for a throwing shouldCompact at L195.
  • Nothing checks structure either. An orphan tool result, a tool call with no result, or a history ending on an assistant turn passes this check, gets emitted as compaction.completed, and is saved. fixed-tail guards against these internally (withResumptionGuard, splitMessagesForCompaction), but custom strategies get no such guard.

Suggested fix: validate inside the try so any bad result becomes failure. Either tag untagged user messages in checkpoint as context.compaction (the way fixed-tail builds its marker), or narrow the public type to messages eve accepts. Also check tool-call/result pairing there, or document it as the strategy's job.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 865ce80: the strategy's result is validated inside the try with the rest of the compaction, so a bad result is reported as a compaction failure and the summary call is still billed. New integration test: a strategy that summarizes then returns a user message without a kind.

Comment thread docs/concepts/compaction-strategies.md Outdated
| `shouldCompact(facts)` | Whether the next request needs compacting, from the numbers eve measured. |
| `compact(history, ctx)` | The messages that replace `history`. |

`facts` holds `inputTokens`, `requestEnvelopeTokens`, `threshold`, `recentWindowSize` and `reason`. A user-requested compaction (`reason: "manual"`) runs whatever `shouldCompact` returns. `history` is the conversation without memory records; eve places recalled memory ahead of the result, so a strategy never sees or reorders it. A user message you return needs a `kind`; build one with `createCompactionUserMessage(content)` from `eve/compaction`, for example to mark a checkpoint.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"runs whatever shouldCompact returns" reads as "runs if shouldCompact says so", but the code does the opposite: force short-circuits and shouldCompact is never called (step.ts:193-195, and the integration test passes shouldCompact: () => false and expects compact to run). Two independent reads of this PR took it the wrong way. Suggest: "A user-requested compaction (reason: \"manual\") calls compact without asking shouldCompact." strategy.ts:16 has the same wording.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 865ce80: the docs now say shouldCompact decides automatic compaction only, and a manual request (reason: "manual") runs compact without calling it.

Comment on lines +11 to +12
async "compaction.completed"() {
return { messages: [{ content: "PROFILE_VALUE=RECALLED", id: "profile" }] };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compaction.completed returns the same content and id as turn.started, and the core already keeps canonical memory ahead of the strategy's result (step.ts:256-289). So the MEMORY_NOT_RECALLED_ONCE check in agent.ts:14 still passes if compaction-time recall never runs. That means the fixture doesn't prove the "memory still recalled" claim. Return a distinct value here (for example PROFILE_VALUE=RECALLED_AFTER_COMPACTION under the same id), then assert that value is visible and the turn-start value is gone.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 865ce80: the compaction.completed recall now returns its own record (PROFILE_RECALLED_AFTER_COMPACTION=YES, id profile-compaction), and the fixture's verify turn requires exactly one of it. The eval passes locally.

Comment thread packages/eve/src/compiler/manifest.ts Outdated
})
.strict();

const compiledCompactionDefinitionSchema: z.ZodType<CompiledCompactionDefinition> = z

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CompiledCompactionDefinition is ModuleSourceRef (L271), and this schema copies moduleSourceRefSchema (L371-378) field for field. Reuse it (compaction: moduleSourceRefSchema.nullable().default(null)) so the two copies can't drift. Also, a top-level compaction resource next to agent.compaction (model/threshold config, L632) is easy to mix up. A name like compactionStrategy would keep the two apart.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reused moduleSourceRefSchema and deleted the copy in 865ce80. I kept the top-level compaction name: it matches the other agent-file resources (memory) and renaming it changes the manifest shape for no behavior.

resolveModel: config.resolveModel,
runtimeIdentity: config.runtimeIdentity,
session: step.session,
strategy: config.compactionStrategy ?? defaultCompactionStrategy,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same config.compactionStrategy ?? defaultCompactionStrategy fallback appears at step.ts:105. maybeCompact is the shared boundary, so apply the default once there (strategy?: CompactionStrategy, defaulting to fixedTail()), or resolve it once when the harness config is built. Then default-strategy.ts, a one-line module, can go, and manual and automatic compaction can't end up with different defaults.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 865ce80: maybeCompact takes an optional strategy and defaults to fixedTail() once. default-strategy.ts is deleted and both call sites pass config.compactionStrategy as is.

Comment thread scripts/guard-invariants.mjs Outdated

const COMPACTION_STRATEGIES_DIR = "packages/eve/src/harness/compaction/strategies/";
const COMPACTION_STRATEGY_INTERNAL_RE =
/(?:from\s+|import\s*\()["']#harness\/compaction\/strategies\/[^"']+\/(?!index\.js["'])[^"'/]+["']/;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rule 54 only matches #harness/compaction/strategies/... specifiers. A relative deep import like ../compaction/strategies/fixed-tail/engine.js gets past it, and harness code does use relative imports (model-call/call.ts imports ./failure.js). Either resolve the specifier against the importing file before matching, or also match /compaction/strategies/<name>/<not index.js> in relative specifiers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 865ce80: rule 54 resolves #harness/ and relative specifiers against the importing file before matching, so ../compaction/strategies/fixed-tail/engine.js is flagged. Checked with a temporary file; it is removed.

Comment thread packages/eve/.generated Resolved
…rsized message

Compaction kept re-triggering on every step when one current message filled
the window. It now keeps a fixed-size tail, cuts the middle of an oversized
current request, trims tail tool results when a summary runs, summarizes
once, counts memory and client context as protected, and fails only when the
system prompt, tools, memory and context alone exceed the limit.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
@ruiconti
ruiconti force-pushed the infra/compaction-progress branch from bce36a8 to 6cd2fd8 Compare October 26, 2026 14:54
Adds a CompactionStrategy interface (shouldCompact, compact) with the fixed-tail
strategy as the default, so an agent can replace how its history is compacted.
The engine's fitting guarantees stay in the harness.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
@ruiconti

Copy link
Copy Markdown
Contributor Author

Compaction pipeline: before and after #5290 + #5332

Compares eve main at cea4eb0f (the base of #5290) with #5332's head 6f2d21dd. The compaction engine on that base is unchanged since 66491fcd, where the loop was first reproduced.

In one line: before, one request near the context limit made eve compact on every step and never get under the threshold; after, compaction keeps a small fixed tail, cuts an oversized request in the middle, always fits its own output, and fails only when the system prompt, tools, memory and context alone fill the window. The pipeline now sits behind a CompactionStrategy that an agent can replace from agent/compaction.ts.

The problem it fixes

A session whose first user message was 408 KB (~164k provider tokens) compacted before every step and its input grew instead of shrinking: 182.9k, 186.9k, 187.0k ... 188.0k against a 180k threshold, $4.19 over 10 steps before it was killed. Two faults combined:

  1. The request was never cut. withResumptionGuard replays the last real user message word for word with no size check (engine.ts:323-354), so compaction could only summarize the few messages around it.
  2. An over-limit result was accepted. At tail size 0 the text-only candidate is returned whether or not it fits (engine.ts:252), and it is measured with the bytes/4 estimate, which ran 1.6x below the provider's count for this text. The next step's provider count was over again, so it compacted again.

Before (main)

Everything is core; there is no strategy seam.

flowchart TD
  T["shouldCompact: last provider count + estimate of new messages + prompt overhead > threshold<br/>step.ts:176 / engine.ts:67"] -->|over, or /compact| C
  C["Cap older tool results to 2,000 chars, no model call<br/>engine.ts:122"] -->|fits| D[History replaced]
  C -->|still over| S["Summarize the older region<br/>engine.ts:219"]
  S --> K["Keep recent tail: up to 10 messages filling threshold minus a 2k reserve<br/>engine.ts:442"]
  K --> R["Replay last string user message whole, never cut<br/>engine.ts:323"]
  R -->|fits| D
  R -->|over| X["Text-only tail, then tail - 1 and summarize again<br/>engine.ts:247-256"]
  X -->|tail = 0| A["Return the text-only result even if over<br/>engine.ts:252"]
  A -->|next step| T
  S -. summarizer error .-> F["Error rethrown out of the step; the workflow retries it<br/>run.ts:136, 145"]
Loading
Step Code (main) What it does
Trigger step.ts:176, engine.ts:67 A core function: provider count + estimate of appended messages + prompt overhead > threshold. /compact sets force.
Fixed-part check none A system prompt + tools over the limit floors the history threshold at 1 (step.ts:257) and compaction can never fit.
Tool-result trim engine.ts:122, :276 Older tool results capped to 2,000 chars (start and end kept). Accepted if it fits on the trigger's ruler plus the provider-minus-estimate offset. The recent tail is never trimmed.
Summary call engine.ts:216-257 Summarizes the older region. Runs again each time the tail shrinks.
Kept tail engine.ts:442 Up to recentWindowSize (10) messages, filling the threshold minus a 2,048-token reserve: relative to the limit, so it can leave no headroom.
Current request engine.ts:323, :359 Latest user message with string content replayed whole. An array-content request (text + file) is skipped, so an older task can be replayed instead.
Shrink order engine.ts:238-256 Verbatim tail, text-only tail, then one message fewer and summarize again; at 0 the result is returned unchecked.
Failure run.ts:136, call.ts:285-290 The error leaves the step and the workflow retries it, re-running the summary (4 attempts measured). No turn.failed, so clients show a generic "ended without a reply". The session carrying the summary's usage is dropped.

After (#5332 head)

The core measures, decides when compaction may run, owns events, memory, billing and failure. The default fixedTail() strategy owns whether to compact and what the new history is (strategy.ts:32-62).

flowchart TD
  subgraph core
    M["Measure: inputTokens, requestEnvelopeTokens incl. memory + client context, threshold<br/>step.ts:197-211"]
    P["Fixed-part check: envelope + memory + context >= threshold -> error<br/>step.ts:57, 287"]
    E["Prepend memory records, emit compaction.completed, recall memory<br/>step.ts:300-340"]
    F["COMPACTION_FAILED: turn.failed then session.failed, no retry; summary usage kept<br/>failure.ts:207, call.ts:297"]
  end
  subgraph strategy["fixedTail strategy"]
    Q["shouldCompact(args): inputTokens + overhead > threshold<br/>engine.ts:45"]
    U["Find latest user request (any shape); over the tail budget -> keep first 20% / last 80%, middle to summarizer<br/>engine.ts:161, 429"]
    C["No cut needed and not forced: cap older tool results, accept if fits<br/>engine.ts:179-202"]
    S["Cap tool results everywhere incl. tail; floor check; summarize once<br/>engine.ts:206-241"]
    K["Kept tail <= min(20k, threshold/2)<br/>engine.ts:642"]
    X["Shrink: text-only tail -> no tail -> clip summary -> cut request, files become a note<br/>engine.ts:250-323"]
  end
  M -->|/compact skips| P
  M --> Q -->|true| P --> U --> C
  C -->|fits| E
  C -->|over| S --> K --> X -->|fits| E
  X -. nothing fits .-> F
  P -. fixed part too big .-> F
Loading
Step Owner Code (#5332) What it does
Trigger strategy, called by core step.ts:212-215, engine.ts:45 shouldCompact({ inputTokens, requestEnvelopeTokens, threshold, recentWindowSize, event: { type: "step.started" } }). /compact never calls it; compact sees ctx.forced. Same numbers and same rule as before.
Fixed-part check core step.ts:57-68 Before any summary call: envelope + memory + client context (projected prompt minus history, step.ts:197) ≥ threshold throws "The agent's system prompt, tools, memory and context take ~N tokens, over this model's ~M-token compaction limit."
Tool-result trim strategy engine.ts:179-206 No-summary path: older results only, as before (newest result the model hasn't answered stays whole). Summary path: every result, tail included, capped to 2,000 chars, so one large file read can't take the tail budget.
Summary call strategy, via core's summarize engine.ts:238, step.ts:250 Exactly one call. The compaction model is resolved on first use, so a strategy that never summarizes needs none.
Kept tail strategy engine.ts:642 min(20,000, threshold/2) estimated tokens, replayed request included. Fixed, not relative to the limit: at a 1.6x undercount 20k is ~32k real, far from any realistic threshold.
Oversized request strategy engine.ts:429-463 Latest real user message of any shape. Over the tail budget: keep the first 20% and last 80% of the budget word for word, a note at the cut, the middle goes to the summarizer in full (as far as its transcript budget allows). File parts stay; no surrogate pair is split.
Shrink order strategy engine.ts:250-323 Verbatim tail → text-only tail → no tail → summary clipped to the largest piece that fits (binary search) → request's files replaced by a note and its text cut to the room left. No second summary. The request is never dropped.
Failure core failure.ts:207, run.ts:136 CompactionOverLimitError (fixed part too big, or floor/shrink can't fit) → turn.failed with code COMPACTION_FAILED and the message, then session.failed; no step retry. Other errors (summarizer down) throw as before. The session with the summary's usage is saved first (call.ts:297).

The seam

  • agent/compaction.ts exports defineCompaction({ shouldCompact, compact }) (public/compaction/index.ts); fixedTail() is exported so a strategy can wrap the default.
  • compact(history, ctx) gets the conversation without memory records; ctx carries the four numbers, forced, summarize, estimateTokens, abortSignal. It returns the replacement messages; user messages are built with createCompactionUserMessage.
  • A remote subagent's compaction.ts is rejected, not ignored.
  • Guard rule 54 (guard-invariants.mjs:157): outside harness/compaction/strategies/, a strategy is imported only through its index.js.

What changed and why

Behavior Before After Why
Kept tail ≤10 messages filling threshold − 2k ≤ min(20k, threshold/2) A tail relative to the limit leaves no headroom when the estimate is low; a fixed cap makes the estimate's error irrelevant. A provider-ratio correction was tried and removed as too fragile (memory records, unsizable PDFs, previous checkpoint).
Oversized request replayed whole start + end kept, middle summarized The 408 KB loop. Middle, not head or tail: instructions sit at the start or end, pasted bulk in the middle.
Request lookup string content only any shape An array-content request (text + file) was skipped and an older task replayed.
Tail tool results always whole capped on the summary path only One 100 KB read would exceed the 20k tail; on the no-summary path the newest unanswered result must stay whole.
Summary calls one per tail shrink one A smaller tail never makes the summary smaller.
Over-limit result accepted at tail 0 shrunk until it fits Compaction controls its own output, so it shouldn't give up; accepting over-limit output is what looped.
Fixed part unchecked checked first, with a message naming it The only part compaction can't shrink is prompt, tools, memory and context; that's a configuration problem, not "start a new session".
File in request kept kept if it fits; note only when nothing else fits Explicit choice: replace a too-large file with a note so the turn continues.
Failure step retried (4 summaries) turn.failed + session.failed, no retry Retrying re-summarizes the same history; clients need the reason, which turn.failed carries.
Summary usage on failure lost kept The new over-limit error is the first failure that can follow a successful summary.
Who decides core functions replaceable strategy Strategies (provider-side compaction, saving cut text to a file) without core edits.

The 408 KB case, real model

Same recipe both sides: sonnet-5.5, 200k window, thresholdPercent 0.9 (180k), ~29k-token fixed prompt, first user message = 408,000 chars of a real thread.

Step Before (main code) After
0 182.9k 183.5k
1 186.9k (compacted) 56.1k (compacted)
2 187.0k (compacted) 56.3k
3 187.3k (compacted) 56.4k
4 — 56.4k, turn completes
9 188.0k (compacted, killed at step 10)
Cost $4.19 over 10 steps $0.65 over 5 steps

The "after" column was measured on #5290 at e3cfe946. The strategy refactor (#5332 at b09e8cb6) measured 183.5k then 56.4k flat (+0.6%), and later commits, which only touch the shrink path and the fixed-part check, gave one compaction and 5 steps on the same recipe. No real-model run has been made on 6f2d21dd itself.

Base automatically changed from infra/compaction-progress to main October 26, 2026 16:35
import { defineCompaction } from "eve/compaction";

export default defineCompaction({
shouldCompact: (args) => args.inputTokens > args.threshold,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think should be ctx and we need to make sure the current model is exposed, as well as its context window if available. We can also use this opportunity to not require a context window if we want. right now we fail build without it, maybe that is a good thing im not 100% sure.

| `shouldCompact(args)` | Whether the next request needs compacting, from the numbers eve measured. |
| `compact(history, ctx)` | The messages that replace `history`. |

`args` holds `inputTokens`, `requestEnvelopeTokens`, `threshold`, `recentWindowSize` and `event`, the eve event that prompted the check (today only `{ type: "step.started" }`; new triggers add members). It decides automatic compaction only: a user-requested compaction runs `compact` without calling it, and `ctx.forced` is `true` there. `ctx` carries the same four numbers (no `event`), plus `forced`. `history` is the conversation without memory records; eve places recalled memory ahead of the result, so a strategy never sees or reorders it. A user message you return needs a `kind`; build one with `createCompactionUserMessage(content)` from `eve/compaction`, for example to mark a checkpoint.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

threshold is very vague to me where does this come from what does it mean

ruiconti added a commit that referenced this pull request Oct 27, 2026
An agent exports a compaction strategy from agent/compaction.ts. The core
owns when compaction can run (a fixed 90% overflow guard, or a hook's
ctx.compact()) and the events, memory, failure and billing around it; the
strategy owns what the compacted history looks like. If a strategy's result
is still over the guard, the default fixedTail() runs once on it.

Adds ctx.view.context.inputTokens for hooks. Removes
compaction.thresholdPercent. Supersedes #5332.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
@ruiconti

Copy link
Copy Markdown
Contributor Author

Superseded by #5410, which rebuilds the compaction API as an authored strategy with ctx.compact().

@ruiconti ruiconti closed this Oct 27, 2026

This branch was successfully deployed

2 active deployments
Preview – eve-docs — 6f2d21dd Deployed Oct 26, 2026 by vercel[bot]
Preview – eve-pkg — 6f2d21dd Deployed Oct 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants