fix(eve): stop compacting on every step when the current message fills the context (#5290)

Signed-off-by: Rui Conti <ruiconti@gmail.com>

Author: ruiconti

.changeset/compaction-oversized-message.md

@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Compaction no longer re-runs on every step when the current user message is most of the context window. After a compaction, eve keeps at most 20,000 tokens of the latest conversation word for word next to the summary, with tool results shortened to 2,000 characters; a current message larger than that keeps its start and end, and the cut-out middle is summarized. A compaction that cannot bring the history under the threshold now fails the session, without retrying the step, with an error that says to start a new session or send a smaller message.

docs/concepts/default-harness.md

@@ -7,7 +7,7 @@ The default harness is eve's built-in agent loop. It manages model calls, compac
 
 ## Compaction
 
-The harness keeps a long session from overflowing the model's context window. Before comparing the conversation with `thresholdPercent` (`0.9` by default), it adds the estimated fixed envelope of the checkpoint prompt used for compaction. It then summarizes the older turns and keeps going. The prompt asks the compaction model to distinguish completed progress and decisions from remaining work and to retain the constraints, preferences, data, and references needed to continue. When eve compacts again, it passes the previous checkpoint separately and without the transcript's per-message truncation, then replaces it with the updated checkpoint. The summary uses the active turn model unless you override it. Tune when and how it kicks in under [`compaction`](../agent-config#compaction) in `agent.ts`:
+The harness keeps a long session from overflowing the model's context window. Before comparing the conversation with `thresholdPercent` (`0.9` by default), it adds the estimated fixed envelope of the checkpoint prompt used for compaction. It then summarizes the older turns and keeps going. The prompt asks the compaction model to distinguish completed progress and decisions from remaining work and to retain the constraints, preferences, data, and references needed to continue. When eve compacts again, it passes the previous checkpoint separately and without the transcript's per-message truncation, then replaces it with the updated checkpoint. The summary uses the active turn model unless you override it. After compacting, the harness keeps the most recent messages word for word next to the summary, up to 20,000 tokens (or half of the threshold, if that is smaller), with tool results shortened to 2,000 characters. The current user message is always part of that, so when it alone is larger than the budget the harness keeps only its start and its end, with a note where the middle was cut, and summarizes the cut-out middle with the older turns. If a compaction cannot bring the history under the threshold, the session fails with an error instead of compacting again, and the step is not retried: start a new session or send a smaller message. Tune when and how it kicks in under [`compaction`](../agent-config#compaction) in `agent.ts`:
 
 ```ts title="agent/agent.ts"
 export default defineAgent({

e2e/fixtures/agent-compaction-regressions/agent/agent.ts

@@ -20,16 +20,23 @@ const TEST_CONTEXT_WINDOW_TOKENS = 32_000;
 // block, and 10 advertised tools; with nothing deferred and no skills or
 // connections, the agent has no catalog tools) at 2,311.5
 // tokens. This allowance sits 230.25 tokens below that estimate, the offset the
-// cases have always run with, so the threshold leaves them a ~670-token history
+// cases have always run with, so the threshold leaves them a ~4,770-token history
 // budget. When eve's fixed instructions or tools change, re-measure: log what
 // `estimateRequestEnvelope` (packages/eve/src/harness/request-envelope.ts)
 // returns while `EVE_E2E_MODEL=mock pnpm exec eve eval --strict
 // content-output-file-stub` runs in this fixture, subtract 230.25, and round up.
 const TEST_REQUEST_ENVELOPE_TOKENS = 2_082;
-// Fit the capped file-output exchange, while forcing the larger review and
-// handoff reports into the assistant checkpoint consumed by the script.
-const TEST_HISTORY_BUDGET_TOKENS = 900;
-const COMPACTION_PRESSURE_USAGE = { inputTokens: TEST_REQUEST_ENVELOPE_TOKENS + 4_096 };
+// History budget above the envelope. Compaction refuses a result that stays
+// over it, so it must hold the assistant checkpoint (the review and handoff
+// reports, about 3k tokens) plus the compaction prompt's own overhead; it
+// must stay below the capped file-output exchange's weight (a 2000x2000 image,
+// ~5k tokens) so that case still compacts.
+const TEST_HISTORY_BUDGET_TOKENS = 5_000;
+// Reported once a tool step has run: just over the budget, so the harness
+// compacts, without claiming a history far bigger than the one it has.
+const COMPACTION_PRESSURE_USAGE = {
+  inputTokens: TEST_REQUEST_ENVELOPE_TOKENS + TEST_HISTORY_BUDGET_TOKENS + 1,
+};
 const MAX_TOOL_CALLS = 10;
 
 type RegressionCase = "content-output-file-stub" | "redundant-tool-calls" | "task-survival";

e2e/fixtures/agent-compaction-regressions/evals/task-survival.eval.ts

@@ -12,7 +12,7 @@ export default defineEval({
         "Alice is updating the library's reading-list application. Please review the repository and confirm these requirements for Bob's handoff.",
         // Sized to a narrow window: long enough that a 280-char summarizer
         // input cap would destroy the tail sentinel, but short enough that
-        // the task alone cannot cross the fixture's ~640-token history budget —
+        // the task alone cannot cross the fixture's history budget —
         // otherwise compaction fires before the first model call and the
         // mock never generates pressure.
         [

e2e/fixtures/agent-compaction-regressions/release-findings.ts

@@ -1,4 +1,4 @@
-export const repositoryFindings = [
+const repositoryNotes = [
   "Application startup loads configuration before registering the catalog, cart, checkout, and order routes. An invalid currency setting prevents the application from accepting requests and identifies the setting that needs correction.",
   "The catalog exposes product identifiers, display names, prices in cents, and availability. Product identifiers remain stable when a display name changes, so a saved cart continues to identify the intended product.",
   "Catalog searches return an empty collection when nothing matches. The storefront distinguishes that state from a failed search and offers a clear way to remove filters without losing the customer's current cart.",
@@ -15,7 +15,7 @@ export const repositoryFindings = [
   "The review covers startup, catalog, cart, checkout, saved orders, and notifications. Release validation should now concentrate on the purchase and cancellation journeys, including retries, without repeating this repository inspection.",
 ];
 
-export const handoffNotes = [
+const handoffEntries = [
   "Repository review is complete for application startup, catalog browsing, cart updates, checkout, order history, and notifications. Carry these findings into release validation; the next reviewer should not repeat the completed inspection.",
   "Start release validation with a catalog containing available and unavailable products. Confirm that filters show an empty state when appropriate and that clearing a filter restores the product list while preserving the current cart contents.",
   "Exercise cart updates with a new product, an existing product, and removal of the final item. Include invalid quantities and confirm that rejected updates leave the last valid cart visible, with an explanation tied to the offending input.",
@@ -31,3 +31,17 @@ export const handoffNotes = [
   "Keep unresolved release checks distinct from completed review work in the handoff. If validation exposes a new defect, record the new observation and affected journey rather than reopening the repository inspection without additional evidence.",
   "After the release checks are recorded, publish the concise handoff with the completed review result and checkpoint status. Link each recorded release check to the journey it covers so later reviewers can trace every result back to this plan.",
 ];
+
+// Each record is about 4,400 tokens, nearly the whole history budget, so the
+// task and a record never fit together: a compaction has to summarize the
+// record into the checkpoint rather than keep it in the tail.
+const RECORD_PASSES = 5;
+
+function repeatPasses(notes: readonly string[]): string[] {
+  return Array.from({ length: RECORD_PASSES }, (_, pass) =>
+    notes.map((note) => (pass === 0 ? note : `${note} (Confirmed again in pass ${pass + 1}.)`)),
+  ).flat();
+}
+
+export const repositoryFindings = repeatPasses(repositoryNotes);
+export const handoffNotes = repeatPasses(handoffEntries);

e2e/fixtures/agent-compaction-regressions/report-evidence.ts

@@ -7,9 +7,12 @@ export function assistantHasReport(
   const escapedReference = reference.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
   const recordReference = new RegExp(`(?<![\\w/-])${escapedReference}(?![\\w/-])`);
 
-  // Only successful tool results introduce these references. Requiring their
-  // retention in assistant text proves they survived a checkpoint or trail.
+  // Only successful tool results introduce these references. Their presence in
+  // what the model is shown (a surviving tool result, or assistant text after a
+  // checkpoint or trail) proves they survived compaction.
   return messages.some(
-    (message) => message.role === "assistant" && recordReference.test(message.text),
+    (message) =>
+      (message.role === "assistant" || message.role === "tool") &&
+      recordReference.test(message.text),
   );
 }

packages/eve/src/execution/catalog/step-catalog.integration.test.ts

@@ -479,11 +479,17 @@ describe("step catalog in the harness (real AI SDK)", () => {
     const historyBeforeCompaction = driver.session.history;
 
     // Compaction replaces history; the next request starts a fresh baseline.
-    // Any real history exceeds a threshold of 1, so the next step compacts.
+    // Compaction refuses a result that stays over the threshold, so the
+    // threshold must leave room for the summary. Old filler, appended after
+    // the last provider-measured prompt, pushes the history over it.
     const compactionStep = mark();
     driver.session = {
       ...driver.session,
-      compaction: { ...driver.session.compaction, threshold: 1 },
+      compaction: { ...driver.session.compaction, threshold: 2_000 },
+      history: [
+        ...driver.session.history,
+        { content: "Earlier context. ".repeat(1_000), kind: "user", role: "user" },
+      ],
     };
     reply(text("Ready for the next request."));
     await drive({ message: "Alice asks what is next." });