Repository navigation
feat(eve): authored compaction strategies and ctx.compact() #5410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
ruiconti
wants to merge
19
commits into
main
Choose a base branch
from
pe/ctx-compact
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
23e4d1a
feat(eve): authored compaction strategies and ctx.compact()
ruiconti 3b584d9
fix(eve): settle a failed requested compaction, validate strategy res…
ruiconti ffcfe27
feat(eve): name the removed compaction.thresholdPercent
ruiconti f8da26f
Measure authored results like the trigger, isolate strategy history, …
ruiconti f7f61ba
Take the summary model from the strategy: fixedTail({ model })
ruiconti 89ca4eb
Record the hook that requested a compaction as its cause
ruiconti 8c209a9
Expose the compaction limit as ctx.view.context.limit
ruiconti bf1e5c2
Subagents without their own compaction use the nearest ancestor's str…
ruiconti eb59890
Remove agent.ts compaction.model: the summary model comes from fixedT…
ruiconti 51de59f
Brand fixedTail() so it can be the compaction.ts export; fix the stra…
ruiconti 2e6cfc9
Annotate the fixture's fixedTail export with the public strategy type
ruiconti 17f0fe2
Fixture: declare the summary-model strategy with defineCompaction
ruiconti 2c64b6f
Compaction review fixes: invalid results fail once, copy keeps URLs, …
ruiconti 70fcd97
Compaction guard: scale the provider undercount by the share of conte…
ruiconti 89b2dd6
fixedTail: recentTokens and modelContextWindowTokens options
ruiconti ead5ef4
fixedTail: drop modelContextWindowTokens; the removed compaction sett…
ruiconti 7899e48
Compaction guard: apply the provider undercount only to a result that…
ruiconti 85c1057
Compaction guard: judge a result by its estimate and require it to sh…
ruiconti 1e2e669
fix(eve): keep session view snapshots free of own keys
ruiconti File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "eve": minor | ||
| --- | ||
|
|
||
| An agent can now choose how its history is compacted. Export a strategy from `agent/compaction.ts` with `defineCompaction` from `eve/compaction`; its `compact` returns the replacement history, and eve keeps the events, memory recall, failure handling and billing. A hook can request a compaction with `ctx.compact()`, and `ctx.view.context.inputTokens` reports the latest model call's input tokens. eve still compacts when a request would fill 90% of the context window, and runs the default `fixedTail()` once if a strategy's result is still over it. The default is unchanged. `compaction.thresholdPercent` is removed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| --- | ||
| title: "Compaction strategies" | ||
| description: "Replace how an agent shortens its history by exporting a strategy from compaction.ts." | ||
| --- | ||
|
|
||
| By default eve compacts with the fixed-tail strategy: a summary of everything but the latest conversation, which stays word for word up to 20,000 tokens. See [Default harness](./default-harness). To compact differently, export a strategy from `agent/compaction.ts`. | ||
|
|
||
| ```ts title="agent/compaction.ts" | ||
| import { defineCompaction } from "eve/compaction"; | ||
|
|
||
| export default defineCompaction({ | ||
| async compact(history) { | ||
| return history.slice(-6); | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ## When it runs | ||
|
|
||
| eve compacts in two cases, and a strategy decides only how: | ||
|
|
||
| - **Overflow guard.** A request that would fill 90% of the model's context window is compacted before the model call. This is fixed. | ||
| - **A hook asks.** A [hook](../guides/hooks) calls `ctx.compact()` and the next model call compacts first: the running turn's next call, or the first call of the next turn when no turn is running. Asking twice before it runs compacts once. | ||
|
|
||
| ```ts title="agent/hooks/trim.ts" | ||
| import { defineHook } from "eve/hooks"; | ||
|
|
||
| export default defineHook({ | ||
| events: { | ||
| "model.settled"(_event, ctx) { | ||
| if ((ctx.view.context.inputTokens ?? 0) > 60_000) ctx.compact(); | ||
| }, | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| `ctx.view.context.inputTokens` is the input tokens of the latest settled model call. | ||
|
|
||
| If a strategy's result still leaves the request over the guard, eve runs the default `fixedTail()` once on it, and that result must fit. | ||
|
|
||
| ## What eve does and what the strategy does | ||
|
|
||
| eve measures the next request and owns everything around the strategy: the `context.started` and `context.settled` events, [memory](../memory) capture and recall, failure, and billing of the summary call. The strategy owns one decision. | ||
|
|
||
| | Member | Decides | | ||
| | ----------------------- | ------------------------------------ | | ||
| | `compact(history, ctx)` | The messages that replace `history`. | | ||
|
|
||
| `ctx` carries `inputTokens`, `requestEnvelopeTokens`, `threshold`, `recentWindowSize`, and `forced`, which is `true` when a user or a hook asked and the request was not over the limit. `history` is the conversation without memory records; eve places recalled memory ahead of the result, so a strategy never sees or reorders it. A user message you return needs a `kind`; build one with `createCompactionUserMessage(content)` from `eve/compaction`, for example to mark a checkpoint. | ||
|
|
||
| `ctx` adds `summarize({ messages, system })`, which runs the compaction model and bills the session, `estimateTokens(messages)`, and, when the turn can be cancelled, an `abortSignal`. A strategy that never summarizes makes no model call. | ||
|
|
||
| ## Failure | ||
|
|
||
| Throw `CompactionOverLimitError` (exported from `eve/compaction`) when no history the strategy could return fits. Any thrown error fails the compaction the way a failed summary does. | ||
|
|
||
| ## Reusing the default | ||
|
|
||
| `fixedTail()` from `eve/compaction` returns the default strategy, so a strategy can wrap it, for example to compact earlier than the threshold. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Further down, the cancel section (line 265) still says "Hook-side
ctx.compactandctx.onceare not available." That now contradicts this section. Suggest droppingctx.compactfrom that sentence.