Skip to content

fix: accumulate tokens across multi-step tool calls instead of overwriting#22077

Open
KonstantinMirin wants to merge 1 commit intoanomalyco:devfrom
KonstantinMirin:fix/multi-step-token-accumulation
Open

fix: accumulate tokens across multi-step tool calls instead of overwriting#22077
KonstantinMirin wants to merge 1 commit intoanomalyco:devfrom
KonstantinMirin:fix/multi-step-token-accumulation

Conversation

@KonstantinMirin
Copy link
Copy Markdown

Issue for this PR

Fixes #21913

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

In multi-step tool-call messages, processor.ts overwrites assistantMessage.tokens on each finish-step event instead of accumulating additive fields. Only the last step's output, reasoning, and cache.write survive. The fix: accumulate output, reasoning, cache.write with += across steps; replace input and cache.read with the latest step's values (they're snapshots).

Additionally fixes custom provider limit.context defaulting to 0 (now 128_000) and limit.output defaulting to 0 (now 4_096), which broke context % display and disabled auto-compaction.

Key changes:

  • processor.ts: field-wise token accumulation instead of wholesale overwrite
  • index.ts: getUsage() derives total from components (input + output + reasoning + cache.read + cache.write) instead of using API's totalTokens (which double-counts cached tokens)
  • message-v2.ts: add promptSize() and totalSize() named helpers
  • overflow.ts: use MessageV2.totalSize() for compaction threshold (output/reasoning are part of next turn's context)
  • provider.ts: named constants DEFAULT_CONTEXT_LIMIT / DEFAULT_OUTPUT_LIMIT instead of magic 0
  • agent.ts: include cache.write in ACP usage reporting

How did you verify your code works?

1903 existing tests pass + 12 new tests covering:

  • promptSize/totalSize helpers (3 unit tests)
  • isOverflow pure unit tests (5 tests: disabled, zero context, underground, overflow, custom reserved)
  • getUsage total derivation from components (1 test)
  • 3-step accumulation simulation mirroring processor.ts logic (1 test)
  • Provider limit defaults (1 test, updated from 0 to 128_000/4_096)
  • Overflow semantics test (1 test, updated from total-based to totalSize-based)

Screenshots / recordings

N/A — no UI changes.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

…iting

Multi-step tool-call messages only retained the last step's output,
reasoning, and cache.write tokens because processor.ts overwrote
assistantMessage.tokens on each finish-step event instead of
accumulating additive fields.

- Accumulate output, reasoning, cache.write with += across steps
- Replace input and cache.read with latest step's values (snapshots)
- Derive total from components instead of API totalTokens (which
  double-counts cached tokens in AI SDK v6)
- Add MessageV2.promptSize() and totalSize() helpers for consistent
  computation across consumers
- Use totalSize() for compaction threshold (output/reasoning are part
  of the next turn's context)
- Fix custom provider limit.context defaulting to 0 (now 128,000) and
  limit.output defaulting to 0 (now 4,096) which broke context %
  display and disabled auto-compaction
- Include cache.write in ACP usage reporting

Fixes anomalyco#21913
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-step tool calls overwrite token counts instead of accumulating (output, reasoning, cache.write lost)

1 participant