diff --git a/docs/user-guide/en/token-saving/tokenless/cli-reference.md b/docs/user-guide/en/token-saving/tokenless/cli-reference.md index 8a4ad829f..fe9d9a4b0 100644 --- a/docs/user-guide/en/token-saving/tokenless/cli-reference.md +++ b/docs/user-guide/en/token-saving/tokenless/cli-reference.md @@ -125,7 +125,7 @@ Field matching and truncation change the response representation seen by the mod Stash applies only to truncation of strings, array tails, and deep subtrees. Blacklisted fields, `null`, and empty values are removed without a retrieval marker. -Most adapters override these standalone defaults. Their shared shell profile uses `65536`, `128`, and `8`; the other-structured-tool profile uses `1048576`, `65536`, and `32`. Content-retrieval tools are skipped. See [Framework integration · Adapter processing rules](framework-integration.md#adapter-processing-rules). +Most adapters override these standalone defaults. Their shared shell profile uses `65536`, `128`, and `8`; the other-structured-tool profile uses `1048576`, `65536`, and `32`. Content-retrieval tools are skipped. See [Framework integration · Adapter processing rules](framework-integration.md#adapter-processing-rules) and [User manual · Compression trigger conditions and thresholds](user-manual.md#compression-trigger-conditions-and-thresholds). ## `compress-toon` and `decompress-toon` diff --git a/docs/user-guide/en/token-saving/tokenless/framework-integration.md b/docs/user-guide/en/token-saving/tokenless/framework-integration.md index 6ba323366..a66c91eee 100644 --- a/docs/user-guide/en/token-saving/tokenless/framework-integration.md +++ b/docs/user-guide/en/token-saving/tokenless/framework-integration.md @@ -43,6 +43,8 @@ The shared response hook, OpenClaw, and Hermes skip inputs shorter than 200 char Claude Code requires version 2.1.121 or later for `updatedToolOutput`. On older or unknown versions, response compression is disabled to avoid duplicating the original. Structured tool outputs preserve their host schema and do not switch to textual TOON; JSON carried as a string can use TOON when it is smaller. +For the full trigger conditions (compression switch, minimum response length, JSON validation, strictly-smaller guard) and threshold semantics, see [User manual · Compression trigger conditions and thresholds](user-manual.md#compression-trigger-conditions-and-thresholds). + ### DeepSeek Harness native processing The DSH bundle requires Node.js 22 or later and a compatible DSH profile. Pass diff --git a/docs/user-guide/en/token-saving/tokenless/measuring-savings.md b/docs/user-guide/en/token-saving/tokenless/measuring-savings.md index 30c5675b2..fee4aeb17 100644 --- a/docs/user-guide/en/token-saving/tokenless/measuring-savings.md +++ b/docs/user-guide/en/token-saving/tokenless/measuring-savings.md @@ -173,6 +173,59 @@ Estimated overall saving rate For example, a 60% payload compression rate with tool payloads representing 20% of the session gives an estimated overall saving of about 12%. This is still not a provider billing guarantee. +## Where the compression rate applies + +The Tokenless compression rate depends on how much removable material a payload contains, and varies widely by scenario. Reference values measured with the standard test load below (measured at commit `2e7d69f1`; re-run locally after upgrading): + +| Scenario | Reference savings (estimated tokens) | Notes | +|----------|--------------------------------------|-------| +| Structured JSON response (uniform records + redundant fields), response compression alone | ~66% | blacklisted fields and empty values removed; over-long strings/arrays truncated | +| Function-calling schema (long descriptions), schema compression alone | ~47% | description truncation; `title`/`examples` and code blocks removed | +| Mixed workload (response + schema): response compression only | ~62% | responses dominate the savings | +| Mixed workload: schema + response stacked | ~65% | both payload types reduced | +| Mixed workload: full stack (schema + response + TOON, deployed gating) | ~65% | the deployed `compress-toon` size guard keeps the original input when TOON does not reduce estimated tokens; on this fixture TOON inflates after response/schema compression, so the deployed rate equals the stacked rate above. The benchmark's ungated `full_stack` config measures ~63% but is not what the deployed path emits | +| TOON encoding only | ~16% | only tabular, regular JSON benefits clearly | + +By scenario: + +- **High savings**: tools returning many uniform records (lists, tables, search results), payloads carrying redundant fields such as `debug`/`trace`/`logs`, or schemas with verbose descriptions. +- **Moderate savings**: shell output is truncated only beyond the Layer 2 thresholds (strings 65,536 chars, arrays 128 items, depth 8); output below the thresholds stays mostly unchanged. +- **Near-zero savings**: responses shorter than the minimum trigger length (200 characters for shared adapters, 500 for Codex); already-compact JSON without redundancy; any input that does not shrink (the size guard keeps the original). +- **Not compressed at all**: content-retrieval tool output (Read/Glob/Grep and the like), non-JSON text, and skill-like text with a YAML frontmatter header. See [User manual · Compression trigger conditions and thresholds](user-manual.md#compression-trigger-conditions-and-thresholds) for the full trigger rules. + +Real session savings must additionally be multiplied by the share of tool payloads in total session tokens; see [Interpret the saving rate correctly](#interpret-the-saving-rate-correctly) above. + +## Standard test load + +The repository ships a deterministic standard test load under `src/tokenless/benchmark/l1-compressor` (a standalone Cargo workspace; Linux only — unsupported on macOS and Windows). The load is generated by `python/gen_fixtures.py`, uses no randomness, is byte-for-byte reproducible, and is committed to the repository: + +| Load file | Content | +|-----------|---------| +| `fixtures/records.json` | 1,000 uniform records | +| `fixtures/tool_response.json` | a realistic tool response (envelope + 60 records + `trace`/`logs` redundant fields) | +| `fixtures/schema_search.json` | a realistic function-calling schema | + +Run the compression-rate report quickly (build required; output covers single-path rates, stacked compression configs, and cost projections): + +```bash +cd src/tokenless/benchmark/l1-compressor +cargo run --release --bin compression_rate # human-readable report +cargo run --release --bin compression_rate -- --json # machine-readable JSON; binary flags must follow `--` +``` + +Run the full quality/adversarial tests plus the rate report (skips criterion performance benches; takes a few minutes): + +```bash +cd src/tokenless/benchmark/l1-compressor +./run-benchmarks.sh --quick +``` + +Notes on using the standard load: + +- Token counts use a bytes/4 heuristic estimate; they are meant for relative comparison across versions, and absolute values do not represent billed tokens. +- Compression rates evolve across versions; cite the corresponding commit or version when quoting numbers. +- The standard load serves cross-version comparison, not your business data; for real-world savings, measure your own workload with a [dry-run comparison](#run-a-dry-run-comparison). + ## Local AgentSight display AgentSight's Token savings view can aggregate `~/.tokenless/stats.db` read-only. When both run as the same user and AgentSight can access that database, SLS is not required to display local Tokenless statistics. diff --git a/docs/user-guide/en/token-saving/tokenless/user-manual.md b/docs/user-guide/en/token-saving/tokenless/user-manual.md index d3b4325ff..e3ea55c24 100644 --- a/docs/user-guide/en/token-saving/tokenless/user-manual.md +++ b/docs/user-guide/en/token-saving/tokenless/user-manual.md @@ -114,6 +114,34 @@ This setting does not disable RTK command rewriting, adapter execution, or retri anolisa adapter disable tokenless ``` +### Compression trigger conditions and thresholds + +Adapters do not compress every tool result. For response compression, compressed content is produced only when all of the following hold: + +1. Compression is not switched off. With `compression_enabled=false` or `TOKENLESS_COMPRESSION_ENABLED=0` the run becomes a dry-run: statistics are still calculated, but the original text is returned (see the previous section). +2. The tool is not a content-retrieval tool. Read/Glob/Grep/LSP/NotebookRead and their aliases skip response compression so their content stays intact. +3. The response reaches the minimum length. The shared response hook, OpenClaw, and Hermes skip responses shorter than 200 characters; Codex skips responses shorter than 500 characters. Length is counted in characters, not bytes. +4. The content is valid JSON. Response compression only handles JSON; plain text passes through unchanged. The shared paths additionally skip skill-like text with YAML frontmatter. +5. The compressed result is strictly smaller. When neither response compression nor TOON encoding makes the content smaller, the original text is kept. + +After these checks, truncation strength depends on the tool category. Categories and thresholds are defined in `tool_categories.json` inside the adapter directory (the single source of truth shared by all adapters); built-in safe fallbacks are used when the file is missing or invalid: + +| Category | Representative tools | String truncation threshold | Array item cap | Maximum nesting depth | +|----------|----------------------|------------------------------|----------------|-----------------------| +| Content retrieval | Read, Glob, Grep, LSP, NotebookRead and aliases | Compression skipped | — | — | +| Shell/exec | Bash, Shell, exec, terminal, etc. | 65,536 characters | 128 items | 8 | +| Other structured tools | Any tool not in the two categories above | 1,048,576 characters | 65,536 items | 32 | + +Threshold semantics: a string longer than the threshold is cut at the threshold (retrievable through Stash when Stash is enabled); an array over the cap keeps only its leading items while the tail is truncated (also retrievable through Stash when enabled); subtrees nested deeper than the depth cap collapse into a truncation marker. + +Per-path differences worth noting: + +- Running `tokenless compress-response` standalone uses the CLI's own defaults (4,096 characters / 32 items / depth 8), overridable with `--truncate-strings-at`, `--truncate-arrays-at`, and `--max-depth`; see the [CLI reference](cli-reference.md). +- Codex adds task-management tools (TodoWrite, Task, TaskStatus) to the shared skip list, and includes compressed content in context only for responses of at least 4,000 characters. +- The OpenClaw plugin can override the tool categories through `skip_tools` and `shell_tools`; the thresholds themselves still come from `tool_categories.json`. See [Configuration and data privacy](configuration-and-privacy.md) for the options. +- TOON encoding is a separate trigger decision: it is only adopted when the encoded result is smaller than the current content. +- The AgentScope framework integration does not use the adapter thresholds above; it selects thresholds by `conservative` / `balanced` / `aggressive` mode. See [Framework integration](framework-integration.md). + ### Reversible compression is conditional Active response and schema truncation stash the removed payload in `~/.tokenless/stash.db` by default and add a marker such as: @@ -167,6 +195,7 @@ Command rewriting also changes the shell command submitted by the host. Most ada | Build the in-process Python runtime | [This page · Build the Python runtime from source](#build-the-python-runtime-from-source) | | Connect an Agent product or integrate AgentScope | [Agent and framework integration](framework-integration.md) | | Compress, retrieve, or run MCP manually | [CLI reference](cli-reference.md) | +| Understand when compression triggers and what the thresholds are | [This page · Compression trigger conditions and thresholds](#compression-trigger-conditions-and-thresholds) | | Inspect savings or content changes, or run a dual comparison | [Measuring savings](measuring-savings.md) | | Change settings or understand local data | [Configuration and data privacy](configuration-and-privacy.md) | | Fix missing statistics, adapter, or Stash issues | [Troubleshooting](troubleshooting.md) | diff --git a/docs/user-guide/zh/token-saving/tokenless/cli-reference.md b/docs/user-guide/zh/token-saving/tokenless/cli-reference.md index 39a548aac..a91cff953 100644 --- a/docs/user-guide/zh/token-saving/tokenless/cli-reference.md +++ b/docs/user-guide/zh/token-saving/tokenless/cli-reference.md @@ -125,7 +125,7 @@ debug, trace, traces, stack, stacktrace, logs, logging Stash 只作用于字符串、数组尾部和深层子树截断。黑名单字段、`null` 和空值会直接移除,不会生成取回标记。 -大多数 Adapter 会覆盖这些独立 CLI 默认值。共享 Shell 策略使用 `65536`、`128`、`8`;其他结构化工具策略使用 `1048576`、`65536`、`32`。内容读取类工具会被跳过。详见[框架集成 · Adapter 处理规则](framework-integration.md#adapter-处理规则)。 +大多数 Adapter 会覆盖这些独立 CLI 默认值。共享 Shell 策略使用 `65536`、`128`、`8`;其他结构化工具策略使用 `1048576`、`65536`、`32`。内容读取类工具会被跳过。详见[框架集成 · Adapter 处理规则](framework-integration.md#adapter-处理规则)和[用户手册 · 压缩的触发条件与阈值](user-manual.md#压缩的触发条件与阈值)。 ## `compress-toon` 与 `decompress-toon` diff --git a/docs/user-guide/zh/token-saving/tokenless/framework-integration.md b/docs/user-guide/zh/token-saving/tokenless/framework-integration.md index 6f004890c..904f6e7c1 100644 --- a/docs/user-guide/zh/token-saving/tokenless/framework-integration.md +++ b/docs/user-guide/zh/token-saving/tokenless/framework-integration.md @@ -43,6 +43,8 @@ OpenCode 当前使用下文说明的随附生命周期脚本,本版本尚未 Claude Code 需要 2.1.121 或更高版本才能使用 `updatedToolOutput`。版本更旧或无法确定时,响应压缩会关闭,以免重复注入原文。结构化工具输出会保留宿主 Schema,不会转换成文本 TOON;以字符串承载的 JSON 在 TOON 更小时可以使用 TOON。 +完整触发条件(压缩开关、最小响应长度、JSON 校验、严格变小保护)与阈值含义见[用户手册 · 压缩的触发条件与阈值](user-manual.md#压缩的触发条件与阈值)。 + ### DeepSeek Harness 原生处理路径 DSH Bundle 要求 Node.js 22 或更高版本,并需要兼容的 DSH profile。应在同一条 diff --git a/docs/user-guide/zh/token-saving/tokenless/measuring-savings.md b/docs/user-guide/zh/token-saving/tokenless/measuring-savings.md index a1393f223..479d0ddfa 100644 --- a/docs/user-guide/zh/token-saving/tokenless/measuring-savings.md +++ b/docs/user-guide/zh/token-saving/tokenless/measuring-savings.md @@ -173,6 +173,59 @@ tokenless stats summary \ 例如,Payload 压缩率为 60%,但工具 Payload 只占会话总 Token 的 20%,则总体估算收益约为 12%。这个结果仍不是提供商账单保证值。 +## 压缩率的适用场景 + +Tokenless 的压缩率取决于 Payload 中可精简成分的多少,不同场景差异很大。使用下文的标准测试负载测得的参考值(测量于 commit `2e7d69f1`,升级后请在本地重新运行确认): + +| 场景 | 参考节省率(估算 Token) | 说明 | +|------|--------------------------|------| +| 结构化 JSON 响应(统一记录 + 冗余字段),单路响应压缩 | 约 66% | 黑名单字段、空值被移除,超长字符串/数组被截断 | +| 函数调用 Schema(长描述),单路 Schema 压缩 | 约 47% | 描述截断,移除 `title`/`examples` 与代码块 | +| 混合负载(响应 + Schema):仅响应压缩 | 约 62% | 响应是主要收益来源 | +| 混合负载:Schema + 响应压缩叠加 | 约 65% | 两类 Payload 同时精简 | +| 混合负载:全栈叠加(Schema + 响应 + TOON,部署门控) | 约 65% | 部署路径的 `compress-toon` 尺寸保护会在 TOON 不能减少估算 Token 时保留原输入;本负载下响应/Schema 压缩后再做 TOON 反而膨胀,因此部署后的节省率与上一行叠加结果相同。基准报告中未加门控的 `full_stack` 组合测得约 63%,但部署路径不会输出该结果 | +| 仅 TOON 编码 | 约 16% | 表格化、规整的 JSON 才有明显收益 | + +按场景归纳: + +- **收益高**:工具返回大量统一结构的记录(列表、表格、搜索结果),或携带 `debug`/`trace`/`logs` 等冗余字段,或 Schema 描述冗长。 +- **收益中等**:Shell 输出中超过 Layer 2 阈值(字符串 65,536 字符、数组 128 项、深度 8)的部分会被截断;未超阈值的输出基本保持原样。 +- **收益接近零**:短于最小触发长度的响应(共享 Adapter 200 字符、Codex 500 字符);本身已经紧凑、无冗余的 JSON;任何压缩后不比原文更小的输入(尺寸保护会保留原文)。 +- **不参与压缩**:内容读取类工具(Read/Glob/Grep 等)的输出、非 JSON 文本、带 YAML frontmatter 的 Skill 文本。触发条件详见[用户手册 · 压缩的触发条件与阈值](user-manual.md#压缩的触发条件与阈值)。 + +实际会话收益还要乘以工具 Payload 在会话总 Token 中的占比,见上文[正确解释节省率](#正确解释节省率)。 + +## 标准测试负载 + +仓库内置了确定性的标准测试负载,位于 `src/tokenless/benchmark/l1-compressor`(独立 Cargo workspace;仅支持 Linux,不支持 macOS/Windows)。负载由 `python/gen_fixtures.py` 生成,不含随机数,字节级可复现,并已提交在仓库中: + +| 负载文件 | 内容 | +|----------|------| +| `fixtures/records.json` | 1,000 条统一结构记录 | +| `fixtures/tool_response.json` | 典型工具响应(外层信封 + 60 条记录 + `trace`/`logs` 冗余字段) | +| `fixtures/schema_search.json` | 典型函数调用 Schema | + +快速运行压缩率报告(需要先构建,输出包括单路压缩率、各压缩组合的叠加结果与成本估算): + +```bash +cd src/tokenless/benchmark/l1-compressor +cargo run --release --bin compression_rate # 人类可读报告 +cargo run --release --bin compression_rate -- --json # 机器可读 JSON;二进制参数必须放在 `--` 之后 +``` + +运行完整质量/对抗测试 + 压缩率报告(跳过 criterion 性能基准,耗时数分钟): + +```bash +cd src/tokenless/benchmark/l1-compressor +./run-benchmarks.sh --quick +``` + +使用标准负载时请注意: + +- Token 数使用字节/4 启发式估算,适合版本间相对比较,绝对值不代表真实计费 Token。 +- 压缩率随版本演进可能变化,引用数字时请注明对应的 commit 或版本号。 +- 标准负载用于横向对比,不代表你的业务数据;评估真实收益仍应使用[双跑对比](#用-dry-run-做双跑对比)在自己的工作负载上测量。 + ## AgentSight 本地展示 AgentSight 的 Token savings 页面可以只读聚合 `~/.tokenless/stats.db`。两者由同一用户运行,且 AgentSight 能访问该数据库时,不需要通过 SLS 才能看到本地 Tokenless 统计。 diff --git a/docs/user-guide/zh/token-saving/tokenless/user-manual.md b/docs/user-guide/zh/token-saving/tokenless/user-manual.md index 172a557c7..e3288870a 100644 --- a/docs/user-guide/zh/token-saving/tokenless/user-manual.md +++ b/docs/user-guide/zh/token-saving/tokenless/user-manual.md @@ -112,6 +112,34 @@ CLI-only 用法不需要 Adapter。 anolisa adapter disable tokenless ``` +### 压缩的触发条件与阈值 + +Adapter 不会压缩每一次工具结果。以响应压缩为例,只有以下条件全部满足,才会实际产出压缩内容: + +1. 压缩未被停用。`compression_enabled=false` 或 `TOKENLESS_COMPRESSION_ENABLED=0` 时进入 dry-run,仍计算统计但返回原文(见上一节)。 +2. 工具不属于内容读取类。Read/Glob/Grep/LSP/NotebookRead 及别名会跳过响应压缩,保留完整内容。 +3. 响应长度达到最小阈值。共享响应 Hook、OpenClaw 和 Hermes 跳过短于 200 字符的响应;Codex 跳过短于 500 字符的响应。长度按字符数而非字节数计算。 +4. 内容是合法 JSON。响应压缩只处理 JSON;纯文本直接透传。共享路径还会跳过带 YAML frontmatter、形似 Skill 的文本。 +5. 压缩结果严格小于原文。响应压缩和 TOON 编码都没有让内容变小时,保留原文。 + +通过上述检查后,截断强度由工具类别决定。分类和阈值定义在 Adapter 目录下的 `tool_categories.json`(各 Adapter 共享的单一事实来源);文件缺失或无效时使用内置的安全回退值: + +| 类别 | 代表工具 | 字符串截断阈值 | 数组保留上限 | 最大嵌套深度 | +|------|----------|----------------|--------------|--------------| +| 内容读取类 | Read、Glob、Grep、LSP、NotebookRead 及别名 | 跳过压缩 | — | — | +| Shell/exec | Bash、Shell、exec、terminal 等 | 65,536 字符 | 128 项 | 8 | +| 其他结构化工具 | 未列入前两类的工具 | 1,048,576 字符 | 65,536 项 | 32 | + +阈值含义:字符串超过阈值时从阈值处截断(启用 Stash 时可取回原文);数组超过上限时只保留前面的项,尾部被截断(启用 Stash 时同样可取回);嵌套超过深度上限的子树折叠为截断标记。 + +几点路径差异: + +- 独立运行 `tokenless compress-response` 时使用 CLI 自身默认值(4,096 字符 / 32 项 / 深度 8),可用 `--truncate-strings-at`、`--truncate-arrays-at`、`--max-depth` 覆盖,详见 [CLI 参考](cli-reference.md)。 +- Codex 的跳过列表在共享分类之外还包含任务管理类工具(TodoWrite、Task、TaskStatus),且只有响应至少 4,000 字符时才把压缩内容附加到上下文。 +- OpenClaw Plugin 可以用 `skip_tools`、`shell_tools` 覆盖工具分类;阈值本身仍来自 `tool_categories.json`,选项说明见[配置与数据隐私](configuration-and-privacy.md)。 +- TOON 编码是独立的触发判断:只有编码结果比当前内容更小时才会采用。 +- AgentScope 框架集成不使用上面的 Adapter 阈值,而是按 `conservative` / `balanced` / `aggressive` 模式选择阈值,见[框架集成](framework-integration.md)。 + ### 可逆压缩是有条件的 启用压缩时,响应和 Schema 截断默认会把被移除的 Payload 写入 `~/.tokenless/stash.db`,并在输出中加入: @@ -165,6 +193,7 @@ Stash 并不能让所有压缩都可逆。被移除的 `debug`/`trace` 字段、 | 从源码构建进程内 Python Runtime | [本页 · 从源码构建 Python Runtime](#从源码构建-python-runtime) | | 接入 Agent 产品或集成 AgentScope | [Agent 与框架集成](framework-integration.md) | | 手动压缩、取回或运行 MCP | [CLI 参考](cli-reference.md) | +| 了解压缩何时触发、阈值多大 | [本页 · 压缩的触发条件与阈值](#压缩的触发条件与阈值) | | 查看节省或内容变化、做双跑对比 | [效果度量](measuring-savings.md) | | 修改配置或了解本地数据 | [配置与数据隐私](configuration-and-privacy.md) | | 解决无统计、Adapter 或 Stash 问题 | [故障排查](troubleshooting.md) |