Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
7b67e37
feat(pyramid): add multi-layer root routing
jac0626 Aug 19, 2026
e6119ac
fix(pyramid): synchronize root routing state
jac0626 Aug 19, 2026
cf1875e
fix(pyramid): validate serialized route count
jac0626 Aug 19, 2026
897c740
fix(pyramid): build root routes from cache
jac0626 Aug 20, 2026
2a966b7
fix(pyramid): validate loads and bound reorder results
jac0626 Aug 20, 2026
804217a
test(autotune): update Pyramid memory expectations
jac0626 Aug 20, 2026
6063965
fix(pyramid): tighten routing validation and analyzer locking
jac0626 Aug 20, 2026
c4b0aec
refactor(pyramid): move routing overlays into index nodes
jac0626 Aug 20, 2026
2da53d7
fix(pyramid): allow unbuilt single-layer roots
jac0626 Aug 20, 2026
07ebef8
refactor(pyramid): unify routed graph construction
jac0626 Aug 21, 2026
7423a3e
test(searcher): use stable flatten factory
jac0626 Aug 21, 2026
c2178e6
perf: accelerate Pyramid and HGraph construction
jac0626 Aug 22, 2026
668d973
refactor: keep graph acceleration scoped to Pyramid
jac0626 Aug 23, 2026
bc31f9c
fix(pyramid): address root validation and search races
jac0626 Aug 23, 2026
735ad99
fix(pyramid): stabilize concurrent add and search
jac0626 Aug 23, 2026
8548c41
refactor(pyramid): simplify root build acceleration
jac0626 Aug 24, 2026
9f11575
fix(pyramid): preserve parallel encoding safety boundary
jac0626 Aug 24, 2026
ab2297d
refactor(pyramid): encapsulate routed entry selection
jac0626 Aug 24, 2026
8153548
refactor(pyramid): simplify routed build lifecycle
jac0626 Aug 24, 2026
a609324
chore: merge latest upstream main
jac0626 Aug 24, 2026
50faa70
test(pyramid): compare restored routed search
jac0626 Aug 24, 2026
14e016c
fix(pyramid): retain safe parallel encode boundary
jac0626 Aug 24, 2026
bd0e62b
refactor(pyramid): simplify routed build implementation
jac0626 Aug 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions docs/docs/en/src/indexes/pyramid.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,10 @@ Build-time parameters live under `index_param`.
| `base_file_path` / `precise_file_path` | string | — | Required for disk-backed storage such as `buffer_io`, `async_io`, `uring_io`, or `mmap_io`. |
| `store_raw_vector` | bool | `false` | Preserve an FP32 copy for `GetRawVectorByIds` and precise distance-by-id calculations. |
| `index_min_size` | int | `0` | Minimum sub-index size; smaller groups fall back to scan. |
| `root_graph_type` | string | `"single_layer"` | Root graph layout: `single_layer` preserves the original sparse bottom graph; `multi_layer` uses a preallocated dense Flat bottom graph with HGraph-style sparse routing layers and joint construction. `multi_layer` requires `graph_type: "nsw"` and a built level 0. |
| `support_duplicate` | bool | `false` | Allow duplicate ids. |
| `build_thread_count` | int | `1` | Threads used for parallel build. |
| `hierarchies` | array | `[]` | Named hierarchy definitions. Each element is either a string (inherits all top-level params) or an object with `name` and optional overrides (`max_degree`, `ef_construction`, `alpha`, `no_build_levels`, `index_min_size`). When present, multi-hierarchy mode is activated and each hierarchy maintains its own independent path tree. |
| `hierarchies` | array | `[]` | Named hierarchy definitions. Each element is either a string (inherits all top-level params) or an object with `name` and optional overrides (`max_degree`, `ef_construction`, `alpha`, `no_build_levels`, `index_min_size`, `root_graph_type`). When present, multi-hierarchy mode is activated and each hierarchy maintains its own independent path tree. |

### RaBitQ split configuration

Expand Down Expand Up @@ -160,7 +161,8 @@ Search-time parameters live under the `pyramid` sub-object:
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `ef_search` | int | `100` | Candidate list size for the leaf-level graph search. |
| `hops_limit` | int | unlimited | Hard cap on hops for root-graph KNN search; ignored when it is not greater than `ef_search`. |
| `factor` | float | unset | KNN main-graph reorder candidate multiplier. When set to `<= 1`, the main-graph budget is `max(ef_search, topk)` candidates; when greater than `1`, it is `min(max(ef_search, topk), floor(topk * factor))`. RaBitQ lower-bound safety candidates may be merged after this budget, matching HGraph; `reorder_candidate_count` reports the actual merged count. The value must be finite and positive. It has no effect on range search or when reorder is disabled. |
| `hops_limit` | int | unlimited | Per-graph KNN hop cap for the root bottom graph and every non-root graph; ignored when it is not greater than `ef_search`. Sparse root routing layers are never hop-limited. FLAT scans and range search are unaffected. |
| `subindex_ef_search` | int | `50` | Candidate list size used when traversing intermediate sub-graphs on the path. |
| `hierarchies` | string[] | `[]` | Select which hierarchy to search. Empty means use the default (unnamed) hierarchy. |
| `hierarchy_op` | string | `"single"` | How to combine results across hierarchies: `single` (search one hierarchy), `union`, or `intersection`. **Note:** `union` and `intersection` are not yet implemented — setting them will cause `KnnSearch`/`RangeSearch` to return an error. |
Expand Down Expand Up @@ -193,7 +195,15 @@ Add a `hierarchies` array inside `index_param`. Each element is either:
`{"name": "category", "max_degree": 64, "no_build_levels": [0]}`

Overridable per-hierarchy parameters: `max_degree`, `ef_construction`, `alpha`,
`no_build_levels`, `index_min_size`.
`no_build_levels`, `index_min_size`, `root_graph_type`.

`root_graph_type: "multi_layer"` changes only the selected hierarchy's root. It uses a preallocated
dense Flat bottom graph, while sparse routing graphs choose a better entry point before the bottom
search. Bulk Build and incremental Add jointly construct the route and bottom layers with the same
HGraph-style insertion protocol. This layout requires `graph_type: "nsw"`; combining `multi_layer`
with `odescent` is rejected during parameter validation. Bottom and routing edges use the precise
codes when an independent precise store exists, otherwise they use the base codes. Query traversal
continues to use the base codes, and final reordering uses the configured reorder source.

```json
{
Expand Down Expand Up @@ -308,6 +318,9 @@ faster.

Use [Index Analysis](../resources/analyze_index.md) to inspect Pyramid tree structure,
per-subindex quality, sampled base recall, and duplicate ratios reported by `GetStats()`.
For each hierarchy, `root_graphs` reports `root_graph_type`, `bottom_graph_storage_type`,
`bottom_graph_node_count`, `bottom_graph_size`, `route_graph_count`, `route_node_counts`, and
`route_graph_size`.
`AnalyzeIndexBySearch` also reports path-scoped query recall, distance, latency, and, when reorder
is enabled, quantization metrics. Its query dataset must carry the same default or named-hierarchy
paths required by `KnnSearch`; when paths are required or supplied for a batched dataset, provide
Expand Down
20 changes: 16 additions & 4 deletions docs/docs/zh/src/indexes/pyramid.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,10 @@ auto result = index->KnnSearch(
| `base_file_path` / `precise_file_path` | string | — | `buffer_io`、`async_io`、`uring_io`、`mmap_io` 等磁盘存储必须设置 |
| `store_raw_vector` | bool | `false` | 保留 FP32 原始向量,用于 `GetRawVectorByIds` 和精确的按 ID 距离计算 |
| `index_min_size` | int | `0` | 子索引的最小规模;小于该值的分区会退化为线性扫描 |
| `root_graph_type` | string | `"single_layer"` | 根图结构:`single_layer` 保留原有稀疏底图;`multi_layer` 使用预分配的稠密 Flat 底图、类似 HGraph 的稀疏路由层以及联合构图流程。`multi_layer` 要求 `graph_type: "nsw"` 且第 0 层参与构建。 |
| `support_duplicate` | bool | `false` | 是否允许重复 ID |
| `build_thread_count` | int | `1` | 构建阶段并发线程数 |
| `hierarchies` | array | `[]` | 命名层级定义。每个元素可以是字符串(继承全部顶层参数)或对象(含 `name` 及可选覆盖参数:`max_degree`、`ef_construction`、`alpha`、`no_build_levels`、`index_min_size`)。设置后激活多层级模式,每个层级维护独立的路径树。 |
| `hierarchies` | array | `[]` | 命名层级定义。每个元素可以是字符串(继承全部顶层参数)或对象(含 `name` 及可选覆盖参数:`max_degree`、`ef_construction`、`alpha`、`no_build_levels`、`index_min_size`、`root_graph_type`)。设置后激活多层级模式,每个层级维护独立的路径树。 |

### RaBitQ split 配置

Expand Down Expand Up @@ -154,7 +155,8 @@ Pyramid 使用 split code 的 code-code 距离完成增量 FLAT→GRAPH 晋升
| 参数 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `ef_search` | int | `100` | 叶子层子图检索的候选集大小 |
| `hops_limit` | int | 不限 | 根图 KNN 检索的最大跳数;不大于 `ef_search` 时忽略 |
| `factor` | float | 未设置 | KNN 主图重排候选倍率。值 `<= 1` 时主图候选预算为 `max(ef_search, topk)`;值大于 `1` 时为 `min(max(ef_search, topk), floor(topk * factor))`。与 HGraph 一致,RaBitQ lower-bound 安全候选可在该预算后额外并入,`reorder_candidate_count` 记录实际合并数量。参数必须为有限正数;范围检索或关闭重排时不生效。 |
| `hops_limit` | int | 不限 | 根节点底图及每个非根 GRAPH 的逐图 KNN 跳数上限;不大于 `ef_search` 时忽略。根节点的稀疏路由层不受限制,FLAT 扫描与范围检索不受影响。 |
| `subindex_ef_search` | int | `50` | 沿路径向下遍历中间子图时的候选集大小 |
| `hierarchies` | string[] | `[]` | 指定检索哪个层级。空数组表示使用默认(匿名)层级。 |
| `hierarchy_op` | string | `"single"` | 多层级结果合并方式:`single`(检索单个层级)、`union`、`intersection`。**注意:** `union` 和 `intersection` 尚未实现——设置后 `KnnSearch`/`RangeSearch` 会返回错误。 |
Expand Down Expand Up @@ -184,7 +186,14 @@ auto result = index->KnnSearch(
`{"name": "category", "max_degree": 64, "no_build_levels": [0]}`

可按层级覆盖的参数:`max_degree`、`ef_construction`、`alpha`、`no_build_levels`、
`index_min_size`。
`index_min_size`、`root_graph_type`。

`root_graph_type: "multi_layer"` 只改变所选层级的根节点:根节点使用预分配的稠密 Flat
底图,稀疏路由图先选择更好的入口点,再进入底图检索。批量 Build 与增量 Add 都使用类似
HGraph 的 route 与 bottom 联合插入流程。该结构要求 `graph_type: "nsw"`;参数校验会拒绝
`multi_layer` 与 `odescent` 的组合。存在独立 precise storage 时,底图和路由图的边统一使用
precise codes 构建,否则使用 base codes;查询遍历继续使用 base codes,最终精排使用配置的
reorder source。

```json
{
Expand Down Expand Up @@ -292,7 +301,10 @@ new_index->Deserialize(binary_set);
如果不需要按路径限定查询范围,[HGraph](hgraph.md) 更简洁,性能通常也更高。

可以通过[索引分析](../resources/analyze_index.md)检查 Pyramid 的树结构、子索引质量、
`GetStats()` 输出的 base 采样召回率和重复比例。`AnalyzeIndexBySearch` 还会输出按路径限定的
`GetStats()` 输出的 base 采样召回率和重复比例。每个 hierarchy 的 `root_graphs` 会报告
`root_graph_type`、`bottom_graph_storage_type`、`bottom_graph_node_count`、
`bottom_graph_size`、`route_graph_count`、`route_node_counts` 和 `route_graph_size`。
`AnalyzeIndexBySearch` 还会输出按路径限定的
query 召回率、距离、耗时,以及开启 reorder 时的量化指标。query 数据集必须包含与
`KnnSearch` 相同的默认或命名 hierarchy 路径;批量数据集在需要或提供路径时,应为每条 query
提供一条路径。`analyze_index` 工具当前无法从 dense query 文件加载 hierarchy 路径,因此
Expand Down
3 changes: 3 additions & 0 deletions include/vsag/constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ extern const char* const PYRAMID_PARAMETER_HIERARCHY_OP;
extern const char* const PYRAMID_NO_BUILD_LEVELS;
extern const char* const PYRAMID_HIERARCHIES;
extern const char* const PYRAMID_INDEX_MIN_SIZE;
extern const char* const PYRAMID_ROOT_GRAPH_TYPE;
extern const char* const PYRAMID_ROOT_GRAPH_TYPE_SINGLE_LAYER;
extern const char* const PYRAMID_ROOT_GRAPH_TYPE_MULTI_LAYER;

extern const char PART_SLASH;
extern const char PART_BAR;
Expand Down
Loading
Loading