Skip to content

[feat](eval_performance): support filtered HGraph workloads in AutoTune - #2736

Open
jac0626 wants to merge 1 commit into
antgroup:mainfrom
jac0626:codex/autotune-hgraph-filter
Open

[feat](eval_performance): support filtered HGraph workloads in AutoTune#2736
jac0626 wants to merge 1 commit into
antgroup:mainfrom
jac0626:codex/autotune-hgraph-filter

Conversation

@jac0626

@jac0626 jac0626 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Change Type

  • Bug fix
  • New feature
  • Improvement/Refactor
  • Documentation
  • CI/Build/Infra

Linked Issue

What Changed

  • Add per-query FilterPtr and invalid BitsetPtr workload inputs to typed HGraph TuneSearch and TuneIndex requests.
  • Route filtered workloads through the shared evaluation core while preserving HGraph's native bitset search path.
  • Validate filter shape, mutual exclusion, supported index scope, and unsupported extra-info filtering.
  • Add separate FilterPtr and BitsetPtr examples, focused regression tests, and English/Chinese user documentation.

Test Evidence

  • make fmt
  • make lint
  • make test
  • make cov, run tests, and collect coverage
  • Other (describe below)

Test details:

clang-format-15 --dry-run --Werror <all changed C++ files>
cmake --build build --target autotune_test eval_dataset_test \
  327_feature_autotune_existing_index \
  330_feature_autotune_existing_index_bitset -j$(nproc)

./build/tools/autotune/autotune_test \
  "AutoTune evaluates per-query filters and bitsets for HGraph"
  All tests passed (29 assertions in 1 test case)

./build/tools/eval/eval_dataset_test \
  "EvalDataset builds a query-only view for id recall"
  All tests passed (21 assertions in 1 test case)

./build/tools/eval/eval_dataset_test \
  "EvaluateSearch validates inputs and propagates search errors"
  All tests passed (23 assertions in 1 test case)

./build/examples/cpp/327_feature_autotune_existing_index
  recall_at_k: 1.0; first neighbor id: 1000

./build/examples/cpp/330_feature_autotune_existing_index_bitset
  recall_at_k: 1.0; first neighbor id: 1000

Compatibility Impact

  • API/ABI compatibility: Additive fields in the build-tree AutoTune API under tools/; no installed VSAG SDK API is changed.
  • Behavior changes: Typed HGraph tuning requests may now evaluate per-query filters. Filtered requests for other index types fail validation explicitly.

Performance and Concurrency Impact

  • Performance impact: The unfiltered evaluation path is unchanged. Filtered trials use the existing native HGraph filter/bitset search overloads.
  • Concurrency/thread-safety impact: The same per-query filters or bitsets may be read by concurrent search workers and therefore must remain immutable and thread-safe during tuning.

Documentation Impact

  • No docs update needed
  • Updated docs:
    • README.md
    • DEVELOPMENT.md
    • CONTRIBUTING.md
    • Other: docs/docs/{en,zh}/src/resources/autotune.md

Risk and Rollback

  • Risk level: medium
  • Rollback plan: Revert commit 92dfeaa4; unfiltered AutoTune behavior is otherwise unchanged.

Checklist

  • I have linked the relevant issue (required for kind/bug and kind/feature; see "Linked Issue" above)
  • I have added/updated tests for new behavior or bug fixes
  • I have considered API compatibility impact
  • I have updated docs if behavior/workflow changed
  • My commit messages follow project conventions (Conventional Commits, optional [skip ci] prefix)

Signed-off-by: jc543239 <jc543239@antgroup.com>
Assisted-by: Codex:gpt-5
Copilot AI lite review requested due to automatic review settings August 21, 2026 03:10
@vsag-bot

vsag-bot commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

/label status/waiting-for-review
/waiting-on reviewer
/request-review @jiaweizone
/request-review @wxyucs
/request-review @inabao
/request-review @LHT129

@jac0626 jac0626 added kind/feature Brand-new functionality or capabilities 引入全新的功能、新特性或新能力 version/1.0 labels Aug 21, 2026
@mergify

mergify Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 3 merge protections satisfied — ready to merge.

Show 3 satisfied protections

🟢 Require kind label

  • label~=^kind/

🟢 Require version label

  • label~=^version/

🟢 Require linked issue for feature/bug PRs

  • body~=(?im)(?:^|[\s\-\*])(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s+(?:#\d+|[\w.\-]+/[\w.\-]+#\d+|https?://github\.com/[\w.\-]+/[\w.\-]+/issues/\d+)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the eval_performance AutoTune typed API to support filtered HGraph workloads by accepting per-query FilterPtr or exclusion BitsetPtr inputs, routing them through the shared evaluation pipeline while keeping HGraph’s native filtered-search overloads, and documenting the new capability.

Changes:

  • Extend typed AutoTune workload/request plumbing to carry per-query FilterPtr / BitsetPtr into EvalDataset, validation, and evaluation.
  • Update search evaluation to apply per-query filters/bitsets, and restrict filtered workloads to HGraph (including rejecting use_extra_info_filter=true).
  • Add regression tests, new bitset example, and English/Chinese documentation updates.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tools/eval/eval_dataset.h Adds per-query filter/bitset storage + accessors and filtered-query accounting.
tools/eval/eval_dataset.cpp Validates filter/bitset inputs and wires them into dataset construction.
tools/eval/eval_dataset_test.cpp Adds coverage for per-query filters/bitsets and input validation.
tools/eval/case/search_eval_case.cpp Applies per-query filter/bitset during KNN evaluation via HGraph overloads.
tools/autotune/autotune.h Extends Workload with query_filters and query_invalid_bitsets.
tools/autotune/autotune.cpp Propagates filtered workload data into eval datasets; enforces HGraph-only for filtered cases; reports filtered count.
tools/autotune/autotune_internal.h Tracks filtered-workload presence in request context.
tools/autotune/autotune_candidate.cpp Rejects extra-info filtering when tuning filtered workloads.
tools/autotune/autotune_test.cpp Adds end-to-end tuning tests for per-query filters/bitsets and validation failures.
examples/cpp/CMakeLists.txt Registers a new bitset-based AutoTune example target.
examples/cpp/327_feature_autotune_existing_index.cpp Extends existing example to demonstrate per-query FilterPtr tuning and usage.
examples/cpp/330_feature_autotune_existing_index_bitset.cpp New example demonstrating per-query exclusion bitsets in AutoTune + final search.
docs/docs/zh/src/resources/autotune.md Documents filtered typed HGraph workloads and links both examples (FilterPtr + bitset).
docs/docs/en/src/resources/autotune.md Documents filtered typed HGraph workloads and links both examples (FilterPtr + bitset).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/eval/eval_dataset.h
Comment thread tools/autotune/autotune_internal.h
@jac0626

jac0626 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

/retest

@jac0626
jac0626 marked this pull request as ready for review August 21, 2026 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Brand-new functionality or capabilities 引入全新的功能、新特性或新能力 module/docs module/example module/tools size/XL version/1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat](eval_performance): support filtered HGraph workloads in AutoTune

4 participants