feat(discovery-core): add paginated last-note-index search with bisection

[m-kus]

Improved Note Index Discovery with Paginated Last-Note-Index Search

TL;DR

Refactored note index discovery to support paginated last-note-index search for outgoing channels, with improved exponential probing and bisection algorithms.

What changed?

• Renamed exponential_ascend to exponential_probe and made it more flexible with configurable max probe offset
• Added find_last_note_index_paginated function that combines exponential probing and binary search to efficiently find the last note index
• Implemented bisect_boundary helper for binary search between known bounds
• Made MAX_PROBE_OFFSET configurable via DEFAULT_MAX_PROBE_OFFSET constant
• Added comprehensive tests for the new functionality
• Improved documentation to clarify the purpose and behavior of each function

---

This change is 

[m-kus]

• feat(e2e): starknet integration demo explorer #467 : 3 dependent PRs (#480 , #502 , #512 )
• feat(discovery-service): add fallback for chain state, e2e test in the starknet integration environment #466
• feat(e2e): add indexer e2e test suite with sdk devnet helpers #486
• feat(sdk): add indexer discovery provider #485
• feat(discovery-service): add preflight check endpoint #484
• feat(discovery-service): add outgoing state sync endpoint #483
• feat(discovery-service): add incoming sync endpoint with validation and error handling #482
• feat(discovery-service): centralize configuration into config module #481
• feat(discovery-core): add transfer preflight check #474
• feat(discovery-core): add outgoing state sync orchestrator #473
• feat(discovery-core): add incoming state sync orchestrator #472
• feat(discovery-core): add paginated last-note-index search with bisection #471 👈 (View in Graphite)
• feat(discovery-core): add paginated outgoing channel discovery #470
• refactor: propagate compliance-to-auditor rename and fee params to sdk and rust #505
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Yoni-Starkware]

@Yoni-Starkware reviewed 1 file and all commit messages, and made 6 comments.
Reviewable status: 1 of 2 files reviewed, 6 unresolved discussions (waiting on m-kus).

---

crates/discovery-core/src/discovery/last_note_index.rs line 48 at r1 (raw file):

///
/// Returns `(last_index, has_more)`. When `has_more` is `false`, the search is complete.
pub async fn find_last_note_index_paginated<S: IViews>(

Can you point me to the PR that uses this function?
(I don't see where you propagate the cache)

---

crates/discovery-core/src/discovery/last_note_index.rs line 57 at r1 (raw file):

    // Phase 1: Ascending (find bounds)
    if cursor.max_note_index.is_none() {
        let start = cursor.last_note_index.map_or(0, |lo| lo + 1);

Suggestion:

     let start = cursor.start_index();

---

crates/discovery-core/src/discovery/last_note_index.rs line 76 at r1 (raw file):

        if cursor.last_note_index.is_none() {
            return Ok((None, false));
        }

probe_complete already considers result.last_found_index.is_some()

Suggestion:

        // Nothing to bisect.
        if cursor.max_note_index.is_none() || cursor.last_note_index.is_none() {
            let has_more = !result.probe_complete
            return Ok((cursor.last_note_index, has_more));
        }

---

crates/discovery-core/src/discovery/last_note_index.rs line 80 at r1 (raw file):

    // Phase 2: Bisection (narrow down to exact boundary, sequential)
    let probe = |idx: u64| {

Please document the return value, also in the bisect_boundary docstring

Code quote:

    // Phase 2: Bisection (narrow down to exact boundary, sequential)
    let probe = |idx: u64| {

---

crates/discovery-core/src/discovery/last_note_index.rs line 113 at r1 (raw file):

        (Some(lo), Some(hi)) => (lo, hi),
        _ => return Ok(true), // Nothing to bisect
    };

The _ should be unreachable, please enforce that

Code quote:

    let (mut lo, mut hi) = match (cursor.last_note_index, cursor.max_note_index) {
        (Some(lo), Some(hi)) => (lo, hi),
        _ => return Ok(true), // Nothing to bisect
    };

---

crates/discovery-core/src/discovery/last_note_index.rs line 174 at r1 (raw file):

                .map(|exp| (1u64 << exp) - 1)
                .take_while(|&off| off <= range && off <= offset_cap),
        )

Please revert to the previous terminology (upper_bound -> max_index, off -> offset, etc)

Code quote:

    let offset_cap = max_probe_offset.unwrap_or(u64::MAX);
    let upper_bound = match prior_max_index {
        None => start_index.saturating_add(offset_cap),
        Some(m) => m.saturating_mul(2).max(start_index),
    };
    let range = upper_bound.saturating_sub(start_index);

    // Offsets: [0, 1, 3, 7, 15, ..., 2^k - 1] where 2^k - 1 <= min(range, offset_cap).
    let offsets: Vec<u64> = std::iter::once(0)
        .chain(
            (1..64)
                .map(|exp| (1u64 << exp) - 1)
                .take_while(|&off| off <= range && off <= offset_cap),
        )

[m-kus]

@m-kus made 1 comment and resolved 5 discussions.
Reviewable status: 1 of 2 files reviewed, 1 unresolved discussion (waiting on Yoni-Starkware).

---

crates/discovery-core/src/discovery/last_note_index.rs line 48 at r1 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

Can you point me to the PR that uses this function?
(I don't see where you propagate the cache)

probe cache is used in notes discovery; find_last_note_index_paginated will be used in outgoing state sync (upcoming PR #483)

[Yoni-Starkware]

@Yoni-Starkware reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on m-kus).

[Yoni-Starkware]

@Yoni-Starkware resolved 1 discussion.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on m-kus).