Skip to content

Use dedicated cache for HTTP API route#9318

Merged
mergify[bot] merged 7 commits into
sigp:unstablefrom
dapplion:dedicated-historical-committee-cache
May 19, 2026
Merged

Use dedicated cache for HTTP API route#9318
mergify[bot] merged 7 commits into
sigp:unstablefrom
dapplion:dedicated-historical-committee-cache

Conversation

@dapplion
Copy link
Copy Markdown
Collaborator

Issue Addressed

BUT the http API route wants to use the committee cache and insert historical committees (i.e. given state at epoch 1000, compute and store the committee for epoch 900).

If we want a single cache to serve both use cases we need to:

  • Have entries in the committee cache that have no PTC: Makes reading PTCs from the cache not deterministic
  • Compute historical PTC: A bunch of complicated code that's useless

Instead we can add a separate cache for the API, very simple one, that caches committees only. And have the one in the beacon chain compute and cache PTCs always.

Performance impact

Slightly additional memory cost for users of the beacon/states/committees route. Caching is almost equivalent, except for queries of recent committees that may already exist in the beacon chain's committee cache.

AI disclousure

This PR was written by hand 90%. Claude fixed some warp type issues

@dapplion dapplion added the ready-for-review The code is ready for review label May 19, 2026
@mergify
Copy link
Copy Markdown

mergify Bot commented May 19, 2026

This pull request has merge conflicts. Could you please resolve them @dapplion? 🙏

@mergify mergify Bot added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels May 19, 2026
@dapplion dapplion force-pushed the dedicated-historical-committee-cache branch from fa32d82 to d54e2fa Compare May 19, 2026 01:51
Comment thread beacon_node/http_api/src/caches.rs Outdated
Comment thread beacon_node/http_api/src/beacon/states.rs
Comment thread beacon_node/http_api/src/beacon/states.rs
Comment thread beacon_node/http_api/src/caches.rs Outdated
Comment thread beacon_node/http_api/src/beacon/states.rs
@michaelsproul michaelsproul added code-quality optimization Something to make Lighthouse run more efficiently. HTTP-API labels May 19, 2026
Comment thread beacon_node/client/src/builder.rs Outdated
sse_logging_components: runtime_context.sse_logging_components.clone(),
historical_committee_cache: Arc::new(http_api::HistoricalCommitteeCache::new(
NonZeroUsize::new(self.http_api_config.historical_committee_cache_size)
.unwrap_or(NonZeroUsize::new(1).expect("1 > 0")),
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to not error if the user passes --shuffling-cache-size=0 as this is today a valid value. Instead default to 1. In the future we can make 0 mean disabled.

Comment thread beacon_node/client/src/builder.rs Outdated
Comment thread beacon_node/http_api/src/test_utils.rs
Copy link
Copy Markdown
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels May 19, 2026
@mergify mergify Bot added the queued label May 19, 2026
@mergify
Copy link
Copy Markdown

mergify Bot commented May 19, 2026

Merge Queue Status

This pull request spent 29 minutes 9 seconds in the queue, including 27 minutes 37 seconds running CI.

Required conditions to merge

mergify Bot added a commit that referenced this pull request May 19, 2026
@mergify mergify Bot merged commit 398efc3 into sigp:unstable May 19, 2026
38 checks passed
@mergify mergify Bot removed the queued label May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-quality HTTP-API optimization Something to make Lighthouse run more efficiently. ready-for-merge This PR is ready to merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants