Skip to content

lore-proto, lore-server: add RepositoryCount RPC - #178

Open
lorelab-graham wants to merge 1 commit into
EpicGames:mainfrom
lorelab-graham:repository_count
Open

lore-proto, lore-server: add RepositoryCount RPC#178
lorelab-graham wants to merge 1 commit into
EpicGames:mainfrom
lorelab-graham:repository_count

Conversation

@lorelab-graham

Copy link
Copy Markdown

Summary

Adds RepositoryService.RepositoryCount(RepositoryCountRequest) -> RepositoryCountResponse for callers that want a repository count without paying RepositoryList's per-entry cost.
RepositoryList streams a full Repository proto per entry, which is orders of magnitude more expensive than the count itself.

The new RPC mirrors RepositoryList's filter surface (optional string creator).

  • Unfiltered path returns candidate_ids.len() directly and skips every per-repo metadata read, deserialization, and proto encoding that RepositoryList performs
  • Filtered path still walks metadata (the creator filter is applied after load, same as RepositoryList) and uses the same JoinSet fan-out to keep tail latency down
  • list_candidate_ids promoted to pub(super) so both handlers share the auth-service-vs-list_local split
  • Not added to ForwardedRepositoryService; cross-server count aggregation is a separate concern

Test Plan

  • Unit tests in repository_count.rs: empty store, N repos with no filter, N repos with creator filter matching M, filter matching none
  • Proto shape test in v1_repository.rs destructures the new request and response
  • cargo test --workspace passes (no regressions)
  • Exercised against live servers. Counts match RepositoryList

@github-actions github-actions Bot added area:server Server, provider integrations, telemetry area:core Core library and its interfaces (lib, C API); revision, storage, transport, protocol internals labels Aug 27, 2026
## Summary

Adds RepositoryService.RepositoryCount(RepositoryCountRequest) -> RepositoryCountResponse for callers that want a repository count without paying RepositoryList's per-entry cost.
RepositoryList streams a full Repository proto per entry, which is orders of magnitude more expensive than the count itself.

The new RPC mirrors RepositoryList's filter surface (optional string creator).

- Unfiltered path returns candidate_ids.len() directly and skips every per-repo metadata read, deserialization, and proto encoding that RepositoryList performs
- Filtered path still walks metadata (the creator filter is applied after load, same as RepositoryList) and uses the same JoinSet fan-out to keep tail latency down
- list_candidate_ids promoted to pub(super) so both handlers share the auth-service-vs-list_local split
- Not added to ForwardedRepositoryService; cross-server count aggregation is a separate concern

## Test Plan

- Unit tests in repository_count.rs: empty store, N repos with no filter, N repos with creator filter matching M, filter matching none
- Proto shape test in v1_repository.rs destructures the new request and response
- cargo test --workspace passes (no regressions)
- Exercised against live servers. Counts match RepositoryList

Signed-off-by: Graham Plumb <graham@lorelab.io>
@mjansson

mjansson commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

What would the use case be? Where is a count needed, but the metadata not?

@lorelab-graham

Copy link
Copy Markdown
Author

We have a tabbed (web) user interface when first logging in. Alongside user profile, we have a repository tab and a badge for the count. Since there's no guarantee a user will even click on that tab, we didn't want to pull all the repo data just to count it.

In the fullness of time, this could be stored separately in a database and webhooks used to catch events that influence the count, but in the short-term, it made more sense to get this from the server directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Core library and its interfaces (lib, C API); revision, storage, transport, protocol internals area:server Server, provider integrations, telemetry

Development

Successfully merging this pull request may close these issues.

2 participants