Skip to content

Signatory: serve keys from memory - #2270

Merged
thesimplekid merged 1 commit into
cashubtc:mainfrom
crodas:feat/signatory-db-persistence-background-load
Jul 30, 2026
Merged

Signatory: serve keys from memory#2270
thesimplekid merged 1 commit into
cashubtc:mainfrom
crodas:feat/signatory-db-persistence-background-load

Conversation

@crodas

@crodas crodas commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Description

The signatory round-tripped the database on the hot path: rotate_keyset re-read the active keyset to compute the next derivation index and then reloaded the whole keyset set to refresh memory. The database's role in steady-state operation was never pinned down.

Treat the database as a persistence layer only. It is read once on boot and written only inside the rotate_keyset transaction; every other operation is served from and mutates in-memory state. Rotation reads the next index from memory and updates the in-memory maps directly after
committing, mirroring what a fresh boot would compute, so it never reads the database back.

Load strictly at boot. new runs the load once and propagates any error,so a failed load fails construction rather than leaving a signatorywithout keys; both the embedded mint and the standalone signatory server want a bad database to surface at startup. The in-memory state lives in DbSignatory's own fields, with no wrapper indirection.

Serialize rotations. rotate_keyset holds a mutex across the whole operation, so two concurrent rotations of the same unit cannot read the same derivation index and derive duplicate keysets. This guards the
standalone gRPC server, which calls rotate_keyset directly rather than through the embedded single-runner service.

Relax the embedded mint boot: a signatory reporting no active keyset is a warning rather than a hard error, so an unconfigured mint still starts and its endpoints return keyset errors until it is configured.

This is groundwork to simplify #2253 (auto-rotate keysets on an age interval): with memory as the source of truth and rotation updating it in place, a periodic rotator only has to call rotate_keyset.

Document the persistence model, the strict boot, and rotation serialization in ADR-0003.


Notes to the reviewers


Suggested CHANGELOG Updates

CHANGED

ADDED

REMOVED

FIXED


Checklist

  • I followed the code style guidelines
  • I ran just quick-check before committing
  • If the Wallet API was modified (added/removed/changed), I have reflected those changes in the FFI bindings (crates/cdk-ffi)

@crodas
crodas requested a review from thesimplekid July 27, 2026 21:26
@crodas crodas self-assigned this Jul 27, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in CDK Jul 27, 2026
@crodas

crodas commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

@cdk-bot review

@crodas
crodas force-pushed the feat/signatory-db-persistence-background-load branch from ed0e8c7 to 626e49e Compare July 27, 2026 21:31
@crodas crodas changed the title Signatory: sserve keys from memory, load from the DB resiliently Signatory: serve keys from memory, load from the DB resiliently Jul 27, 2026

@cdk-bot cdk-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified findings approved for disclosure:

  • Consistent lock ordering is needed in rotate_keyset (medium) - A concurrent manual/management key rotation can permanently deadlock the signatory's in-memory keyset locks; while stuck, operations that need keysets.read() such as signing and proof verification can also stall until restart.
    Additional locations included in summary:
    • crates/cdk-signatory/src/db_signatory.rs:379

Comment thread crates/cdk-signatory/src/db_signatory.rs Outdated
@crodas
crodas force-pushed the feat/signatory-db-persistence-background-load branch from 626e49e to 9e6a282 Compare July 27, 2026 21:51
@crodas

crodas commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

@cdk-bot review

@crodas
crodas force-pushed the feat/signatory-db-persistence-background-load branch from 9e6a282 to b0e1be8 Compare July 27, 2026 21:58

@cdk-bot cdk-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified findings approved for disclosure:

  • Embedded mint resilient boot fails before the retry path can run (high) - The embedded mint's advertised resilient boot path does not work when the initial keyset database load fails: build_with_seed fails during construction before the background retry can recover and publish keysets.
    Unanchored locations included in summary:
    • crates/cdk-signatory/src/db_signatory.rs:306
    • crates/cdk/src/mint/builder.rs:589

Comment thread crates/cdk-signatory/src/db_signatory.rs Outdated
@crodas

crodas commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

@cdk-bot review

Comment thread docs/adr/0003-signatory-database-persistence-only.md Outdated
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.42857% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.19%. Comparing base (f417136) to head (783efcb).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
crates/cdk/src/mint/mod.rs 62.50% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2270      +/-   ##
==========================================
- Coverage   74.20%   74.19%   -0.01%     
==========================================
  Files         363      363              
  Lines       85179    85241      +62     
==========================================
+ Hits        63206    63248      +42     
- Misses      21973    21993      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@crodas
crodas force-pushed the feat/signatory-db-persistence-background-load branch from 3cbc305 to c6a7392 Compare July 28, 2026 02:23
@crodas

crodas commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

@cdk-bot review

@crodas crodas changed the title Signatory: serve keys from memory, load from the DB resiliently Signatory: serve keys from memory Jul 28, 2026

@cdk-bot cdk-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified findings approved for disclosure:

  • Comment still describes dropped background keyset loading (low) - Maintainers may be led to debug or code for a transient background-loading state that this PR explicitly removed; it also contradicts the ADR shipped in the same PR.
  • Remove leftover unused KeysetsNotLoaded error variant (low) - This adds a public error variant and saga failure classification that are never exercised under the PR's strict-boot model, creating misleading API surface and future maintenance risk.
    Additional locations included in summary:
    • crates/cdk-common/src/error.rs:748

Comment thread crates/cdk/src/mint/mod.rs Outdated
Comment thread crates/cdk-common/src/error.rs Outdated
@crodas
crodas force-pushed the feat/signatory-db-persistence-background-load branch from c6a7392 to 214d40b Compare July 28, 2026 03:30
@crodas

crodas commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

@cdk-bot review

Comment thread crates/cdk-signatory/src/db_signatory.rs
Comment on lines +176 to +180
// The signatory may not report an active keyset yet (for example a
// remote signatory that is still connecting). Start anyway and let the
// keyset drain task install keysets if they arrive over the
// subscription; endpoints return keyset errors until then.
if active_keys == 0 {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think I prefer a fail fast if the mint does not have keysets?

Comment thread crates/cdk-signatory/src/db_signatory.rs
Comment thread docs/adr/0003-signatory-database-persistence-only.md
@github-project-automation github-project-automation Bot moved this from Backlog to In progress in CDK Jul 28, 2026

@thesimplekid thesimplekid left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ill merge this now and move a few things to make sure we clean up in the follow up pr.

The signatory round-tripped the database on the hot path: rotate_keyset
re-read the active keyset to compute the next derivation index and then
reloaded the whole keyset set to refresh memory. The database's role in
steady-state operation was never pinned down.

Treat the database as a persistence layer only. It is read once on boot
and written only inside the rotate_keyset transaction; every other
operation is served from and mutates in-memory state. Rotation reads the
next index from memory and updates the in-memory maps directly after
committing, mirroring what a fresh boot would compute, so it never reads
the database back.

Load strictly at boot. new runs the load once and propagates any error,
so a failed load fails construction rather than leaving a signatory
without keys; both the embedded mint and the standalone signatory server
want a bad database to surface at startup. The in-memory state lives in
DbSignatory's own fields, with no wrapper indirection.

Serialize rotations. rotate_keyset holds a mutex across the whole
operation, so two concurrent rotations of the same unit cannot read the
same derivation index and derive duplicate keysets. This guards the
standalone gRPC server, which calls rotate_keyset directly rather than
through the embedded single-runner service.

Relax the embedded mint boot: a signatory reporting no active keyset is
a warning rather than a hard error, so an unconfigured mint still starts
and its endpoints return keyset errors until it is configured.

This is groundwork to simplify cashubtc#2253 (auto-rotate keysets on
an age interval): with memory as the source of truth and rotation
updating it in place, a periodic rotator only has to call rotate_keyset.

Document the persistence model, the strict boot, and rotation
serialization in ADR-0003.
@thesimplekid
thesimplekid force-pushed the feat/signatory-db-persistence-background-load branch from 214d40b to 783efcb Compare July 30, 2026 11:46
@thesimplekid
thesimplekid merged commit 783efcb into cashubtc:main Jul 30, 2026
2 of 4 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in CDK Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants