Skip to content

feat: history pruner service#3596

Merged
rodrodros merged 20 commits intomainfrom
feat/history-pruner
May 7, 2026
Merged

feat: history pruner service#3596
rodrodros merged 20 commits intomainfrom
feat/history-pruner

Conversation

@EgeCaner
Copy link
Copy Markdown
Contributor

@EgeCaner EgeCaner commented May 2, 2026

Summary

Adds a pruner service that bounds on-disk storage by deleting block data and state history below a configurable retention window. With --retained-blocks=N, the node keeps blocks in (l1_head - N, l2_head] and prunes everything below.

  • RPC stays fully functional inside the retention window. Any block in (l1_head - N, l2_head] answers normally (events, traces, state-at-block, etc.). Below the floor: event_filter surfaces a typed ErrBlockPruned; statebackend returns db.ErrKeyNotFound (i.e. block-not-found), same as if the block never existed.
  • Floor is anchored on the L1-verified head, never on the local L2 head. Pruned blocks are reorg-safe by construction — anything at or below the L1 head cannot be reorged out. New L2 heads only act as triggers; the floor itself is always recomputed from L1.
  • Retention can be changed across restarts. Lowering --retained-blocks shrinks the window on the next sweep. Raising it grows the window gradually — the pruner pauses until the L1 head advances enough to reach the new floor; nothing is "un-pruned".
  • Disabled by default. --retained-blocks=0 keeps existing behaviour.
  • Irreversible. Data deleted under a small window cannot be recovered without re-syncing.

Open for discussion

starknet_getEvents with an unset from_block currently errors with ErrBlockPruned instead of silently clamping to the oldest retained block. Strict over silent — an unbounded query against a pruned node has clearly lost data, and we'd rather surface that than return a partial result the caller doesn't know is partial. Happy to flip to clamp-to-oldest if reviewers prefer.

Details

See the package doc on pruner/pruner.go and the contract on PruneUpto in pruner/accessors.go for the exact deletion set, the two intentional carve-outs (BlockHashLag headers and the parent-hash mapping), and resume semantics on cancellation.

Touches outside pruner/:

  • blockchain/event_filter.go, blockchain/statebackend/* — gate reads on retention via pruner.RequireRetained / pruner.HeaderBy*IfStateRetained.
  • core/block.goBlockHashLag and BlockHashStorageContract moved here from sync so the pruner carve-out and RPC getRevealedBlockHash share one source of truth.
  • node/node.go, cmd/juno/juno.go--retained-blocks flag and service wiring (sequencer and sync paths).
  • node/metrics.go — Prometheus listener for prune events.

@EgeCaner EgeCaner requested a review from MaksymMalicki May 2, 2026 00:44
@codecov
Copy link
Copy Markdown

codecov Bot commented May 2, 2026

Codecov Report

❌ Patch coverage is 72.72727% with 147 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.93%. Comparing base (0b15c38) to head (6c5f239).

Files with missing lines Patch % Lines
pruner/accessors.go 54.16% 23 Missing and 21 partials ⚠️
node/metrics.go 0.00% 37 Missing ⚠️
node/node.go 0.00% 25 Missing and 1 partial ⚠️
pruner/pruner.go 81.01% 10 Missing and 5 partials ⚠️
pruner/retention.go 75.00% 7 Missing and 3 partials ⚠️
pruner/event_listener.go 50.00% 3 Missing ⚠️
pruner/testutils/helpers.go 98.64% 0 Missing and 3 partials ⚠️
blockchain/event_filter.go 77.77% 1 Missing and 1 partial ⚠️
blockchain/statebackend/statebackend.go 66.66% 1 Missing and 1 partial ⚠️
rpc/v10/events.go 0.00% 1 Missing ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3596      +/-   ##
==========================================
+ Coverage   75.80%   75.93%   +0.12%     
==========================================
  Files         381      386       +5     
  Lines       34377    34818     +441     
==========================================
+ Hits        26059    26438     +379     
- Misses       6492     6516      +24     
- Partials     1826     1864      +38     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rodrodros
Copy link
Copy Markdown
Contributor

anything at or below the L1 head cannot be reorged out

What happens if there is a re-org on L1?

@EgeCaner EgeCaner requested a review from rodrodros May 4, 2026 10:46
@EgeCaner EgeCaner force-pushed the feat/history-pruner branch from 15a5437 to 22030a1 Compare May 5, 2026 12:51
Copy link
Copy Markdown
Contributor

@rodrodros rodrodros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looking good so far, some initial comments

Comment thread blockchain/event_filter.go
Comment thread node/node.go Outdated
Comment thread cmd/juno/juno.go
Comment thread pruner/testutils/helpers.go Outdated
Comment thread core/block.go
Comment thread pruner/accessors.go Outdated
Copy link
Copy Markdown
Contributor

@rodrodros rodrodros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two test files should be correctly set as pruner_test

Comment thread pruner/pruner_test.go Outdated
Comment thread pruner/accessors_test.go Outdated
@EgeCaner EgeCaner force-pushed the feat/history-pruner branch from 4c820cc to b899d50 Compare May 7, 2026 17:06
@rodrodros rodrodros merged commit 9fce7fd into main May 7, 2026
25 checks passed
@rodrodros rodrodros deleted the feat/history-pruner branch May 7, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants