fix(consensus/XDPoS,core,eth): repair missing V2 gap snapshots at startup - #2507
fix(consensus/XDPoS,core,eth): repair missing V2 gap snapshots at startup#2507gzliudan wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR adds a startup-time repair step for XDPoS v2 “gap block” snapshots to recover from a crash window where the chain head markers are persisted but the corresponding V2 snapshot key is missing, which can otherwise break voting/mining (and stall syncing on epoch transitions).
Changes:
- Invoke
EngineV2.RepairGapSnapshotsduringethbackend startup after engine initialization. - Add a DB existence probe (
rawdb.HasXdposV2Snapshot) and implement snapshot rebuilding from historical state (BuildSnapshotFromState) with logic to consider the last two relevant gap blocks. - Add focused unit tests covering snapshot derivation and the repair candidate selection/behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| eth/backend.go | Calls the new startup repair routine after consensus engine initialization. |
| core/rawdb/accessors_xdc.go | Adds HasXdposV2Snapshot to probe snapshot existence without loading/decoding it. |
| core/blockchain_reader.go | Declares BlockChain implements engine_v2.GapStateReader (supports StateAt). |
| consensus/XDPoS/engines/engine_v2/snapshot.go | Implements snapshot derivation from state and startup repair of missing gap snapshots. |
| consensus/XDPoS/engines/engine_v2/snapshot_test.go | Adds tests for deterministic snapshot building and repair behavior across candidate gap blocks. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
a750bf7 to
b7d2e26
Compare
1b6ab85 to
a06b0bc
Compare
c7d7679 to
b6451ef
Compare
…rtup A node can lose its persisted V2 gap snapshot when the process exits between writeHeadBlock and UpdateM1 in writeBlockWithState: the head markers are already on disk, the snapshot is not, and nothing recreates it. Every consumer of that snapshot then fails for a whole epoch. A node that keeps up with the chain head still imports blocks, because the fetcher path verifies headers with fullVerify disabled and trusts header.Validators, but it can no longer mine or vote. A node that falls behind takes the downloader path, where fullVerify is enabled, and stalls on the epoch switch block while reporting it as a bad block. Repair the hole once at startup, before the chain is used: for each gap block whose snapshot is missing, rebuild it from the state committed at that block and store it. Only the last two gap blocks at or below the head are considered. A gap block G is among them exactly while the head is in [G, G+2*Epoch), and G's snapshot is consulted exactly while the head is in [G+Gap, G+Gap+Epoch), so every hole that can still affect the running chain is covered. Checking a single gap block is not enough: a node killed at G restarts with the head on G itself, while a node that stalled on the following epoch switch restarts several hundred blocks later, and no single derivation yields both. The derivation is shared with core.BlockChain.UpdateM1 and the downloader through the unstable xdc_sort ordering, which must not drift, since a different equal-stake order yields a different masternode set. It also refuses to produce an empty snapshot, which would load back fine and permanently mask the missing masternode list. The rebuild is skipped when a snapshot is already stored, so a decode or I/O error never overwrites a masternode set persisted through a reorg, and at or below the V2 switch block, where the snapshot still comes from Initial(). It needs the gap block's state root to still be readable, which the targeted case implies: a node only keeps that root when it was committed, and this trie database never prunes what it has written. Otherwise loadLastState finds no head state, repair() rewinds to an ancestor that has one, and re-importing the gap block runs UpdateM1 and writes the snapshot again. Roots that are gone for good, such as after offline pruning or below a fast sync pivot, cannot be recovered, so every failure is only logged and startup continues.
b6451ef to
cc9c8a2
Compare
Proposed changes
A node can lose its persisted V2 gap snapshot when the process exits between writeHeadBlock and UpdateM1 in writeBlockWithState: the head markers are already on disk, the snapshot is not, and nothing recreates it. Every consumer of that snapshot then fails for a whole epoch. A node that keeps up with the chain head still imports blocks, because the fetcher path verifies headers with fullVerify disabled and trusts header.Validators, but it can no longer mine or vote. A node that falls behind takes the downloader path, where fullVerify is enabled, and stalls on the epoch switch block while reporting it as a bad block.
Repair the hole once at startup, before the chain is used: for each gap block whose snapshot is missing, rebuild it from the state committed at that block and store it.
Only the last two gap blocks at or below the head are considered. A gap block G is among them exactly while the head is in [G, G+2*Epoch), and G's snapshot is consulted exactly while the head is in [G+Gap, G+Gap+Epoch), so every hole that can still affect the running chain is covered. Checking a single gap block is not enough: a node killed at G restarts with the head on G itself, while a node that stalled on the following epoch switch restarts several hundred blocks later, and no single derivation yields both.
The derivation is shared with core.BlockChain.UpdateM1 and the downloader through the unstable xdc_sort ordering, which must not drift, since a different equal-stake order yields a different masternode set. It also refuses to produce an empty snapshot, which would load back fine and permanently mask the missing masternode list.
The rebuild is skipped when a snapshot is already stored, so a decode or I/O error never overwrites a masternode set persisted through a reorg, and at or below the V2 switch block, where the snapshot still comes from Initial(). It needs the gap block's state root to still be readable, which the targeted case implies: a node only keeps that root when it was committed, and this trie database never prunes what it has written. Otherwise loadLastState finds no head state, repair() rewinds to an ancestor that has one, and re-importing the gap block runs UpdateM1 and writes the snapshot again. Roots that are gone for good, such as after offline pruning or below a fast sync pivot, cannot be recovered, so every failure is only logged and startup continues.
Types of changes
What types of changes does your code introduce to XDC network?
Put an
✅in the boxes that applyImpacted Components
Which parts of the codebase does this PR touch?
Put an
✅in the boxes that applyChecklist
Put an
✅in the boxes once you have confirmed below actions (or provide reasons on not doing so) that