Add opt-in SessionOptions::trust_fastresume to skip the startup rechecka - #628
Open
ikaradimas wants to merge 1 commit into
Open
Add opt-in SessionOptions::trust_fastresume to skip the startup rechecka#628ikaradimas wants to merge 1 commit into
ikaradimas wants to merge 1 commit into
Conversation
…e startup recheck With `fastresume: true`, librqbit still re-verifies every restored torrent on every session start: `validate_fastresume` loads the persisted piece bitfield and, even when it is present and the correct length, SHA-1-validates a sample of pieces (at least one per file, plus ~2% of the rest). For a large library of many-file torrents the ">=1 piece per file" minimum alone forces thousands of piece reads on every launch, and Session::new_with_opts does not return until they finish. Add an opt-in SessionOptions::trust_fastresume (default false, so existing behavior is unchanged). When set and validate_fastresume loads a bitfield whose length matches piece_bitfield_bytes(), it trusts the bitfield and returns immediately, skipping the sampling loop. Safety is preserved: a previously-errored torrent still has its bitfield cleared and gets a full recheck; a missing or wrong-length bitfield still falls through to the full initial_check; on-disk corruption is still caught per-piece while seeding/downloading. This matches clients like qBittorrent/Transmission, which trust fast-resume data on start.
Owner
|
I get the pain - fastresume still checks too much, I need to replace that algorithm, it's too slow. But regardless, my idea is to still do some sanity checks, we just need to do fewer of them. Do you think this change will still be needed if we do say 64 reads per torrent (instead of a potentially larger number today)? |
Owner
|
#632 - related |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With
fastresume: true, librqbit still re-verifies every restored torrent onevery session start.
TorrentStateInitializing::validate_fastresumeloads thepersisted piece bitfield and then, even when it is present and the correct
length, SHA-1-validates a sample of pieces: at least one piece per file,
plus a ~2% random sample of the rest.
For a large library of many-file torrents (music discographies, season packs
with hundreds of files each) the "≥1 piece per file" minimum alone forces
thousands of piece reads on every launch. On one user's Windows session this
was ~410s to restore 26 torrents (22 complete and seeding), re-hashing data
it already had a valid bitfield for. Since
Session::new_with_optsdoesn'treturn until every torrent is added and the checks contend for the disk, the
client is unusable until it finishes.
There's no way to opt out today:
pausedstill checks,list_onlyskipsadding the torrent, and
concurrent_init_limitonly changes parallelism.Change
Add an opt-in
SessionOptions::trust_fastresume(defaultfalse, so existingbehavior is unchanged). When it is set and
validate_fastresumeloads abitfield whose length matches
piece_bitfield_bytes(), it trusts the bitfieldand returns immediately, skipping the sampling loop.
Safety is preserved for the risky cases:
full recheck (
have_piecesisNonebeforevalidate_fastresume);initial_check;seeding/downloading.
This matches clients like qBittorrent/Transmission, which trust fast-resume
data on start and recheck only on explicit request.
Testing
Ran against a real session of complete torrents with
trust_fastresume: true:the trust branch is taken for every restored torrent and
initial_check("Doing initial checksum validation…") never runs. With the flag
false(default), behavior is unchanged.