Garbage-collect orphaned thumbnails via reference markers - #43
Open
fanchenggang wants to merge 1 commit into
Open
Garbage-collect orphaned thumbnails via reference markers#43fanchenggang wants to merge 1 commit into
fanchenggang wants to merge 1 commit into
Conversation
Thumbnails are content-addressed and shared between files whose generated thumbnails have the same digest, so deleting a file could never remove its thumbnail and they accumulated forever (longern#16). Each referencing file now keeps an immutable marker object under _$flaredrive$/thumbnails/refs/<digest>/<path>: - put / multipart create / copy write the marker before the referencing object, so a concurrent delete of the last other file using the thumbnail cannot collect it mid-upload - delete (file or directory) removes the markers first and collects a thumbnail only once no marker remains, so duplicates sharing a thumbnail are handled and concurrent deletes can at worst over-retain (never break a thumbnail still in use) - overwriting a file releases its previous thumbnail reference - delete-all also wipes the internal thumbnails subtree, which listAll skips - the fd-thumbnail header is validated as a hex digest
This was referenced Sep 6, 2026
fanchenggang
added a commit
to fanchenggang/Davflare
that referenced
this pull request
Sep 6, 2026
缩略图按内容摘要寻址并被内容相同的文件共享,原删除逻辑从不清理 _$flaredrive$/thumbnails/。每个引用文件现在用不可变 marker 做引用计数,清零才回收。同步上游 longern/FlareDrive#43。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #16.
Problem
Thumbnails are content-addressed (
_$flaredrive$/thumbnails/<digest>.png) and intentionally shared between files whose generated thumbnails have the same digest.delete.tsnever removed them, so every deleted image left an orphaned thumbnail behind, forever.Solution: per-path reference markers
Each referencing file keeps an immutable marker object at
_$flaredrive$/thumbnails/refs/<digest>/<path>. A thumbnail is deleted only once no marker remains for its digest. No counters, no read-modify-write: every operation is a single object PUT or DELETE.PUT/ multipart create /COPYDELETE(file)DELETE(directory)PUT/COPYover an existing key)DELETE/)_$flaredrive$/thumbnails/subtree, whichlistAllskipsWhy this is safe
PUTverifies the thumbnail object still exists after referencing it and returns409otherwise, closing the window where a concurrent GC removes the blob between the client's thumbnail upload and the file upload — the client can simply retry.fd-thumbnailheader is now validated as a hex digest (16–128 chars) before it is ever used in an object key.Notes
COPY/MOVEof thumbnail-carrying objects propagates markers, even though they never sendfd-thumbnailthemselves.PUTthat fails on a precondition — those are cleaned up, but an aborted multipart flow is not.Happy to adjust the marker layout or add a scheduled sweep for legacy orphans if you'd prefer.