Repository navigation
Use upload sessions to avoid orphaned files in cloud storage #18156
Replies: 2 comments
|
My current recommendation here is to implement your own cleanup logic until we build this into core. I have design specs written up for "upload sessions" as a concept where we store any uploads in this temporary state, allowing us to also track, revert, resume or otherwise clean-up file uploads. I'd categorise this as intended behaviour as it's a known limitation so I'll move it to a github discussion and mark it as a feature request, this fits well into our DAM project |
|
Thanks for clarifying, Paul — that makes sense. We’ve implemented application-level cleanup for now using collection We verified the original reproduction case with a text file renamed to We’ll keep this workaround in place until upload sessions or equivalent lifecycle cleanup lands in core. The upload sessions concept sounds like the right long-term abstraction, and we’d be happy to test it against an R2 + Thanks! |
Uh oh!
There was an error while loading. Please reload this page.
Describe the Bug
With
clientUploads: true, a file is written to S3-compatible storage before Payload creates the upload collection document. If Payload's server-side content detection later rejects that file, the collection request correctly returns400and no database document is created, but the already-uploaded cloud object remains at its final key.This is easiest to reproduce with a text or JSON file renamed to
.jpg:.jpgand requests a signed upload URL;PUTto object storage succeeds;POST /api/mediathen returns400withFile type text/plain (from extension jpg) is not allowed.;.jpgobject is still present in the bucket.The validation itself is working as intended. The problem is that the rejected client upload is not rolled back.
In a real editorial workflow, accidental or malformed uploads are fairly ordinary. Over time, every rejected direct upload can become a billable object that is invisible to Payload's database-backed media management and normal delete flow. Upload permission is still required, so I would describe this as a storage lifecycle/resource-leak issue rather than a security vulnerability.
Source inspection suggests this occurs because the client upload is reconstructed into
req.file, collectionbeforeOperationhooks run, and core file processing/content sniffing can throw later ingenerateFileData/checkFileRestrictions. The storage adapter does not appear to receive a rollback callback for that failure path.Expected behavior: when a direct client upload is rejected before the upload document is committed, Payload (or the storage adapter contract) should best-effort delete the exact object identified by the client upload context.
Possible implementation directions could be:
An application-level collection
afterErrorhook can work around this by checkingreq.file.clientUploadContextand deleting the exact key, but generic cleanup in the client-upload/storage lifecycle would protect every upload collection and storage adapter consistently.Link to the code that reproduces this issue
The official S3 client-upload test configuration is a suitable base:
https://github.com/payloadcms/payload/blob/v3.88.0/test/storage-s3/client-uploads/config.ts
Use an upload collection restricted to image MIME types and the reproduction steps below. The relevant v3.88.0 validation path is also visible in:
Reproduction Steps
Configure
@payloadcms/storage-s3withclientUploads: trueagainst S3 or an S3-compatible provider.Configure an upload collection with an image allowlist, for example:
Create a small text or JSON file and name it
not-an-image.jpg.In the admin panel, upload that file and fill any required fields.
Observe the request sequence:
200;PUT:200;POST:400, withFile type text/plain (from extension jpg) is not allowed.Confirm that no upload collection document was created.
List the bucket at the configured prefix.
not-an-image.jpgis still present at the final key.Control case: a genuinely valid JPEG follows the same client-upload path, creates the document, and can later be removed through the normal Payload delete flow.
Which area(s) are affected?
area: coreplugin: cloud-storageplugin: storage-*Environment Info
All reactions