Repository navigation
S3 gateway: replace Content-Type on CopyObject with REPLACE directive - #10593
Open
RaphaelFakhri wants to merge 1 commit into
Open
RaphaelFakhri wants to merge 1 commit into
RaphaelFakhri wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10577
Change Description
Background
With
x-amz-metadata-directive: REPLACE, S3 replaces the object's content type with theContent-Typesent on the copy request. Copying an object onto itself with a new content type is the usual way to fix a wrong one. The lakeFS S3 gateway replaces only the user metadata and keeps the source content type.Bug Fix
aws s3api copy-object --metadata-directive REPLACE --content-type application/jsonthrough the S3 gateway keeps the source content type (for exampletext/plain). The new user metadata is applied.handleCopypasses only thex-amz-meta-*headers toCatalog.CopyEntry, andCopyEntrycopiesContentTypefrom the source entry in every case.Catalog.CopyEntry(andcloneEntry) take acontentTypeargument and set it on the new entry whenreplaceSrcMetadatais true, together with the metadata. The gateway passes the request'sContent-Type. As with PUT, an empty value stores no content type and reads return the default. The API copy handler passes""withreplaceSrcMetadataset to false, so its behavior does not change.Testing Details
TestCopyObjectReplaceContentTypeinpkg/gateway/handler_test.go. It uploads an object withtext/plainthrough the gateway, copies it withREPLACEand withCOPYwhile sendingContent-Type: application/json, and checks theContent-Typethat HEAD returns:application/jsonforREPLACE,text/plainforCOPY.replacesubtest fails (expected: "application/json",actual: "text/plain") and thecopysubtest passes. With the fix, both pass.GetBasicHandlerinpkg/gateway/testutilnow gives the handler the catalog's block adapter whenUSE_BLOCK_ADAPTERis unset ormem. Each in-memory adapter keeps its own data, so before this change a copy through the test handler could not read an object written through the same handler. WhenUSE_BLOCK_ADAPTERnames another adapter, the handler still gets that adapter.go test ./pkg/gateway/... ./pkg/catalog/,go test -run Copy ./pkg/api/,go build ./...,go vet, andgo tool golangci-lint run ./pkg/gateway/... ./pkg/catalog/ ./pkg/api/(0 issues) pass.Breaking Change?
No API, CLI, or client change. S3 gateway clients that send
REPLACEnow get the request'sContent-Typeon the copy, which matches S3.Additional info
Reproduction from the issue, against the S3 gateway:
Contact Details
mail@raphaelfakhri.com