Skip to content

S3 CopyToFile fails with PermanentRedirect when source and target are in different AWS regions #336

Description

@dmcilvain44

Bug

File.CopyToFile between two S3 files fails with an AWS PermanentRedirect (HTTP 301) error when the source and target buckets are in different AWS regions.

Root Cause

In backend/s3/file.go, isSameAuth determines whether to use the native S3 CopyObject API or fall back to TouchCopyBuffered (client-side stream copy). It compares only AccessKeyID and SessionToken:

isSameAccount := (fileOptions.AccessKeyID == targetOptions.AccessKeyID) &&
    (fileOptions.SessionToken == targetOptions.SessionToken)
return isSameAccount, ACL

When source and target use the same AWS credentials but are configured with different regions (e.g. us-west-2ap-south-1), isSameAuth returns true. This causes CopyObject to be called using the source filesystem's S3 client, which hits the source region's endpoint. AWS rejects this with PermanentRedirect because the target bucket lives in a different region.

There is also an edge case at lines 608–611: when both filesystems have empty Options{} (relying on instance IAM credentials), isSameAuth unconditionally returns true regardless of whether the buckets are in different regions.

Expected Behavior

When source and target S3 buckets are in different regions, CopyToFile should fall through to TouchCopyBuffered, which reads from the source client and writes via the target client — each using their own correctly configured regional endpoint.

Proposed Fix

Add a Region comparison to isSameAuth:

isSameAccount := (fileOptions.AccessKeyID == targetOptions.AccessKeyID) &&
    (fileOptions.SessionToken == targetOptions.SessionToken) &&
    (fileOptions.Region == targetOptions.Region)

Same credentials + different region → false → falls through to TouchCopyBuffered → succeeds.

Steps to Reproduce

  1. Create two S3 Options with the same AccessKeyID/SecretAccessKey but different Region values (e.g. us-west-2 and ap-south-1)
  2. Create two VFS S3 filesystems using these options
  3. Create a source file on the us-west-2 filesystem and a target file on the ap-south-1 filesystem
  4. Call sourceFile.CopyToFile(targetFile)

Result: AWS returns PermanentRedirect — the copy fails.
Expected: Falls through to TouchCopyBuffered and succeeds.

Affected Version

Reproduced on v7.10.0.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions