Repository navigation
Add STS GetFederationToken for scoped temporary credentials - #1203
harshalbrahmbhatt-cohere wants to merge 2 commits into
Conversation
On the backends that keep a stub per multipart upload -- filesystem, transient, sftp -- the stub was named by upload id alone, so a request naming one key could upload parts to, complete or abort an upload begun for another. Completing it published, under the requester's key, the ACL and metadata the original upload had been given. S3 answers NoSuchUpload to an upload id paired with the wrong key. Name the stub by upload id and a digest of its key, so that no stub answers to the wrong pair, and refuse UploadPart, UploadPartCopy and ListParts for an upload that does not exist under that key as Complete and Abort already did. The name rather than the stub's metadata holds the key because a client chooses metadata and some stores keep none. Uploads begun before this change keep a stub named by id alone, which is still honoured so that an upgrade does not strand them.
A configured identity may now mint short-lived credentials narrowed by a session policy, through the STS GetFederationToken action on the S3 endpoint. Stock AWS SDKs and the CLI work unchanged: they POST a SigV4 form signed for the sts service and sign later S3 requests with the returned access key, secret and x-amz-security-token. Enabled with s3proxy.sts.enabled; s3proxy.sts.max-duration caps session length. Nothing about a session is stored. The token is an AES-256-GCM encryption of the temporary access key id, expiry, name and policy under a key derived with HKDF from the minting identity's secret, and the temporary secret is an HMAC of the access key id under another, so any replica with the identity can verify a token and changing the secret revokes every session it minted. Policies are a subset of IAM: Allow and Deny, Action and Resource with wildcards, and StringEquals/StringLike on s3:prefix and s3:delimiter. Anything else is refused with MalformedPolicyDocument rather than ignored. Each operation is checked against the IAM action S3 requires of it, including a copy's source and every key of a DeleteObjects, and keys with empty, "." or ".." segments are refused for sessions since a filesystem store resolves them to another object than the policy saw. docs/STS.md describes configuration, the policy subset and the limitations; run-sts-cli-tests.sh exercises it end to end with the AWS CLI.
4d50f81 to
478fe68
Compare
|
Thanks, this is a careful piece of work. I read the STS side closely and found nothing to fix:
CI is red because of the multipart commit.
I reproduced the native-backend case on master against Azurite and filed it as #1235. I created an upload for That matters for this PR. A session's policy is checked against the key the request names, while the store completes the upload the ID belongs to. So a session limited to
Two smaller things:
|
Closes #1183.
Following the discussion in #1183, this implements the STS-style approach rather than R2-style. It is limited to
GetFederationToken, the one STS action that issues credentials narrowed by a session policy without needing an IAM role store. A configured identity mints short-lived credentials limited to some buckets, key prefixes and operations, and stock AWS clients use them unchanged.Design. It is fully stateless. The session token is AES-256-GCM over the temporary key ID, expiry, name and policy, under a key derived with HKDF from the minting identity's secret. The temporary secret is an HMAC of the key ID under a second derived key. Any replica with the same identity verifies tokens, and rotating the identity's secret revokes every session it minted. It is opt-in through
s3proxy.sts.enabled, and requiresaws-v4oraws-v2-or-v4authorization.Policy subset.
ActionandResourcewith wildcards, andStringEquals/StringLikeons3:prefixands3:delimiter.MalformedPolicyDocumentrather than ignored: everything else.PutObjectAclfor a non-privatex-amz-acl.Demonstrated with existing tools. You mentioned
session_policyin s3-tests. Those tests all go throughAssumeRoleWithWebIdentity, which needs an IAM role API, OIDC providers and Keycloak, so they are out of scope here. Instead:StsFederationTokenTest: AWS SDK v2StsClientandS3Clientagainst a live proxy. It covers allowed and denied matrices, copy, multipart, DeleteObjects, presigned URLs, tampered, expired or mismatched tokens, parent-secret rotation, and body-tamper on STS requests.src/test/resources/run-sts-cli-tests.sh: the stock AWS CLI end to end against the packaged jar. It is not yet wired into CI; happy to add it if you'd like.Separate commit, also relevant without STS. On stub backends (filesystem, transient, sftp), a multipart upload ID was not bound to its key. A request naming one key could complete or abort another key's upload and inherit its ACL and metadata. The stub is now named by upload ID plus a key digest, and wrong pairs get
NoSuchUploadas in S3. Stubs written before the upgrade are still honoured.Limitations (also in
docs/STS.md):GetFederationToken; no AssumeRole, GetSessionToken or GetCallerIdentity.s3proxy.service-path, SDKs can't mint, because the path without a trailing slash redirects.