Skip to content

Add STS GetFederationToken for scoped temporary credentials - #1203

Open
harshalbrahmbhatt-cohere wants to merge 2 commits into
gaul:masterfrom
harshalbrahmbhatt-cohere:harshal/sts-federation-token
Open

harshalbrahmbhatt-cohere wants to merge 2 commits into
gaul:masterfrom
harshalbrahmbhatt-cohere:harshal/sts-federation-token

Conversation

@harshalbrahmbhatt-cohere

@harshalbrahmbhatt-cohere harshalbrahmbhatt-cohere commented Oct 5, 2026 •

Copy link
Copy Markdown

Closes #1183.

Following the discussion in #1183, this implements the STS-style approach rather than R2-style. It is limited to GetFederationToken, the one STS action that issues credentials narrowed by a session policy without needing an IAM role store. A configured identity mints short-lived credentials limited to some buckets, key prefixes and operations, and stock AWS clients use them unchanged.

aws --endpoint-url http://127.0.0.1:8080 sts get-federation-token \
    --name alice-job --duration-seconds 3600 \
    --policy '{"Statement": [
      {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject"],
       "Resource": "arn:aws:s3:::data/users/alice/*"},
      {"Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::data",
       "Condition": {"StringLike": {"s3:prefix": "users/alice/*"}}}]}'

Design. It is fully stateless. The session token is AES-256-GCM over the temporary key ID, expiry, name and policy, under a key derived with HKDF from the minting identity's secret. The temporary secret is an HMAC of the key ID under a second derived key. Any replica with the same identity verifies tokens, and rotating the identity's secret revokes every session it minted. It is opt-in through s3proxy.sts.enabled, and requires aws-v4 or aws-v2-or-v4 authorization.

Policy subset.

  • Supported: Allow and Deny (explicit Deny wins), Action and Resource with wildcards, and StringEquals/StringLike on s3:prefix and s3:delimiter.
  • Refused with MalformedPolicyDocument rather than ignored: everything else.
  • Each operation is checked against the IAM action AWS documents for it. That includes the copy source, every DeleteObjects key, and PutObjectAcl for a non-private x-amz-acl.

Demonstrated with existing tools. You mentioned session_policy in s3-tests. Those tests all go through AssumeRoleWithWebIdentity, which needs an IAM role API, OIDC providers and Keycloak, so they are out of scope here. Instead:

  • StsFederationTokenTest: AWS SDK v2 StsClient and S3Client against a live proxy. It covers allowed and denied matrices, copy, multipart, DeleteObjects, presigned URLs, tampered, expired or mismatched tokens, parent-secret rotation, and body-tamper on STS requests.
  • src/test/resources/run-sts-cli-tests.sh: the stock AWS CLI end to end against the packaged jar. It is not yet wired into CI; happy to add it if you'd like.
  • s3-tests results are unchanged with STS enabled.

Separate commit, also relevant without STS. On stub backends (filesystem, transient, sftp), a multipart upload ID was not bound to its key. A request naming one key could complete or abort another key's upload and inherit its ACL and metadata. The stub is now named by upload ID plus a key digest, and wrong pairs get NoSuchUpload as in S3. Stubs written before the upgrade are still honoured.

Limitations (also in docs/STS.md):

  • Only GetFederationToken; no AssumeRole, GetSessionToken or GetCallerIdentity.
  • No POST-policy uploads and no SigV2 with session credentials.
  • Parent secrets must be strong, since temporary credentials allow offline guessing.
  • On a case-insensitive filesystem, the filesystem backend doesn't separate keys that differ only in case.
  • Under s3proxy.service-path, SDKs can't mint, because the path without a trailing slash redirects.

Harshal Brahmbhatt added 2 commits October 5, 2026 15:14
On the backends that keep a stub per multipart upload -- filesystem,
transient, sftp -- the stub was named by upload id alone, so a request
naming one key could upload parts to, complete or abort an upload begun
for another.  Completing it published, under the requester's key, the
ACL and metadata the original upload had been given.  S3 answers
NoSuchUpload to an upload id paired with the wrong key.

Name the stub by upload id and a digest of its key, so that no stub
answers to the wrong pair, and refuse UploadPart, UploadPartCopy and
ListParts for an upload that does not exist under that key as Complete
and Abort already did.  The name rather than the stub's metadata holds
the key because a client chooses metadata and some stores keep none.
Uploads begun before this change keep a stub named by id alone, which is
still honoured so that an upgrade does not strand them.
A configured identity may now mint short-lived credentials narrowed by a
session policy, through the STS GetFederationToken action on the S3
endpoint.  Stock AWS SDKs and the CLI work unchanged: they POST a
SigV4 form signed for the sts service and sign later S3 requests with
the returned access key, secret and x-amz-security-token.  Enabled with
s3proxy.sts.enabled; s3proxy.sts.max-duration caps session length.

Nothing about a session is stored.  The token is an AES-256-GCM
encryption of the temporary access key id, expiry, name and policy
under a key derived with HKDF from the minting identity's secret, and
the temporary secret is an HMAC of the access key id under another, so
any replica with the identity can verify a token and changing the
secret revokes every session it minted.

Policies are a subset of IAM: Allow and Deny, Action and Resource with
wildcards, and StringEquals/StringLike on s3:prefix and s3:delimiter.
Anything else is refused with MalformedPolicyDocument rather than
ignored.  Each operation is checked against the IAM action S3 requires
of it, including a copy's source and every key of a DeleteObjects, and
keys with empty, "." or ".." segments are refused for sessions since a
filesystem store resolves them to another object than the policy saw.

docs/STS.md describes configuration, the policy subset and the
limitations; run-sts-cli-tests.sh exercises it end to end with the AWS
CLI.
@gaul

gaul commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Thanks, this is a careful piece of work. I read the STS side closely and found nothing to fix:

  • Token: HKDF with the parent in the salt, AES-GCM with the header as associated data, and the token's access key bound to the signature.
  • Requests: checks are deny-by-default, with authorize on every operation; copy sources and DeleteObjects keys are checked too, and non-canonical keys are refused.
  • Policy: unsupported syntax fails closed.
  • STS endpoint: no-policy sessions get deny-all, sessions can't mint more, and signatures are scoped to their service.

CI is red because of the multipart commit. MultipartUploadKeyBindingTest runs on every lane, but the binding exists only on the stub backends:

  • On Azure, GCS and Swift, Complete Multipart Upload with another key's upload ID isn't refused (line 94).
  • On LocalStack and MiniStack, Abort isn't (line 97).

I reproduced the native-backend case on master against Azurite and filed it as #1235. I created an upload for a and uploaded a part, then sent Complete naming b. The response was 200 with Key: b, but it finished the upload for a, and b was never created.

That matters for this PR. A session's policy is checked against the key the request names, while the store completes the upload the ID belongs to. So a session limited to users/alice/* can complete an upload in progress under users/bob/*, given its upload ID, which ListMultipartUploads reveals. I think the binding needs to hold on every backend before STS can rely on key-scoped policies. Could you move the multipart commit into its own PR and either:

  • check the pairing for every backend, for example in the handler before Complete, Abort, UploadPart and ListParts; or
  • if that is too broad for now, gate the test to the stub backends and refuse CompleteMultipartUpload and AbortMultipartUpload for sessions on the other backends until they bind?

Two smaller things:

  • run-sts-cli-tests.sh isn't run anywhere. Either wire it into the filesystem lane or drop it, since a script nothing runs goes stale.
  • s3-tests results are fine to leave as you stated them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Interest in scoped-down temporary credentials?

2 participants