Add STS GetFederationToken for scoped temporary credentials

README.md

@@ -152,6 +152,11 @@ S3Proxy can modify its behavior based on middlewares:
 * [user metadata replacer](https://github.com/gaul/s3proxy/wiki/Middleware-user-metadata-replacer)
 * [no cache override](https://github.com/gaul/s3proxy/wiki/Middleware-no-cache)
 
+## Temporary credentials
+
+S3Proxy can issue scoped-down temporary credentials through the STS
+`GetFederationToken` action; see [docs/STS.md](docs/STS.md).
+
 ## SSL Support
 
 S3Proxy can listen on HTTPS by setting the `secure-endpoint` and [configuring a keystore](http://wiki.eclipse.org/Jetty/Howto/Configure_SSL#Generating_Keys_and_Certificates_with_JDK_keytool). You can read more about how configure S3Proxy for SSL Support in [the dedicated wiki page](https://github.com/gaul/s3proxy/wiki/SSL-support) with Docker, Kubernetes or simply Java.

docs/STS.md

@@ -0,0 +1,116 @@
+# Temporary credentials (STS)
+
+Status: experimental
+
+S3Proxy can answer the STS `GetFederationToken` action, letting a configured
+identity mint short-lived credentials that are scoped down by a session
+policy: limited to some buckets, some key prefixes, some operations.  Clients
+use the stock AWS SDKs and CLI pointed at S3Proxy's endpoint; nothing
+client-side is S3Proxy-specific.
+
+## Configuration
+
+```
+s3proxy.authorization=aws-v4
+s3proxy.identity=local-identity
+# use a long random secret: see Security below
+s3proxy.credential=<at least 32 random characters>
+s3proxy.sts.enabled=true
+# optional: the longest session issued, in seconds (900 to 129600)
+s3proxy.sts.max-duration=3600
+```
+
+STS requires `aws-v4` or `aws-v2-or-v4` authorization.  Temporary
+credentials sign with AWS Signature Version 4 only.
+
+## Minting credentials
+
+```
+aws --endpoint-url http://127.0.0.1:8080 sts get-federation-token \
+    --name alice-job --duration-seconds 3600 \
+    --policy '{"Version": "2012-10-17", "Statement": [
+      {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject"],
+       "Resource": "arn:aws:s3:::data/users/alice/*"},
+      {"Effect": "Allow", "Action": "s3:ListBucket",
+       "Resource": "arn:aws:s3:::data",
+       "Condition": {"StringLike": {"s3:prefix": "users/alice/*"}}}]}'
+```
+
+The response carries an `AccessKeyId`, `SecretAccessKey` and `SessionToken`
+which any AWS client accepts as session credentials.  A session minted
+without a policy may do nothing.  Session credentials cannot mint further
+sessions.
+
+## Session policies
+
+Policies use a subset of the IAM policy language:
+
+* `Effect`: `Allow` or `Deny`; an explicit Deny wins, and anything no
+  statement allows is denied
+* `Action`: `s3:` actions, or `*`, with `*` and `?` wildcards
+* `Resource`: `arn:aws:s3:::bucket` and `arn:aws:s3:::bucket/key` ARNs, or
+  `*`, with wildcards
+* `Condition`: `StringEquals` and `StringLike` on `s3:prefix` and
+  `s3:delimiter`, which scope listings
+
+Anything else -- `NotAction`, `NotResource`, `Principal`, other condition
+operators or keys, policy variables -- is refused with
+`MalformedPolicyDocument` rather than ignored.  Policies are limited to 2048
+bytes of UTF-8, so that the token carrying one fits in a request header.
+
+Each S3 operation requires the IAM action AWS documents for it, e.g.
+`HeadObject` requires `s3:GetObject`, multipart uploads require
+`s3:PutObject`, a copy requires `s3:GetObject` on its source and
+`s3:PutObject` on its destination, `DeleteObjects` requires `s3:DeleteObject`
+on every key, `GetObjectAttributes` requires `s3:GetObject` as well, and
+writing with a non-private `x-amz-acl` also requires `s3:PutObjectAcl`.
+Operations naming a `versionId` require the `*Version` actions, e.g.
+`s3:GetObjectVersion`, `s3:DeleteObjectVersion`.
+
+Keys with empty, `.` or `..` path segments (e.g. `home/alice/../bob/x`, or
+a key starting with `/`) are refused for temporary credentials: a filesystem
+backend resolves them to a different object than the one the policy
+matched.  A trailing `/`, as directory markers have, is allowed.
+
+## Design
+
+S3Proxy stores nothing about the sessions it issues.  The minting
+identity's secret is expanded with HKDF into two keys.  The session token is
+an AES-256-GCM encryption of the temporary access key id, expiry, name and
+policy under one; the temporary secret access key is an HMAC of the access
+key id under the other.  Consequently:
+
+* any S3Proxy configured with the same identity and secret accepts the
+  token, so replicas need no shared state
+* changing or removing the identity's secret revokes every session it
+  minted
+* an individual session cannot be revoked before it expires
+
+## Security
+
+Use a strong parent secret: at least 32 random characters, as an AWS secret
+access key has.  Temporary credentials are derived from it, so whoever holds
+them can test guesses at the parent secret offline, and a correct guess
+grants everything the parent may do, including minting further sessions.  A
+short or memorable secret such as the sample `local-credential` would fall
+quickly.
+
+The session token is redacted from S3Proxy's header trace log.
+
+## Limitations
+
+* Only `GetFederationToken` is implemented; `AssumeRole`,
+  `GetSessionToken` and `GetCallerIdentity` are not.
+* Browser POST uploads (POST policy forms) do not accept session
+  credentials.
+* Temporary credentials do not work with AWS Signature Version 2.
+* Policies match keys case-sensitively, as S3 does.  A filesystem backend
+  on a case-insensitive filesystem (e.g. macOS's default APFS) does not:
+  there `home/alice/SECRET/x` opens the same file as `home/alice/secret/x`,
+  so do not rely on a Deny, or on prefixes differing only in case, to
+  separate them.
+* With `s3proxy.virtual-host` set, STS requests must carry the virtual host
+  itself as their Host; any other host name is read as a bucket.
+* With `s3proxy.service-path` set, the AWS SDKs POST STS requests to the
+  service path without a trailing slash, which S3Proxy redirects; they
+  cannot mint credentials through it.

src/main/java/org/gaul/s3proxy/AwsHttpHeaders.java

@@ -62,6 +62,7 @@ public final class AwsHttpHeaders {
     public static final String REQUEST_ID = "x-amz-request-id";
     public static final String SDK_CHECKSUM_ALGORITHM =
             "x-amz-sdk-checksum-algorithm";
+    public static final String SECURITY_TOKEN = "x-amz-security-token";
     public static final String SERVER_SIDE_ENCRYPTION =
             "x-amz-server-side-encryption";
     public static final String SERVER_SIDE_ENCRYPTION_AWS_KMS_KEY_ID =

src/main/java/org/gaul/s3proxy/S3ErrorCode.java

@@ -44,15 +44,21 @@ public enum S3ErrorCode {
             "Your proposed upload is smaller than the minimum allowed object" +
             " size. Each part must be at least 5 MB in size, except the last" +
             " part."),
+    EXPIRED_TOKEN(HttpServletResponse.SC_BAD_REQUEST,
+            "The provided token has expired."),
     INCOMPLETE_BODY(HttpServletResponse.SC_BAD_REQUEST,
             "You did not provide the number of bytes specified by the" +
             " Content-Length HTTP header."),
     INTERNAL_ERROR(HttpServletResponse.SC_INTERNAL_SERVER_ERROR,
             "An internal error occurred. Try again."),
     INVALID_ACCESS_KEY_ID(HttpServletResponse.SC_FORBIDDEN, "Forbidden"),
+    INVALID_ACTION(HttpServletResponse.SC_BAD_REQUEST,
+            "The action or operation requested is invalid."),
     INVALID_ARGUMENT(HttpServletResponse.SC_BAD_REQUEST, "Bad Request"),
     INVALID_BUCKET_NAME(HttpServletResponse.SC_BAD_REQUEST,
             "The specified bucket is not valid."),
+    INVALID_CLIENT_TOKEN_ID(HttpServletResponse.SC_FORBIDDEN,
+            "The security token included in the request is invalid."),
     INVALID_CORS_ORIGIN(HttpServletResponse.SC_BAD_REQUEST,
             "Insufficient information. Origin request header needed."),
     INVALID_CORS_METHOD(HttpServletResponse.SC_BAD_REQUEST,
@@ -77,15 +83,22 @@ public enum S3ErrorCode {
     INVALID_REQUEST(HttpServletResponse.SC_BAD_REQUEST, "Bad Request"),
     INVALID_STORAGE_CLASS(HttpServletResponse.SC_BAD_REQUEST,
             "The storage class you specified is not valid."),
+    INVALID_TOKEN(HttpServletResponse.SC_BAD_REQUEST,
+            "The provided token is malformed or otherwise invalid."),
     INVALID_U_R_I(HttpServletResponse.SC_BAD_REQUEST,
             "Couldn't parse the specified URI."),
+    MALFORMED_POLICY_DOCUMENT(HttpServletResponse.SC_BAD_REQUEST,
+            "The policy document is malformed or uses unsupported" +
+            " elements."),
     MALFORMED_X_M_L(HttpServletResponse.SC_BAD_REQUEST,
             "The XML you provided was not well-formed or did not validate" +
             " against our published schema."),
     MAX_MESSAGE_LENGTH_EXCEEDED(HttpServletResponse.SC_BAD_REQUEST,
             "Your request was too big."),
     METHOD_NOT_ALLOWED(HttpServletResponse.SC_METHOD_NOT_ALLOWED,
             "Method Not Allowed"),
+    MISSING_ACTION(HttpServletResponse.SC_BAD_REQUEST,
+            "The request is missing an action."),
     MISSING_CONTENT_LENGTH(HttpServletResponse.SC_LENGTH_REQUIRED,
             "Length Required"),
     NO_SUCH_BUCKET(HttpServletResponse.SC_NOT_FOUND,
@@ -103,6 +116,8 @@ public enum S3ErrorCode {
     REQUEST_TIME_TOO_SKEWED(HttpServletResponse.SC_FORBIDDEN, "Forbidden"),
     REQUEST_TIMEOUT(HttpServletResponse.SC_BAD_REQUEST, "Bad Request"),
     SIGNATURE_DOES_NOT_MATCH(HttpServletResponse.SC_FORBIDDEN, "Forbidden"),
+    VALIDATION_ERROR(HttpServletResponse.SC_BAD_REQUEST,
+            "The request did not satisfy a parameter constraint."),
     X_AMZ_CONTENT_S_H_A_256_MISMATCH(HttpServletResponse.SC_BAD_REQUEST,
             "The provided 'x-amz-content-sha256' header does not match what" +
             " was computed.");

src/main/java/org/gaul/s3proxy/S3Operation.java

@@ -50,6 +50,7 @@ public enum S3Operation {
     PUT_BUCKET_VERSIONING("PutBucketVersioning"),
     LIST_OBJECT_VERSIONS("ListObjectVersions"),
     OPTIONS_OBJECT("OptionsObject"),
+    GET_FEDERATION_TOKEN("GetFederationToken"),
     UNKNOWN("Unknown");
 
     private final String value;

src/main/java/org/gaul/s3proxy/S3Proxy.java

@@ -21,6 +21,7 @@
 
 import java.net.URI;
 import java.net.URISyntaxException;
+import java.time.Duration;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.Objects;
@@ -45,6 +46,7 @@
 import org.eclipse.jetty.util.thread.QueuedThreadPool;
 import org.gaul.s3proxy.auth.AuthenticationType;
 import org.gaul.s3proxy.blobstore.BlobStore;
+import org.gaul.s3proxy.sts.StsHandler;
 import org.jspecify.annotations.Nullable;
 
 /**
@@ -163,7 +165,9 @@ public final class S3Proxy {
                 builder.v4MaxNonChunkedRequestSize,
                 builder.v4MaxChunkSize,
                 builder.ignoreUnknownHeaders, builder.corsRules,
-                builder.servicePath, builder.maximumTimeSkew, metrics);
+                builder.servicePath, builder.maximumTimeSkew, metrics,
+                builder.stsEnabled ?
+                        new StsHandler(builder.stsMaxDuration) : null);
 
         var context = new ServletContextHandler();
         if (builder.servicePath != null && !builder.servicePath.isEmpty()) {
@@ -235,6 +239,8 @@ public static final class Builder {
         private int metricsPort = -1;
         @Nullable private String metricsHost;
         private long stopTimeout = STOP_TIMEOUT_MS;
+        private boolean stsEnabled;
+        private Duration stsMaxDuration = StsHandler.MAX_DURATION;
 
         Builder() {
         }
@@ -424,6 +430,19 @@ public static Builder fromProperties(Properties properties)
                 builder.metricsHost(metricsHost);
             }
 
+            String stsEnabled = properties.getProperty(
+                    S3ProxyConstants.PROPERTY_STS_ENABLED);
+            if (!Strings.isNullOrEmpty(stsEnabled)) {
+                builder.stsEnabled(Boolean.parseBoolean(stsEnabled));
+            }
+
+            String stsMaxDuration = properties.getProperty(
+                    S3ProxyConstants.PROPERTY_STS_MAX_DURATION);
+            if (!Strings.isNullOrEmpty(stsMaxDuration)) {
+                builder.stsMaxDuration(Duration.ofSeconds(
+                        Long.parseLong(stsMaxDuration)));
+            }
+
             return builder;
         }
 
@@ -554,6 +573,16 @@ public Builder metricsHost(String metricsHost) {
             return this;
         }
 
+        public Builder stsEnabled(boolean stsEnabled) {
+            this.stsEnabled = stsEnabled;
+            return this;
+        }
+
+        public Builder stsMaxDuration(Duration stsMaxDuration) {
+            this.stsMaxDuration = requireNonNull(stsMaxDuration);
+            return this;
+        }
+
         public Builder servicePath(String s3ProxyServicePath) {
             String path = Strings.nullToEmpty(s3ProxyServicePath);
 
@@ -611,15 +640,18 @@ public boolean equals(Object object) {
                             that.v4MaxNonChunkedRequestSize &&
                     this.v4MaxChunkSize == that.v4MaxChunkSize &&
                     this.ignoreUnknownHeaders == that.ignoreUnknownHeaders &&
-                    Objects.equals(this.corsRules, that.corsRules);
+                    Objects.equals(this.corsRules, that.corsRules) &&
+                    this.stsEnabled == that.stsEnabled &&
+                    Objects.equals(this.stsMaxDuration, that.stsMaxDuration);
         }
 
         @Override
         public int hashCode() {
             return Objects.hash(endpoint, secureEndpoint, sslContext,
                     keyStorePath, keyStorePassword, virtualHost, servicePath,
                     maxSinglePartObjectSize, v4MaxNonChunkedRequestSize,
-                    v4MaxChunkSize, ignoreUnknownHeaders, corsRules);
+                    v4MaxChunkSize, ignoreUnknownHeaders, corsRules,
+                    stsEnabled, stsMaxDuration);
         }
     }
 

src/main/java/org/gaul/s3proxy/S3ProxyConstants.java

@@ -165,6 +165,20 @@ public final class S3ProxyConstants {
     public static final String PROPERTY_METRICS_HOST =
             "s3proxy.metrics.host";
 
+    /**
+     * Answer STS GetFederationToken, letting a configured identity mint
+     * temporary credentials scoped down by a session policy.
+     */
+    public static final String PROPERTY_STS_ENABLED =
+            "s3proxy.sts.enabled";
+
+    /**
+     * The longest session GetFederationToken issues, in seconds; a request
+     * for longer is shortened to it.  Defaults to the AWS limit of 129600.
+     */
+    public static final String PROPERTY_STS_MAX_DURATION =
+            "s3proxy.sts.max-duration";
+
     public static final String PROPERTY_ALT_JCLOUDS_PREFIX = "alt.";
 
     private S3ProxyConstants() {