Add AWS SDK sts module to ocfl-java-aws for IRSA / web identity support

[mohideen]

Summary

When using ocfl-java-aws in a Kubernetes/EKS deployment that relies on IAM Roles for Service Accounts (IRSA), the S3-backed OCFL repository initialization fails because the AWS SDK v2 sts service module is not on the classpath.

The AWS SDK’s default credential chain attempts to use web identity tokens, but without the sts module it cannot do so and emits a warning. This results in OCFL S3 initialization failing in applications that depend on ocfl-java-aws (for example, Fedora when configured with fcrepo.storage=ocfl-s3).

Environment

• ocfl-java-aws: 2.2.1
• Java: 11
• Deployment: Kubernetes (EKS) with IRSA
• Storage: S3 backend via ocfl-java-aws

Expected behavior

ocfl-java-aws should be able to use the pod’s IRSA/IAM role via the AWS SDK’s default credential provider chain, which requires the software.amazon.awssdk:sts module to be available on the classpath. With sts present, the SDK can exchange the projected web identity token for temporary credentials and S3 access should succeed.

Actual behavior

At startup, the AWS SDK emits a warning that the sts module is missing, and OCFL fails to initialize the S3 storage because it cannot list objects under the configured prefix. Below is a representative excerpt from the application logs (Fedora using ocfl-s3):

INFO  (OcflPropsConfig) Fedora storage type: ocfl-s3
INFO  (OcflPropsConfig) Fedora OCFL S3 bucket: <REDACTED>
INFO  (OcflPropsConfig) Fedora OCFL S3 prefix: fedora/ocfl-root

WARN  (WebIdentityCredentialsUtils) To use web identity tokens, the 'sts' service module must be on the class path.

...
Caused by: io.ocfl.aws.OcflS3Exception: Failed to list objects under fedora/ocfl-root/
    at io.ocfl.aws.OcflS3Client.directoryExists(OcflS3Client.java:443)
    at io.ocfl.core.storage.cloud.CloudStorage.directoryIsEmpty(CloudStorage.java:116)
    at io.ocfl.core.storage.DefaultOcflStorageInitializer.directoryIsEmpty(DefaultOcflStorageInitializer.java:383)
    at io.ocfl.core.storage.DefaultOcflStorageInitializer.initializeStorage(DefaultOcflStorageInitializer.java:85)
    at io.ocfl.core.storage.DefaultOcflStorage.doInitialize(DefaultOcflStorage.java:627)
    at io.ocfl.core.storage.AbstractOcflStorage.initializeStorage(AbstractOcflStorage.java:63)
    at io.ocfl.core.storage.ObjectDetailsDbOcflStorage.doInitialize(ObjectDetailsDbOcflStorage.java:65)
    at io.ocfl.core.storage.AbstractOcflStorage.initializeStorage(AbstractOcflStorage.java:63)
    at io.ocfl.core.storage.CachingOcflStorage.doInitialize(CachingOcflStorage.java:59)
    at io.ocfl.core.storage.AbstractOcflStorage.initializeStorage(AbstractOcflStorage.java:63)
    at io.ocfl.core.OcflRepositoryBuilder.buildInternal(OcflRepositoryBuilder.java:421)
    at io.ocfl.core.OcflRepositoryBuilder.buildMutable(OcflRepositoryBuilder.java:411)
    at org.fcrepo.persistence.ocfl.impl.OcflPersistentStorageUtils.createRepository(OcflPersistentStorageUtils.java:182)
    at org.fcrepo.persistence.ocfl.impl.OcflPersistentStorageUtils.createS3Repository(OcflPersistentStorageUtils.java:145)
    at org.fcrepo.persistence.ocfl.impl.OcflPersistenceConfig.repository(OcflPersistenceConfig.java:74)
    ...

Root cause

According to the AWS SDK for Java 2.x documentation and EKS IRSA guidance, use of web identity tokens requires the software.amazon.awssdk:sts module to be present on the classpath:

“When using Java, you must include the sts module on the classpath.”
“If you see the error ‘To use web identity tokens, the sts service module must be on the class path’, you need to add the STS module dependency.” [AWS SDK for Java 2.x / IRSA / troubleshooting references]

In ocfl-java-aws, the POM currently includes:

<!-- AWS -->
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3</artifactId>
</dependency>
<dependency>
    <groupId>software.amazon.awssdk.crt</groupId>
    <artifactId>aws-crt</artifactId>
</dependency>
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3-transfer-manager</artifactId>
</dependency>

There is no software.amazon.awssdk:sts dependency, so when the default credential provider chain reaches the web identity provider, it cannot load the STS service module and fails as shown above.

Proposed solution

Add the STS module to the ocfl-java-aws dependencies so that the AWS SDK v2 can use web identity tokens (IRSA) to obtain credentials:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>sts</artifactId>
</dependency>

This keeps existing static-credentials configurations working, and enables standard IRSA-based deployments on Kubernetes/EKS without requiring downstream applications to ship their own copy of sts.

If this approach sounds acceptable, I’m happy to open a PR with this change.

[pwinckles]

If your application needs sts on the classpath, can you not just add it to your application's dependencies?

[mohideen]

Yes — adding  sts  to the application’s dependencies is a viable downstream fix, and I’m using that approach temporarily in a derived Fedora image.

My question is mainly about where the dependency belongs long‑term. Since  ocfl-java-aws  is the component that depends on the AWS SDK for S3-backed OCFL, and IRSA/web-identity is a standard AWS SDK credential path, I was wondering whether  sts  should be considered part of that module’s required runtime dependencies so downstream consumers do not have to discover and add it themselves.

I’m also trying to avoid falling back to static access key/secret for pods, since AWS now strongly recommends IRSA/pod identity and temporary credentials over long‑lived secrets, and having this fixed in the library would make that best‑practice setup work out of the box for people running S3-backed OCFL on EKS or similar platforms. If most consumers of this module are not running on AWS infrastructure, I understand if it’s preferable to keep this in downstream applications instead; I mainly wanted to raise the question in case a library-level change would be helpful for others in a similar environment.

[pwinckles]

I would be in favor of adding a note to the docs that you may want to add a dependency on sts. I'd prefer to not pull in optional dependencies for all users. This is perhaps something that Fedora should be pulling in however. @Surfrdan

This is just my opinion though. If others in the community disagree, we can add the dependency here.

[mohideen]

@pwinckles, I've added a PR for the note #141. I can update if you have any feedback on it.