[gavin-thompson-postman]

Stacked on #1139. This branch contains #1139's commits. Until that merges, review only the three commits on top of it, b6c12d86 (delete markers), b40b0691 (review fixes) and d4b1917b (AccessDenied when a version listing is refused): diff against #1139. I'll rebase once #1139 is in.

Azure keeps no delete markers. This emulates them the way GCloudBlobStore already does, and only when s3proxy.azureblob.versioning=true. An unversioned store is unchanged.

Delete markers. A delete naming no version writes an empty blob carrying s3proxy_delete_marker=true. Azure versions it over the current blob like any other write, and S3Proxy reports it as the marker S3 would lay down (DeleteMarker: true plus its version id). A key whose current version is a marker:

• GET / HEAD: 404 NoSuchKey with x-amz-delete-marker: true and the marker's version id. Naming the marker's version returns 405.
• ListObjects: omits the key. ListObjectVersions reports the marker as a DeleteMarker entry.
• Copy: from the key, NoSuchKey. Naming the marker's version, InvalidRequest.
• If-None-Match: * on a put or multipart complete: replaces the marker, conditioned on its ETag, instead of failing.
• DeleteObjects: with versioning on, it goes key by key through removeBlob, because a batch can't write blobs. It reports each marker it writes.

Promotion. Azure can't make an existing version current again, so deleting the current version by id still leaves the key with no current version. S3 promotes the newest remaining version. Reads, copies and listings now treat that version as current, as GCloudBlobStore.resolveVersion does when its live generation is deleted. Deleting a marker this way is how S3 undeletes a key.

This only works together with markers. A plain delete now writes a marker rather than demoting the current blob, so the newest remaining version is never content a plain delete just removed. If-None-Match: * fails, as on S3, on a key whose current version was deleted by id while a version remains.

Costs:

• With versioning on, deletes become writes. DeleteObjects stops using Azure's batch API.
• Listings request blob metadata.
• A read that finds no current version, or an If-None-Match: * write, makes one extra version listing.
• Copies make one extra properties call.

Not covered: suspending versioning per bucket (Azure versioning is account-wide), so a fresh bucket still reports Enabled. A plain listing still omits a key whose current version was deleted by id, the same gap GCloudBlobStore has.

Testing:

• Unit tests: marker rows in ListObjectVersions, newest-is-latest when no version is current, and markers dropped from plain listings. The full mvn test passes.
• Against a real versioned storage account: before relying on the doubles, I confirmed four things:
  • a marker upload gets its own version id
  • version and flat listings return metadata on every row
  • openInputStream returns the marker's metadata
  • deleting a current marker by id leaves the earlier versions intact
• End to end through S3Proxy:
  • marker on delete, and 404 with the marker headers
  • undelete by removing the marker
  • markers in ListObjectVersions and from DeleteObjects
  • deleting the current version by id serves the previous one
  • deleting both versions of a key with DeleteObjects leaves nothing
  • with versioning off, results are unchanged

Fix: refused version listings (d4b1917b). With versioning on, a read that finds no current version lists the key's versions to find one to promote. Copies and If-None-Match: * writes do the same. A credential that can read and write blobs but can't list the container (for example an account SAS with srt=o, which Azure refuses with 403 AuthorizationResourceTypeMismatch) got 500 InternalError from:

• a GET or HEAD of a missing key
• a copy from a missing source
• an If-None-Match: * put of a missing key

These now answer 403 AccessDenied, and the message names the missing permission: RBAC containers/blobs/read, or for a SAS, permission l with resource type c. S3 does the same: it answers a missing key with 403, not 404, when the caller lacks list permission. A 404 would also wrongly say the key is absent when a version might remain to be promoted. Other listing failures are translated as before. The fix adds S3Exceptions.accessDenied, which #1234 reuses.

• Unit tests (local fake Azure endpoint in AzureBlobStoreTest): with the listing refused, GET, HEAD, copy and If-None-Match: * PUT answer AccessDenied and nothing is written. With the listing allowed, GET, HEAD and copy still serve the newest remaining version. The full mvn test (661 tests) and mvn verify -DskipTests -Pjdeps pass on JDK 25.
• Real account, account SAS srt=o, racwdlx: GET of a missing key, HEAD of a missing key, a copy from a missing source, and a GET of a key whose current version was deleted by id each went from 500 to 403 AccessDenied. Reads of existing keys are still 200. (Before is the earlier build of this stack.) With the account key, a missing key is still 404 NoSuchKey, and the marker, undelete and promotion checks pass on this branch alone.

Generated with Claude Code

[gavin-thompson-postman]

A Copilot review I ran outside this PR, not on the PR itself, found three issues. I reproduced all three against a real Azure storage account, checked the expected behaviour against AWS S3, and fixed them in b40b069:

1. User metadata could fake a delete marker. A PUT with x-amz-meta-s3proxy_delete_marker: true succeeded, but GET then returned 404 and listings hid the key. PUT, copy with REPLACE, and CreateMultipartUpload now drop the store's own metadata names (s3proxy_delete_marker, s3proxy_target_blob_name) in any letter case, as multipart already did for its target name.
2. Promoted keys were missing from ListObjectsV2. After the current version was deleted by id, including undeleting a key by deleting its delete marker, GET served the newest remaining version but ListObjectsV2 left the key out. With versioning on, the listing now includes versions and reports each key by its latest version, skipping keys whose latest version is a marker. A key's versions can span Azure pages, so paging resumes by key name, as listVersions already does, rather than by Azure's continuation token.
3. Versions and markers could list out of order. Azure lists Last-Modified in whole seconds, so a PUT followed by a DELETE in the same second tied, and the handler's merge put the older version ahead of its delete marker. On Azure this happened for 9 of 10 keys. Versions are now timed by their version id, which Azure stamps to 100ns, and compareOrder compares whole instants, with the latest entry first on a tie. The XML still reports whole seconds.

After the fix, Azure matches S3 for all three. That includes PUT, DELETE, PUT within one second, and paging over promoted, undeleted and hidden keys with max-keys 2 and 0, with and without a delimiter, and with V1 markers. New unit tests cover each case.