Skip to content

fix(local): decode binary responses for subtype wildcards - #9354

Open
miruky wants to merge 4 commits into
aws:developfrom
miruky:fix/binary-response-subtype-wildcards
Open

miruky wants to merge 4 commits into
aws:developfrom
miruky:fix/binary-response-subtype-wildcards

Conversation

@miruky

@miruky miruky commented Oct 21, 2026 •

Copy link
Copy Markdown

Which issue(s) does this change fix?

Reproduced on develop at 003fb8bd6969ffd93c78336a9a9c86eef0ade041. Related work: #8018 addresses request-body encoding.

Why is this change necessary?

With BinaryMediaTypes: ["image/*"], a REST API Lambda proxy response with Content-Type: image/png and isBase64Encoded: true reaches the client as Base64 text in sam local start-api. The response check only recognizes exact MIME types and */*.

API Gateway supports wildcard binary media types. The local response path should recognize a matching subtype range as well.

How does it address the issue?

Recognize the selected MIME type's type/* range in the response decoding predicate. Add unit coverage for matching ranges, mismatches, a false encoding flag, and a missing selection. Functional tests drive the real local Flask service with a stub Lambda runner, checking PNG, JPEG and PDF bytes along with exact-match and */* controls.

What side effects does this change have?

Matching subtype ranges now enable binary response decoding when isBase64Encoded is true.

Validation

Revalidated at f8491eb after merging develop at 003fb8b. The response patch remains unchanged.

  • macOS / Python 3.12.13: make pr passed, with 9,662 unit tests passed, 28 skipped and coverage 94.20%. All local unit tests plus the functional regression also passed: 1,354 tests.
  • Linux/amd64 Docker / Python 3.10.22: the same local scope passed 1,354 tests; repository Ruff, Black and configured mypy checks passed. Unit/static-check containers used --network none and dummy credentials.
  • Real sam local start-api with a Python 3.12.15 arm64 Lambda container: image~1* returns the original PNG bytes with this patch and Base64 text on the base. Text/plain, a false encoding flag and an unmatched Accept remain Base64 in both revisions (4 checks per revision).
  • A 10-case Flask matrix per revision covered subtype ranges, exact/global controls, mismatches, flags and template media-type normalization. The same functional regression fails in 3 cases on the base and passes all 5 with the fix.

The local CLI/container checks used dummy credentials and Python socket guards. No live AWS comparison or deployment was run.

Mandatory Checklist

  • Read the generative AI contribution guidelines
  • Add input/output type hints to new functions/methods
  • Design document considered — not required for this existing response-decoding fix
  • Write/update unit tests
  • Integration testing considered — ran local CLI/Lambda-container checks against the base and this patch
  • Write/update functional tests
  • make pr passes
  • Reproducible requirements considered — no dependency changes
  • Documentation considered — no configuration or CLI interface changes

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@miruky
miruky requested a review from a team as a code owner October 21, 2026 00:01
@github-actions github-actions Bot added area/local/start-api sam local start-api command area/local/invoke sam local invoke command area/local/start-invoke pr/external stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. labels Oct 21, 2026
@aws-sam-tooling-bot

aws-sam-tooling-bot Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🧪 Hands-on test plan (not run: external contributor)

Commit f8491eb. A maintainer can run these checks; nothing here has been tested.

Summary: Run the PR's new tests and the existing apigw unit tests. Then run an in-process matrix that drives LocalApigwService through the Flask test client with a stub Lambda runner and classifies each response body as BINARY or BASE64, comparing PR to merge base. Finally run real sam local start-api with BinaryMediaTypes image~1* and a Python 3.12 function to confirm PNG/JPEG bytes are returned decoded and text/pdf controls stay Base64.

For a maintainer to check:

  • Scenario S4 pulls the public Lambda Python 3.12 emulation image from public.ecr.aws (a registry SAM CLI already uses for sam local).

Needs cloud resources:

  • Compare against a real API Gateway REST API with BinaryMediaTypes image/* to confirm parity for: uppercase subtype (Image/PNG), Content-Type with parameters, missing Accept header, and a Content-Type…

Planned checks (4):

  1. PR's new unit and functional tests plus the existing apigw unit tests pass on the PR checkout (checks for regressions in the response path)
  2. Main path and controls through the real Flask service: image/* and application/* ranges decode; exact type and */* still decode; mismatched type and …
  3. Edge cases the new predicate touches: Content-Type with parameters, no Accept header, Content-Type without a slash (the split builds 'image/*' from '…
  4. End to end: real sam local start-api with BinaryMediaTypes image~1* (template normalization) and a Python 3.12 Lambda returning isBase64Encoded true;…
Commands
python -m venv .venv
. .venv/bin/activate && pip install --upgrade pip && SAM_CLI_DEV=1 pip install -e '.[dev]'
# S1
. .venv/bin/activate && python -m pytest -q -p no:cacheprovider tests/unit/local/apigw tests/unit/local/apigw/test_binary_response_media_patterns.py tests/functional/local/test_binary_response_media_types.py 2>&1 | tail -n 30
# S2
cat > ho_check.py <<'EOF'
import base64, json, sys
from unittest.mock import Mock
from samcli.lib.providers.provider import Api
from samcli.local.apigw.local_apigw_service import LocalApigwService
from samcli.local.apigw.route import Route
P = b"\x89PNG\r\n\x1a\n\x00\xff"
def run(cid, bt, ct, accept, enc=True, et=Route.API):
    def invoke(name, event_str, stdout=None, stderr=None, tenant_id=None, **kw):
        stdout.write_str(json.dumps({"statusCode": 200, "headers": {"Content-Type": ct}, "body": base64.b64encode(P).decode(), "isBase64Encoded": enc}))
    r = Mock(); r.is_debugging.return_value = False; r.invoke.side_effect = invoke
    api = Api(routes=[Route(function_name="F", path="/x", methods=["GET"], event_type=et)])
    if bt: api.binary_media_types_set.add(bt)
    s = LocalApigwService(api=api, lambda_runner=r); s.create()
    h = {"Accept": accept} if accept is not None else {}
    try:
        resp = s._app.test_client().get("/x", headers=h)
        kind = "BINARY" if resp.data == P else ("BASE64" if resp.data == base64.b64encode(P) else "OTHER")
        print(f"case={cid} status={resp.status_code} body={kind}")
    except Exception as e:
        print(f"case={cid} ERROR {type(e).__name__}: {e}")
EOF
cat >> ho_check.py <<'EOF'
C = {
 "wild_png": ("image/*", "image/png", "image/png"),
 "wild_jpeg_acceptall": ("image/*", "image/jpeg", "*/*"),
 "wild_pdf": ("application/*", "application/pdf", "application/pdf"),
 "mismatch_pdf": ("image/*", "application/pdf", "application/pdf"),
 "flag_false": ("image/*", "image/png", "image/png", False),
 "exact_png": ("image/png", "image/png", "image/png"),
 "global": ("*/*", "image/png", "image/png"),
 "ct_params": ("image/*", "image/png; charset=binary", "image/png"),
 "no_accept": ("image/*", "image/png", None),
 "ct_no_slash": ("image/*", "image", "*/*"),
 "upper_ct": ("image/*", "Image/PNG", "image/png"),
 "http_api_text": ("", "text/plain", "*/*", True, Route.HTTP),
}
for k in (sys.argv[1:] or C):
    run(k, *C[k])
EOF
. .venv/bin/activate && python ho_check.py wild_png wild_jpeg_acceptall wild_pdf mismatch_pdf flag_false exact_png global
# S3
. .venv/bin/activate && python ho_check.py ct_params no_accept ct_no_slash upper_ct http_api_text
# S4
mkdir -p ho_sam/src && cat > ho_sam/template.yaml <<'EOF'
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Globals:
  Api:
    BinaryMediaTypes:
      - image~1*
Resources:
  Fn:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.12
      Handler: app.handler
      CodeUri: src/
      Timeout: 10
      Events:
        Img:
          Type: Api
          Properties:
            Path: /img
            Method: get
EOF
cat > ho_sam/src/app.py <<'EOF'
import base64
P = b"\x89PNG\r\n\x1a\n\x00\xff"
def handler(event, context):
    q = event.get("queryStringParameters") or {}
    return {"statusCode": 200, "headers": {"Content-Type": q.get("ct", "image/png")}, "body": base64.b64encode(P).decode(), "isBase64Encoded": True}
EOF
cat > ho_sam/classify.py <<'EOF'
import sys, base64
P = b"\x89PNG\r\n\x1a\n\x00\xff"
d = open(sys.argv[1], "rb").read()
k = "BINARY" if d == P else ("BASE64" if d == base64.b64encode(P) else "OTHER:" + repr(d[:80]))
print(f"case={sys.argv[2]} http={sys.argv[3]} body={k}")
EOF
cat > ho_sam/run.sh <<'EOF'
set -u
cd ho_sam
SAM_CLI_TELEMETRY=0 AWS_DEFAULT_REGION=us-east-1 sam local start-api -t template.yaml --port 3011 > server.log 2>&1 &
PID=$!
for i in $(seq 1 90); do curl -s -o /dev/null --max-time 2 http://127.0.0.1:3011/ && break; sleep 2; done
for spec in 'png|image%2Fpng|image/png' 'jpeg|image%2Fjpeg|*/*' 'text|
… (truncated)

🤖 Automated hands-on test — not a review approval · model claude-opus-5.5 · commit f8491eb

History (3)
Commit Result When (UTC) Build
f8491eb Plan only (not run) 2026-10-29 14:40
59df766 Plan only (not run) 2026-10-27 13:18
0688353 Plan only (not run) 2026-10-23 23:16

@miruky

miruky commented Oct 25, 2026

Copy link
Copy Markdown
Author

Revalidated 0688353 after the develop merge:

  • make pr passed: 9,598 unit tests, 28 skips, coverage 94.21%. The full unit suite plus the new functional cases passed 9,603 tests.
  • Linux/amd64 Docker on Python 3.10.22 passed 321 related tests and the repository lint, formatting and configured type checks.
  • A real sam local start-api with the Python 3.12.15 arm64 Lambda container returned raw PNG bytes for image~1*; the base returned Base64 text. Text/plain, a false encoding flag and an unmatched Accept stayed unchanged in both revisions. The 10-case Flask matrix also passed on both revisions with their expected outputs.

For the synthetic helper edges, parameters match and a slashless mocked MIME type returns True, as the plan notes. With an actual slashless Content-Type/Accept request, Werkzeug selects no matching MIME type and the response remains Base64 on both revisions. I left the existing malformed-type and case-sensitive behavior unchanged.

All validation ran locally with dummy credentials. The PR description now has the current validation results. Could a maintainer approve the queued workflows when convenient?

@miruky

miruky commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

Merged the latest develop (003fb8b), keeping the three-file response fix unchanged.

On f8491eb, make pr passes (9,662 unit tests, 28 skipped, 94.20% coverage). Linux/amd64 Python 3.10 passes all 1,354 local tests and the static checks. The real Lambda-container comparison still returns PNG bytes with the fix and Base64 on the base, while the three controls are unchanged.

The upstream workflows still need maintainer approval before they can run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/local/invoke sam local invoke command area/local/start-api sam local start-api command area/local/start-invoke pr/external stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants