Skip to content

Detect response byte-order marks independently of string encoding - #327

Open
OskarEichler wants to merge 1 commit into
ruby:masterfrom
OskarEichler:codex/bom-http
Open

Detect response byte-order marks independently of string encoding#327
OskarEichler wants to merge 1 commit into
ruby:masterfrom
OskarEichler:codex/bom-http

Conversation

@OskarEichler

Copy link
Copy Markdown

Summary

Compare BOM bytes with getbyte so detection does not depend on the String encoding tag. Preserve the supported UTF-16BE, UTF-16LE and UTF-8 signatures and leave the body bytes untouched.

Reproduction

Net::HTTPOK.new('1.1', '200', 'OK').send(:check_bom, ['feff0061'].pack('H*')) returns nil before this change and Encoding::UTF_16BE afterward. The corresponding fffe and efbbbf signatures reproduce the same issue. Actual response bodies arrive as binary strings.

Verification

  • Ruby 4.0.6 through rbenv; existing bundle exec rake test: 201 tests, 914 assertions, zero failures/errors, both baseline and this isolated patch.
  • 39 focused checks cover all three signatures under four encoding tags, short/non-BOM input, unchanged bytes and complete response-body encoding.
  • Ruby syntax and git diff --check pass. Supplemental Lint scan retains the same 25 pre-existing findings; no new findings by file/cop.
  • No test/spec files were added or modified, per this contribution's explicit no-new-tests constraint. Focused reproductions were run externally.

Compatibility and limitations

No API or dependency change. Automatic body encoding now recognizes these existing supported BOMs in binary responses. UTF-32 detection is outside this patch.

Based on master 23e859e92e10e43cad520fcacac3fd43640dba7b. Optional external OpenSSL tests and other Ruby/OS combinations were not run. Existing related issues/PRs were checked; this is a focused correctness change, not a claim of exhaustive coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant