Skip to content

Fix UTF-8 handling in VimHTML - #3

Open
mateu wants to merge 3 commits into
rjbs:mainfrom
mateu:koan/upstream-vimhtml-utf8-fix
Open

mateu wants to merge 3 commits into
rjbs:mainfrom
mateu:koan/upstream-vimhtml-utf8-fix

Conversation

@mateu

@mateu mateu commented Apr 17, 2026

Copy link
Copy Markdown

This fixes UTF-8 mojibake in Pod::Elemental::Transformer::VimHTML when highlighted input arrives as already-decoded Perl text.

In the failing case, content like guillemets in #!vim perl blocks can render as «...» instead of «...».

The fix keeps the change narrow:

  • normalize likely double-encoded-looking decoded input before passing it to Text::VimColor
  • avoid pre-encoding already-decoded Perl strings to octets
  • only decode HTML output if it is not already flagged as UTF-8
  • add an explicit use Carp qw(confess); import while touching the module

Downstream context:

Happy to adjust the shape if you’d prefer a different approach.

@mateu

mateu commented Apr 17, 2026

Copy link
Copy Markdown
Author

I added a narrow upstream regression test for this too:

  • t/vimhtml_utf8.t

It exercises Pod::Elemental::Transformer::VimHTML directly, checks decoded Perl text and UTF-8 octet input, and asserts we preserve guillemets without leaving «/» mojibake markers behind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant