feat(ByteLevel): skip per-byte transform for printable-ASCII tokens#2038

Open

KimYannn wants to merge 3 commits intohuggingface:mainfrom

KimYannn:feat/byte-level-printable-ascii-fast-path

Commits on Apr 26, 2026

ByteLevel: skip per-byte transform for printable-ASCII-only tokens

KimYannn
and
claude
committed

Commits on Apr 27, 2026

fmt: cargo fmt on byte_level tests
KimYannn
committed

Commits on Apr 28, 2026

Merge branch 'main' into feat/byte-level-printable-ascii-fast-path
KimYannn
authored