Skip to content

Merge branch 'huggingface:main' into mf/performance-improvements

3445a78
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
Open

feat: Optimize BPE tokenization: sharded cache, packed merge keys, FxHash (10-15% speedup) #1967

Merge branch 'huggingface:main' into mf/performance-improvements
3445a78
Select commit
Loading
Failed to load commit list.

Annotations

5 warnings
build (ubuntu-latest)
succeeded Apr 13, 2026 in 5m 5s