Real-world (batch × input_length) tokenizer benchmark + cross-library leaderboard#2030

ArthurZucker wants to merge 1 commit intomainhuggingface/tokenizers:mainfrom

full-benchhuggingface/tokenizers:full-benchCopy head branch name to clipboard