[stephantul]

This PR adds faster pooling. Each text is now truncated, tokenized and pooled in one parallel pass. FastWordPiece encodes into a per-thread buffer that is reused across texts. encode_single pools directly instead of building a one-element batch. Pooling uses fixed-size loops for 64/128/256/384/512/768/1024 dims, and a general loop otherwise.

Measured against main on an Apple M3 Pro, default build:

Model (dim)	Batched queries	Batched docs	encode_single queries
potion-base-8M (256)	1.23x	1.31x	1.37–1.43x
potion-retrieval-32M (512)	1.35x	1.24x	1.21–
static-retrieval-mrl-en-v1 (1024)	1.23x	1.19x	1.17–1.18x

[greptile-apps]

Confidence Score: 4/5

[Medium impact] The PR changes the shared text-to-embedding path and adds local comparison tools.

The PR appears safe to merge, with non-blocking improvements needed for Unigram efficiency and benchmark reliability.

_{Reviews (1) · Last reviewed commit: "merge" · Reviewed by Greptile}

[Pringled]

Gogogo