|
1 | 1 | # @corbits/embedding |
2 | 2 |
|
3 | | -One API for every embedding provider. Point at any OpenAI-compatible `/v1/embeddings` endpoint, swap models with a config change, and get ordered vectors back. |
| 3 | +Batched text embeddings over any OpenAI-compatible `/v1/embeddings` endpoint, built on `@intx/inference` retries and error classification. A retrieval building block for Corbits and Interchange agents that also works standalone. |
4 | 4 |
|
5 | | -OpenAI, Ollama, TEI, vLLM, and Jina all serve the same wire shape. This package is one code path over that shape, with parameters for what varies between providers: |
| 5 | +## Why @corbits/embedding? |
6 | 6 |
|
7 | | -- `dimensions` — Matryoshka truncation for models that support it, including OpenAI `text-embedding-3` (down to 256), Jina v3 (32), and Jina v4 (128). Sent only when set, so models without MRL stay happy. |
8 | | -- `encodingFormat` — `float` (default) or compact `base64`. |
9 | | -- `batchSize` — inputs per request (default 32). Tune it to provider caps and per-request token ceilings. |
| 7 | +1. **One client for every provider.** OpenAI, Ollama, TEI, vLLM and Jina all speak the same wire format. Switching models is a config change. |
| 8 | +2. **Order-safe batching.** Inputs are split into batches, and vectors come back in input order, placed by the index each response echoes. Short or duplicate replies throw instead of misaligning results. |
| 9 | +3. **Interchange retry semantics.** Requests run through `@intx/inference`, so a 429 backs off and a 401 fails the same way it does for inference calls. Failures throw a typed `EmbeddingRequestError`. |
10 | 10 |
|
11 | | -Transport, error classification, and retry build on `@intx/inference`: requests go through the shared `deps.fetch` path, failures classify into `InferenceError` just like chat calls, and `createDefaultRetryPolicy` guides backoff. A 429 from an embeddings endpoint behaves like one from a chat endpoint, and credential failures short-circuit. |
| 11 | +It does not store or search vectors. For that, pair it with [`@corbits/memory`](https://github.com/corbitsdev/corbits-memory). |
12 | 12 |
|
13 | 13 | ## Install |
14 | 14 |
|
15 | 15 | ```bash |
16 | | -bun add @corbits/embedding |
| 16 | +bun add @corbits/embedding @intx/inference@^0.4.0 @intx/types@^0.4.0 |
17 | 17 | ``` |
18 | 18 |
|
19 | | -Runs on Bun >= 1.2 or Node >= 24. The built `dist/` entry is the default import. |
| 19 | +Runs on Bun >= 1.2 or Node >= 24. |
20 | 20 |
|
21 | 21 | ## Quickstart |
22 | 22 |
|
23 | | -This package never touches credential storage or config sources — it takes an |
24 | | -already-built `EmbedConfig`. Where that config comes from is the host's |
25 | | -choice; `@corbits/memory`, for example, reads it straight from |
26 | | -`EMBED_BASE_URL`/`EMBED_MODEL`/`EMBED_API_KEY` env vars, all-or-nothing on the |
27 | | -first two. A host wraps `embedTexts` in a function of its own that builds the |
28 | | -config once (at startup, or per call) and probes dimensionality up front so a |
29 | | -later model swap is caught as a migration, not a silent width mismatch. |
| 23 | +Needs a local Ollama with `nomic-embed-text` pulled. |
30 | 24 |
|
31 | 25 | ```ts |
32 | | -import { createDefaultScheduler } from "@intx/inference"; |
33 | | -import { |
34 | | - embedTexts, |
35 | | - probeEmbedDims, |
36 | | - type EmbedConfig, |
37 | | -} from "@corbits/embedding"; |
38 | | - |
39 | | -export function loadEmbedConfig(): EmbedConfig | undefined { |
40 | | - const baseURL = process.env["EMBED_BASE_URL"]; |
41 | | - const model = process.env["EMBED_MODEL"]; |
42 | | - if (baseURL === undefined || model === undefined) return undefined; |
43 | | - |
44 | | - const apiKey = process.env["EMBED_API_KEY"]; |
45 | | - return { baseURL, model, ...(apiKey !== undefined ? { apiKey } : {}) }; |
46 | | -} |
47 | | - |
48 | | -export async function embed( |
49 | | - texts: readonly string[], |
50 | | - config: EmbedConfig, |
51 | | -): Promise<number[][]> { |
52 | | - const deps = { fetch, scheduler: createDefaultScheduler() }; |
53 | | - return embedTexts(texts, config, { deps }); |
54 | | -} |
55 | | - |
56 | | -// At startup: fail fast if the configured model's width doesn't match |
57 | | -// what's already stored. |
58 | | -const config = loadEmbedConfig(); |
59 | | -if (config !== undefined) { |
60 | | - const deps = { fetch, scheduler: createDefaultScheduler() }; |
61 | | - const dims = await probeEmbedDims(config, { deps }); |
62 | | -} |
| 26 | +import { embedTexts } from "@corbits/embedding"; |
| 27 | + |
| 28 | +const vectors = await embedTexts(["a", "b"], { |
| 29 | + baseURL: "http://localhost:11434/v1", |
| 30 | + model: "nomic-embed-text", |
| 31 | +}); |
| 32 | +console.log(vectors.length, vectors[0]?.length); // 2 768 |
63 | 33 | ``` |
64 | 34 |
|
65 | | -`embedTexts(texts, config, options)` batches sequentially, posts each batch to |
66 | | -`{baseURL}/embeddings`, and returns vectors in input order. `options` carries |
67 | | -`deps` (a host's real `fetch` plus `createDefaultScheduler()`, the production |
68 | | -scheduler), with optional `retryPolicy`, `extractRetryAfterMs`, and `signal`. |
| 35 | +## Where it fits |
69 | 36 |
|
70 | | -## Dimensionality |
| 37 | +[Interchange](https://github.com/faremeter/interchange) runs AI agents as principals (accounts that hold their own identity, permissions and credentials). Corbits packages add what an agent product needs around it. |
71 | 38 |
|
72 | | -`probeEmbedDims(config, { deps })` embeds one probe string and reports the length received. Dimensionality follows the provider and the `dimensions` setting, so callers that persist vectors discover it at startup and treat a model swap as a migration. Representative widths: 1536 for OpenAI `text-embedding-3-small`, 768 for `nomic-embed-text`, 384 for `bge-small-en-v1.5`. |
| 39 | +- **Runs in:** any process: the Interchange hub (the multi-tenant control plane), an agent sidecar (the runtime next to each agent), or a plain script. No hub is required. |
| 40 | +- **Plugs into:** [`@intx/inference`](https://github.com/faremeter/interchange) for transport, retries and error classification, and [`@intx/types`](https://github.com/faremeter/interchange). |
| 41 | +- **Pairs with:** [`@corbits/reranking`](https://github.com/corbitsdev/corbits-reranking) to reorder results and [`@corbits/memory`](https://github.com/corbitsdev/corbits-memory) to store and search vectors. |
73 | 42 |
|
74 | | -## Errors |
| 43 | +## Reference |
75 | 44 |
|
76 | | -Every request failure — transport, HTTP status, or a 200 with an unexpected body — throws `EmbeddingRequestError` (`extends Error`), carrying the classified `InferenceError` as `reason` and the request `url`. A config that fails `EmbedConfigSchema` throws arktype's `TraversalError` before any request. |
| 45 | +| Export | Description | |
| 46 | +| -------------------------------------------- | ----------------------------------------------------------- | |
| 47 | +| `embedTexts(texts, config, options?)` | Embeds the texts and returns one vector per text, in order. | |
| 48 | +| `probeEmbedDims(config, options?)` | Embeds one probe string and returns the vector dimension. | |
| 49 | +| `EmbedConfigSchema`, `EmbedConfig` | Schema and type for `config`. | |
| 50 | +| `EmbedOptions` | Type for `options`. | |
| 51 | +| `EmbeddingRequestError` | Thrown when a request fails. | |
| 52 | +| `RequestDependencies`, `RetryAfterExtractor` | Types for `options.deps` and `options.extractRetryAfterMs`. | |
77 | 53 |
|
78 | | -## Interchange |
| 54 | +### Config |
79 | 55 |
|
80 | | -Built on the shared `@intx/inference` transport, so an embeddings call fails and retries the same way a chat call does: same `deps.fetch` path, same `InferenceError` classification, same `createDefaultRetryPolicy` backoff. A retrieval pipeline typically pairs `probeEmbedDims` at startup with `embedTexts` batches at ingest and query time, as described above. |
| 56 | +| Field | Type | Description | |
| 57 | +| ---------------- | ---------------------- | ------------------------------------------------------------------------ | |
| 58 | +| `baseURL` | `string` | Server root with its version path, such as `http://host:11434/v1`. | |
| 59 | +| `model` | `string` | Model name. | |
| 60 | +| `apiKey` | `string?` | Sent as a bearer token. | |
| 61 | +| `dimensions` | `number?` | Requested output dimension. Sent only when set; not all models honor it. | |
| 62 | +| `encodingFormat` | `"float" \| "base64"?` | Wire format. Defaults to `float`. Results are always `number[]`. | |
| 63 | +| `batchSize` | `number?` | Texts per request. Defaults to 32. | |
| 64 | +| `timeoutMs` | `number?` | Per-request timeout. Defaults to 30000. | |
81 | 65 |
|
82 | | -## Versioning |
| 66 | +### Options |
83 | 67 |
|
84 | | -Semver. Releases run `bun run build && npm publish` with green CI. |
| 68 | +| Field | Description | |
| 69 | +| --------------------- | ------------------------------------------------------------------------------- | |
| 70 | +| `deps` | `{ fetch, scheduler }`. Defaults to global `fetch` and Interchange's scheduler. | |
| 71 | +| `retryPolicy` | Retry policy. Defaults to Interchange's policy. | |
| 72 | +| `extractRetryAfterMs` | Reads the server's retry delay. Defaults to parsing `Retry-After`. | |
| 73 | +| `signal` | Aborts all pending requests. | |
85 | 74 |
|
86 | | -## Development |
| 75 | +### Errors |
87 | 76 |
|
88 | | -```bash |
89 | | -bun install |
90 | | -bun test ./src |
91 | | -bunx tsc --noEmit |
92 | | -``` |
| 77 | +A failed request throws `EmbeddingRequestError` with a classified `reason` and the request `url`. An invalid config throws before any request is sent. |
| 78 | + |
| 79 | +### Dimensions |
| 80 | + |
| 81 | +Each model has a fixed output dimension: 768 for `nomic-embed-text`, 1536 for OpenAI `text-embedding-3-small`. If you store vectors, call `probeEmbedDims` at startup. Changing models changes the dimension, so treat it as a schema migration. |
| 82 | + |
| 83 | +## Using with Interchange |
| 84 | + |
| 85 | +`@intx/inference` and `@intx/types` are peer dependencies, so your host's Interchange version supplies them. |
| 86 | + |
| 87 | +To share your host's retry scheduler, pass it as `options.deps.scheduler` along with `fetch`. |
| 88 | + |
| 89 | +## Upgrading from 0.1 |
| 90 | + |
| 91 | +- Install `@intx/inference` and `@intx/types` (^0.4.0) yourself. They are now peer dependencies. |
| 92 | +- `ModelRequestError` is renamed to `EmbeddingRequestError`. |
| 93 | +- `runJSONRequest`, `extractRetryAfterMs` and `RunRequestOptions` are no longer exported. To change how retry waits are read, pass `options.extractRetryAfterMs`. |
| 94 | +- Config and returned vectors are unchanged. |
93 | 95 |
|
94 | 96 | ## License |
95 | 97 |
|
96 | | -LGPL-2.1-only — see [`LICENSE`](LICENSE). |
| 98 | +[LGPL-2.1-only](https://github.com/corbitsdev/corbits-embedding/blob/main/LICENSE) |
0 commit comments