Skip to content

Commit 219239d

Browse files
TheGreatAxiosTheGreatAxios
authored
docs: registry-first README for @corbits/embedding (CL-8775) (#4)
* docs: registry-first README for @corbits/embedding Replace the git-install README with the Interchange section shape: Install / Use / Full example / How it works / Contributing. * docs: align README headings with Interchange (CL-8775) Use Quickstart / How it works / Development / License instead of Install / Use / Full example / Contributing. * docs: product-voice README (CL-8775) * docs: add PRODUCT, ARCHITECTURE, IMPLEMENTATION for @corbits/embedding (CL-8775) (#5)
1 parent 46e9247 commit 219239d

4 files changed

Lines changed: 294 additions & 51 deletions

File tree

‎ARCHITECTURE.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Architecture
2+
3+
The package is two layers: a one-shot JSON transport, and an embedding client
4+
built on it. There is no provider registry. Every call is HTTP POST to
5+
`{baseURL}/embeddings`.
6+
7+
## Surfaces
8+
9+
Callers import from `@corbits/embedding`. The barrel is the public surface:
10+
11+
| Export | Role |
12+
| --- | --- |
13+
| `embedTexts` | Batch texts, post sequentially, return `number[][]` in input order |
14+
| `probeEmbedDims` | Embed one probe string; report the vector length that came back |
15+
| `EmbedConfig` / `EmbedConfigSchema` | Endpoint, model, and optional knobs |
16+
| `EmbedOptions` | Harness deps plus optional retry, `Retry-After` override, abort |
17+
| `runJSONRequest` | Classified, retried JSON POST (sibling clients reuse this) |
18+
| `extractRetryAfterMs` | Default `Retry-After` reader (seconds or HTTP-date) |
19+
| `ModelRequestError` | Transport, HTTP, or non-JSON 200 — `reason` is the classified `InferenceError` |
20+
21+
Quickstart shape (matches the README):
22+
23+
```ts
24+
import { createDefaultScheduler } from "@intx/inference";
25+
import { embedTexts, probeEmbedDims } from "@corbits/embedding";
26+
27+
const deps = { fetch, scheduler: createDefaultScheduler() };
28+
const [vector] = await embedTexts(
29+
["hello world"],
30+
{ baseURL: "http://localhost:11434/v1", model: "nomic-embed-text" },
31+
{ deps },
32+
);
33+
```
34+
35+
`probeEmbedDims(config, { deps })` is the same config and deps; it is not a
36+
separate protocol.
37+
38+
## Request dependencies
39+
40+
The transport reads only `fetch` and `scheduler` from the harness
41+
(`RequestDependencies`). That is deliberately narrower than Interchange
42+
`Dependencies`, whose `adapters` registry a caller should not have to
43+
assemble in order to embed a string. A real `Dependencies` still satisfies
44+
the pick structurally.
45+
46+
Time for retry backoff comes from `deps.scheduler`, not wall-clock globals,
47+
so a virtual-clock test scheduler drives the loop deterministically. The
48+
HTTP-date form of `Retry-After` is the exception: it names an absolute
49+
instant and is resolved against wall time.
50+
51+
## Control flow
52+
53+
```
54+
embedTexts(texts, config, options)
55+
→ split on batchSize (default 32)
56+
→ for each batch, sequentially:
57+
build POST {baseURL}/embeddings
58+
runJSONRequest (classify + retry)
59+
parse body: place by echoed index, reject short / duplicate / OOB
60+
→ concatenate batch results in input order
61+
```
62+
63+
`probeEmbedDims` is `embedTexts` of one fixed probe string, then
64+
`vector.length`.
65+
66+
Batches are sequential, not concurrent: the retry policy is per request, not
67+
per burst, and a rate-limited provider must not see the whole corpus at once.
68+
69+
## Reply integrity
70+
71+
The OpenAI-compatible reply is not promised in request order. Placement is
72+
by the echoed `index`, never by array position. A reordered reply is
73+
reordered back; a short reply, a duplicate index, or a missing slot is
74+
rejected. Callers pair returned vectors with their own texts positionally —
75+
a silent mispair would persist the wrong vector.
76+
77+
`embedding` on the wire is either `number[]` (`float`) or a base64-packed
78+
little-endian float32 buffer (`base64`). The client always returns
79+
`number[]`.
80+
81+
## Transport vs `runInference`
82+
83+
`runInference` is turn-shaped and streams SSE. An embedding is one-shot JSON.
84+
What carries over is everything around the body: the single `deps.fetch`
85+
path, the `InferenceError` classifiers (`classifyHTTPError`,
86+
`classifyNetworkError`, `classifyAbortError`, `classifyProtocolMismatch`),
87+
and `createDefaultRetryPolicy` (back off retryables, abort the rest). A 429
88+
is therefore classified and backed off as a chat 429 would be; a
89+
`credential_failure` aborts immediately.
90+
91+
`runJSONRequest` is duplicated, modulo comments, in `@corbits/reranking`. It
92+
is not shared infrastructure yet — the intended home is `@intx/inference` as
93+
a non-streaming sibling of `runInference`. Until then, `ModelRequestError` is
94+
a distinct class in each package: `instanceof` does not hold across the two.
95+
Catch on `error.name === "ModelRequestError"`.
96+
97+
## Failure modes
98+
99+
- Transport, HTTP status, or a 200 whose body is not JSON →
100+
`ModelRequestError` with classified `reason` and the URL.
101+
- Malformed embeddings JSON, wrong count, bad index → thrown `Error` (not
102+
`ModelRequestError`); the protocol succeeded, the payload did not.
103+
- `batchSize` not an integer `>= 1` → `RangeError` before any request
104+
(`i += size` of 0 would never advance).
105+
- Empty `texts` → `[]`, no request.
106+
- Caller `signal` or per-attempt timeout (default 30s) abort the attempt;
107+
aborting mid-delay wakes immediately and the next attempt fails its
108+
entry-time signal check.

‎IMPLEMENTATION.md‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Implementation
2+
3+
## Package
4+
5+
`@corbits/embedding` is an ESM TypeScript package. Development runtime is
6+
Bun `>= 1.2.0`. Node `>= 24` consumes built `dist/`; native Node does not
7+
load this package's TypeScript source.
8+
9+
```json
10+
"exports": {
11+
".": {
12+
"intx-src": "./src/index.ts",
13+
"types": "./dist/index.d.ts",
14+
"default": "./dist/index.js"
15+
}
16+
}
17+
```
18+
19+
`bun run build` is `tsc -p tsconfig.build.json` (`outDir: dist`,
20+
`rootDir: src`, tests excluded). Published files are `dist`, `README.md`,
21+
and `LICENSE`. Install from the registry:
22+
23+
```bash
24+
npm add @corbits/embedding
25+
pnpm add @corbits/embedding
26+
yarn add @corbits/embedding
27+
bun add @corbits/embedding
28+
```
29+
30+
## Dependencies
31+
32+
| Package | Use |
33+
| --- | --- |
34+
| `@intx/inference` | `BuiltRequest`, `createDefaultRetryPolicy`, error classifiers, `createDefaultScheduler` (caller-side) |
35+
| `@intx/types` | `RetryPolicy`, `InferenceError` |
36+
| `@intx/log` | Transitive Interchange surface (not imported by this package's sources) |
37+
| `arktype` | `EmbedConfigSchema` and the embeddings reply shape |
38+
39+
Dev: `@intx/inference-testing`, `@types/bun`, `prettier`, `typescript` 5.9.
40+
41+
## HTTP
42+
43+
Each batch is:
44+
45+
- `POST {baseURL}/embeddings` (`baseURL` already includes the version
46+
prefix, e.g. `http://localhost:11434/v1` or `https://api.openai.com/v1`)
47+
- `content-type: application/json`
48+
- `authorization: Bearer {apiKey}` only when `apiKey` is set
49+
- `redirect: "manual"`
50+
- body:
51+
52+
```json
53+
{
54+
"model": "<config.model>",
55+
"input": ["…"],
56+
"dimensions": 256,
57+
"encoding_format": "float"
58+
}
59+
```
60+
61+
`dimensions` and `encoding_format` are omitted unless configured. Models
62+
without Matryoshka reject `dimensions` outright, so an unset knob must not
63+
appear. Client field is `encodingFormat`; wire field is `encoding_format`
64+
(`float` | `base64`). Default `batchSize` is 32 (OpenAI rejects arrays over
65+
2048). Default per-attempt timeout is 30s (`AbortSignal.timeout`, combined
66+
with a caller `signal` via `AbortSignal.any`).
67+
68+
A 200 is read as text then `JSON.parse`. `res.json()` is not used: an HTML
69+
interstitial or truncated body in front of a local model server must classify
70+
as protocol mismatch, not escape as `SyntaxError`.
71+
72+
## arktype
73+
74+
`EmbedConfigSchema` (`src/embed.ts`):
75+
76+
- required: `baseURL: string`, `model: string`
77+
- optional: `apiKey`, `dimensions` (`number > 0`), `encodingFormat`
78+
(`'float'|'base64'`), `batchSize` (`number.integer >= 1`), `timeoutMs`
79+
(`number > 0`)
80+
81+
Reply: `{ data: { index: number, embedding: number[] | string }[] }`.
82+
Base64 strings decode with `atob` → `Uint8Array` → little-endian
83+
`Float32Array` → `number[]`.
84+
85+
`embedTexts` also range-checks `batchSize` at runtime so a value that
86+
bypassed the schema cannot stall `batches` (`i += size`).
87+
88+
## Errors and retry
89+
90+
`runJSONRequest` (`src/request.ts`) loops:
91+
92+
1. `deps.fetch` POST
93+
2. non-OK → `classifyHTTPError(status, detail, detail, retryAfterMs)`
94+
3. network throw → `classifyNetworkError` or `classifyAbortError`
95+
4. JSON parse fail → `classifyProtocolMismatch`
96+
5. `retryPolicy` (default `createDefaultRetryPolicy`) → `abort` throws
97+
`ModelRequestError`, else sleep `delayMs` on `deps.scheduler`
98+
99+
`extractRetryAfterMs` reads `Retry-After` as seconds or HTTP-date, clamped
100+
at zero. Callers override via `EmbedOptions.extractRetryAfterMs` (same
101+
signature as Interchange's unexported extractor; re-declared so this
102+
package depends only on the published surface).
103+
104+
`ModelRequestError.name` is `"ModelRequestError"`. Discriminate on the
105+
name when catching both this package and `@corbits/reranking`.
106+
107+
## Tests
108+
109+
`bun run test` is `bun test ./src`. Coverage is the client contract: empty
110+
input, URL and body knobs, auth header presence, batch split and order,
111+
`batchSize` rejection, malformed / short / duplicate-index replies,
112+
index-not-position placement, 429 retry, 401 abort, `probeEmbedDims`,
113+
overridable `Retry-After`.

‎PRODUCT.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Product
2+
3+
`@corbits/embedding` is the client you call when you need vectors from an
4+
OpenAI-compatible `/v1/embeddings` endpoint. Given a `baseURL`, a model, and
5+
some texts, `embedTexts` batches them, posts each batch, validates the reply,
6+
and returns vectors in input order.
7+
8+
## Why it exists
9+
10+
Embedding is not a chat turn. Callers persist vectors next to text and later
11+
retrieve by similarity. A wrong pairing, an assumed dimension, or a one-off
12+
HTTP client that does not retry a 429 the way a chat call does all corrupt
13+
that store. This package is the one place that does the boring work: batch,
14+
validate, place by echoed `index`, and fail loudly instead of silently
15+
mispairing.
16+
17+
One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM, and Jina
18+
all serve `/v1/embeddings`. What differs between them is parameters
19+
(`dimensions`, `encodingFormat`, `batchSize`), not shape. Swap models without
20+
rewriting the call site — but treat a swap as a migration, because output
21+
dimensionality is not knowable from a model name.
22+
23+
## Who it is for
24+
25+
- Hosts that persist embeddings (memory, search, recall) and must discover
26+
dimensionality before writing a row.
27+
- Sibling one-shot JSON clients that want the same classified, retried request
28+
path without assembling an inference `AdapterRegistry`.
29+
- Anyone already holding an Interchange harness `Dependencies` — only `fetch`
30+
and `scheduler` are required; a full harness object satisfies that.
31+
32+
## Goals
33+
34+
- Callers can `embedTexts` and `probeEmbedDims` from `@corbits/embedding`
35+
with `{ fetch, scheduler }` and an `EmbedConfig`.
36+
- A 429 from an embedding endpoint behaves as one from a chat endpoint; a
37+
credential failure aborts immediately.
38+
- Dimensionality is discovered, never assumed. `probeEmbedDims` embeds one
39+
probe string and reports the length it got back. `dimensions` (Matryoshka
40+
truncation) changes that length when set.
41+
- Empty input returns no vectors and issues no request.
42+
43+
## Non-goals
44+
45+
- A provider enum or adapter per vendor. There is one `/v1/embeddings` path.
46+
- Inferring vector width from a model name.
47+
- Concurrent batches. Sequential posting is how a rate-limited provider is
48+
not hit with the whole corpus at once.

‎README.md‎

Lines changed: 25 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,92 +1,66 @@
11
# @corbits/embedding
22

3-
Embedding client for OpenAI-compatible `/v1/embeddings` endpoints. Given a
4-
`baseURL`, a model, and some texts, `embedTexts` batches them, posts each batch,
5-
validates the response, and returns vectors in input order.
3+
One API for every embedding provider. Point at any OpenAI-compatible `/v1/embeddings` endpoint, swap models with a config change, and get ordered vectors back.
64

7-
One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM and Jina all
8-
serve `/v1/embeddings` — what differs between them is parameters:
5+
OpenAI, Ollama, TEI, vLLM, and Jina all serve the same wire shape. This package is one code path over that shape, with parameters for what varies between providers:
96

10-
- `dimensions` — Matryoshka truncation, supported by OpenAI text-embedding-3
11-
(down to 256), Jina v3 (32) and v4 (128). Models without MRL reject it, so it
12-
is only sent when set.
13-
- `encodingFormat` — `float` or `base64`.
14-
- `batchSize` — providers cap inputs per request; OpenAI rejects arrays over
15-
2048, and a smaller batch is also how you stay under a token ceiling.
7+
- `dimensions` — Matryoshka truncation for models that support it, including OpenAI `text-embedding-3` (down to 256), Jina v3 (32), and Jina v4 (128). Sent only when set, so models without MRL stay happy.
8+
- `encodingFormat` — `float` (default) or compact `base64`.
9+
- `batchSize` — inputs per request (default 32). Tune it to provider caps and per-request token ceilings.
1610

17-
Transport, error classification and retry come from `@intx/inference` rather
18-
than being reimplemented here: `deps.fetch` is the single request path,
19-
failures become an `InferenceError` through the same classifiers a chat call
20-
uses, and `createDefaultRetryPolicy` decides whether to back off or abort. A 429
21-
from an embedding endpoint therefore behaves exactly as one from a chat
22-
endpoint, and a `credential_failure` aborts immediately rather than retrying a
23-
bad key.
11+
Transport, error classification, and retry build on `@intx/inference`: requests go through the shared `deps.fetch` path, failures classify into `InferenceError` just like chat calls, and `createDefaultRetryPolicy` guides backoff. A 429 from an embeddings endpoint behaves like one from a chat endpoint, and credential failures short-circuit.
2412

25-
## Installation
26-
27-
Not published to npm yet. Until a registry publish, `npm install @corbits/embedding`
28-
404s. Git is the install path.
13+
## Install
2914

3015
```bash
31-
bun add github:corbitsdev/corbits-embedding
16+
bun add @corbits/embedding
3217
```
3318

34-
Requires Node >= 24 or Bun >= 1.2. The default export is built `dist/`; native Node
35-
does not load this package's extensionless TypeScript source.
19+
Runs on Bun >= 1.2 or Node >= 24. The built `dist/` entry is the default import.
20+
21+
## Quickstart
3622

3723
```ts
3824
import { createDefaultScheduler } from "@intx/inference";
3925
import { embedTexts, probeEmbedDims } from "@corbits/embedding";
4026

41-
// Only `fetch` and `scheduler` are needed — a full harness `Dependencies`
42-
// also satisfies this if you already have one.
4327
const deps = { fetch, scheduler: createDefaultScheduler() };
4428

4529
const config = {
46-
baseURL: "http://localhost:11434/v1", // include the provider's version prefix
30+
baseURL: "http://localhost:11434/v1", // provider root, version prefix included
4731
model: "nomic-embed-text",
4832
};
4933

5034
const dims = await probeEmbedDims(config, { deps });
5135
const [vector] = await embedTexts(["hello world"], config, { deps });
5236
```
5337

54-
## Dimensionality is discovered, never assumed
38+
`embedTexts(texts, config, options)` batches sequentially, posts each batch to `{baseURL}/embeddings`, and returns vectors in input order. Options carry `deps`, with optional `retryPolicy`, `extractRetryAfterMs`, and `signal`.
5539

56-
`probeEmbedDims` embeds one probe string and reports the length it got back.
57-
Dimensionality is not knowable from a model name — it varies by provider and is
58-
changed by `dimensions` — so a caller that persists vectors must discover it.
59-
Defaults differ enough to matter: 1536 for OpenAI `text-embedding-3-small`, 768
60-
for `nomic-embed-text`, 384 for `bge-small-en-v1.5`. Swapping models is a
61-
migration, not a config change.
40+
## Dimensionality
6241

63-
## Errors
42+
`probeEmbedDims(config, { deps })` embeds one probe string and reports the length received. Dimensionality follows the provider and the `dimensions` setting, so callers that persist vectors discover it at startup and treat a model swap as a migration. Representative widths: 1536 for OpenAI `text-embedding-3-small`, 768 for `nomic-embed-text`, 384 for `bge-small-en-v1.5`.
6443

65-
Every failure — transport, HTTP status, or a 200 whose body is not JSON —
66-
raises `ModelRequestError`, carrying the classified `InferenceError` as `reason`
67-
and the URL.
44+
## Errors
6845

69-
Note that `@corbits/embedding` and `@corbits/reranking` each carry their own
70-
copy of this class until the shared transport is upstreamed, so `instanceof`
71-
does not hold across the two. Catching both? Discriminate on
72-
`error.name === "ModelRequestError"`.
46+
Every failure mode — transport, HTTP status, or a 200 with an unexpected body — raises `ModelRequestError`, carrying the classified `InferenceError` as `reason` plus the request URL. The embedding and reranking packages each carry their own copy of this class while the shared transport is upstreamed, so code catching both discriminates on `error.name === "ModelRequestError"`.
7347

7448
## Transport exports
7549

76-
The barrel re-exports the transport `embedTexts` is built on, for sibling
77-
one-shot JSON clients that want the same classified, retried request path
78-
without touching `@intx/inference` internals:
50+
The barrel also re-exports the one-shot JSON transport `embedTexts` is built on, for sibling clients that want the same classified, retried request path:
7951

8052
- `runJSONRequest` — one JSON POST with error classification and retry.
81-
- `extractRetryAfterMs` — the default `Retry-After` reader, overridable per
82-
call via `extractRetryAfterMs`.
83-
- `ModelRequestError` — the error every failure mode raises, carrying the
84-
classified `InferenceError` as `reason` and the URL.
53+
- `extractRetryAfterMs` — default `Retry-After` reader, overridable per call.
54+
- `ModelRequestError` — error for every failure mode, with `reason` and URL.
8555
- Types `RunRequestOptions` and `RetryAfterExtractor`.
8656

57+
## Interchange
58+
59+
Interchange hubs and ingestion workers use this package for embeddings behind the shared `@intx/inference` transport: same fetch path, same error taxonomy, and same retry behavior as chat. Memory and retrieval pipelines pair `probeEmbedDims` at startup with `embedTexts` batches at ingest and query time.
60+
8761
## Versioning
8862

89-
Semver. Released manually with `bun run build && npm publish` once CI is green.
63+
Semver. Releases run `bun run build && npm publish` with green CI.
9064

9165
## Development
9266

0 commit comments

Comments
 (0)