Initial commit: embedding client on the Interchange inference primitives

`embedTexts` embeds a corpus in batches against any /v1/embeddings endpoint;
`probeEmbedDims` discovers a model's output dimensionality by embedding one
probe string.

One wire format, no adapter layer. OpenAI, Ollama, TEI, vLLM and Jina all serve
/v1/embeddings and differ only in parameters, so a provider switch would be
three copies of the same function. Verified live rather than assumed: Ollama
returns 768 dims there, matching its native /api/embed, and a TEI container
running bge-small-en-v1.5 returns 384, matching its native /embed. This is the
opposite conclusion from @corbits/reranking, and deliberately so — that protocol
has no standard, this one does.

Dimensionality is discovered, never assumed. It varies by provider and is
changed by `dimensions` (Matryoshka truncation: OpenAI text-embedding-3 to 256,
Jina v3 to 32, v4 to 128), so anything persisting vectors must probe for it.

Vectors are placed by the `index` each entry echoes, never by reply position,
and a reply that is not exactly one entry per input is rejected. OpenAI does not
promise reply order, and callers pair vectors with their own texts positionally
— so a reordered or short reply would not fail loudly, it would silently persist
text/vector pairs that are wrong. The base64 encoding is decoded as a
little-endian float32 buffer, so the option the config advertises actually
round-trips.

Built on @intx/inference rather than reimplementing it. Requests are
BuiltRequests sent through the single deps.fetch path; deps.scheduler drives
elapsed time and backoff so virtual-clock tests run the retry loop with no wall
time; failures classify through the shared taxonomy, including a 200 whose body
is not JSON (classifyProtocolMismatch), which a local model server behind a
proxy will produce; createDefaultRetryPolicy decides back-off vs abort, and an
abort mid-delay wakes immediately as the harness documents. A caller signal and
timeoutMs compose via AbortSignal.any rather than the signal silently winning.
The client this replaces had no retry at all.

deps is Pick<Dependencies, "fetch" | "scheduler">: callers should not have to
assemble a registry of inference providers in order to embed a string. A real
Dependencies still satisfies it structurally.

Batches are sequential rather than concurrent. A provider that rate-limits would
otherwise see the whole corpus at once, and the retry policy is per request, not
per burst.

Naming follows the published @intX surface rather than assumption: baseURL (used
18 times upstream, baseUrl zero) and uppercase acronyms in identifiers.

Tests: 19 pass. Typecheck, prettier and build clean, all four gated in CI. The
retry, index-mapping and timeout paths were regression-verified by reintroducing
each bug and confirming the tests fail.

Refs CL-4539, CL-4549, CL-4550

Claude-Session: https://claude.ai/code/session_017GTgGzn5xAwvkU2GAPAHpF

Author: TheGreatAxios

.github/workflows/ci.yml

@@ -0,0 +1,20 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+      - run: bun install --frozen-lockfile
+      - run: bunx tsc --noEmit
+      - run: bun test ./src
+      - run: bunx prettier --check src
+      - run: bun run build

.gitignore

@@ -0,0 +1,3 @@
+node_modules
+dist
+*.tsbuildinfo

.prettierrc.json

@@ -0,0 +1,7 @@
+{
+  "trailingComma": "all",
+  "tabWidth": 2,
+  "useTabs": false,
+  "semi": true,
+  "singleQuote": false
+}

LICENSE

@@ -0,0 +1,76 @@
+# @corbits/embedding
+
+Embedding client for OpenAI-compatible `/v1/embeddings` endpoints. Given a
+`baseURL`, a model, and some texts, `embedTexts` batches them, posts each batch,
+validates the response, and returns vectors in input order.
+
+One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM and Jina all
+serve `/v1/embeddings` — what differs between them is parameters:
+
+- `dimensions` — Matryoshka truncation, supported by OpenAI text-embedding-3
+  (down to 256), Jina v3 (32) and v4 (128). Models without MRL reject it, so it
+  is only sent when set.
+- `encodingFormat` — `float` or `base64`.
+- `batchSize` — providers cap inputs per request; OpenAI rejects arrays over
+  2048, and a smaller batch is also how you stay under a token ceiling.
+
+Transport, error classification and retry come from `@intx/inference` rather
+than being reimplemented here: `deps.fetch` is the single request path,
+failures become an `InferenceError` through the same classifiers a chat call
+uses, and `createDefaultRetryPolicy` decides whether to back off or abort. A 429
+from an embedding endpoint therefore behaves exactly as one from a chat
+endpoint, and a `credential_failure` aborts immediately rather than retrying a
+bad key.
+
+```ts
+import { createDefaultScheduler } from "@intx/inference";
+import { embedTexts, probeEmbedDims } from "@corbits/embedding";
+
+// Only `fetch` and `scheduler` are needed — a full harness `Dependencies`
+// also satisfies this if you already have one.
+const deps = { fetch, scheduler: createDefaultScheduler() };
+
+const config = {
+  baseURL: "http://localhost:11434/v1", // include the provider's version prefix
+  model: "nomic-embed-text",
+};
+
+const dims = await probeEmbedDims(config, { deps });
+const [vector] = await embedTexts(["hello world"], config, { deps });
+```
+
+## Dimensionality is discovered, never assumed
+
+`probeEmbedDims` embeds one probe string and reports the length it got back.
+Dimensionality is not knowable from a model name — it varies by provider and is
+changed by `dimensions` — so a caller that persists vectors must discover it.
+Defaults differ enough to matter: 1536 for OpenAI `text-embedding-3-small`, 768
+for `nomic-embed-text`, 384 for `bge-small-en-v1.5`. Swapping models is a
+migration, not a config change.
+
+## Errors
+
+Every failure — transport, HTTP status, or a 200 whose body is not JSON —
+raises `ModelRequestError`, carrying the classified `InferenceError` as `reason`
+and the URL.
+
+Note that `@corbits/embedding` and `@corbits/reranking` each carry their own
+copy of this class until the shared transport is upstreamed, so `instanceof`
+does not hold across the two. Catching both? Discriminate on
+`error.name === "ModelRequestError"`.
+
+## Versioning
+
+Semver. Released manually with `bun run build && npm publish` once CI is green.
+
+## Development
+
+```bash
+bun install
+bun test ./src
+bunx tsc --noEmit
+```
+
+## License
+
+LGPL-2.1-only — see [`LICENSE`](LICENSE).

README.md

@@ -0,0 +1,44 @@
+{
+  "name": "@corbits/embedding",
+  "description": "Embedding client for OpenAI-compatible /v1/embeddings endpoints, built on the Interchange inference primitives",
+  "version": "0.1.0",
+  "license": "LGPL-2.1-only",
+  "type": "module",
+  "exports": {
+    ".": {
+      "intx-src": "./src/index.ts",
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    }
+  },
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "typecheck": "tsc --noEmit",
+    "test": "bun test ./src",
+    "format": "prettier -w ."
+  },
+  "dependencies": {
+    "@intx/inference": "0.2.2",
+    "@intx/log": "0.2.2",
+    "@intx/types": "0.2.2",
+    "arktype": "^2.1.29"
+  },
+  "devDependencies": {
+    "@intx/inference-testing": "0.2.2",
+    "@types/bun": "1.3.9",
+    "prettier": "3.6.2",
+    "typescript": "5.9.3"
+  },
+  "files": ["dist", "README.md", "LICENSE"],
+  "sideEffects": false,
+  "publishConfig": {
+    "access": "public"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/corbitsdev/corbits-embedding.git"
+  },
+  "engines": {
+    "bun": ">=1.2.0"
+  }
+}