|
1 | 1 | # @corbits/reranking |
2 | 2 |
|
3 | | -Cross-encoder reranking for retrieval pipelines. Send a query plus candidates, get scored ids back in ranked order. |
| 3 | +Cross-encoder reranking over Hugging Face Text Embeddings Inference (TEI), Cohere/Jina and Voyage rerank endpoints, built on `@intx/inference` retries and error classification. A retrieval building block for Corbits and Interchange agents that also works standalone. |
4 | 4 |
|
5 | | -Give `rerankDocuments` a query and your own `{id, text}` docs; it posts them to the reranker, maps the reply back to your ids, and returns `{id, score}` sorted descending. |
| 5 | +## Why @corbits/reranking? |
6 | 6 |
|
7 | | -## Runtime support |
| 7 | +1. **One call for three wire formats.** `rerankDocuments` speaks TEI, Cohere (also Jina) and Voyage. Switching providers is a config change. |
| 8 | +2. **Your ids back, sorted.** Providers score by array position. The client maps each score back to your `id` and returns `{ id, score }`, best first. |
| 9 | +3. **Interchange retry semantics.** 429s and 5xx are retried under the `@intx/inference` retry policy, and every failure is a `RerankRequestError` carrying a classified `InferenceError`. |
8 | 10 |
|
9 | | -Runs on Bun >= 1.2 or Node >= 24. The built `dist/` entry is the default import. |
| 11 | +## Install |
| 12 | + |
| 13 | +```bash |
| 14 | +bun add @corbits/reranking @intx/inference@^0.4.0 @intx/types@^0.4.0 |
| 15 | +``` |
| 16 | + |
| 17 | +Runs on Bun >= 1.2 or Node >= 24. |
10 | 18 |
|
11 | 19 | ## Quickstart |
12 | 20 |
|
| 21 | +This example needs a local TEI reranker: |
| 22 | + |
13 | 23 | ```bash |
14 | | -bun add @corbits/reranking |
| 24 | +docker run -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cpu-latest \ |
| 25 | + --model-id BAAI/bge-reranker-base |
| 26 | +``` |
| 27 | + |
| 28 | +```ts |
| 29 | +import { rerankDocuments } from "@corbits/reranking"; |
| 30 | + |
| 31 | +const docs = [ |
| 32 | + { id: "weather", text: "Rain is expected across the valley tomorrow." }, |
| 33 | + { id: "paris", text: "Paris is the capital and largest city of France." }, |
| 34 | + { id: "recipe", text: "Whisk the eggs before folding in the flour." }, |
| 35 | +]; |
| 36 | + |
| 37 | +const ranked = await rerankDocuments("What is the capital of France?", docs, { |
| 38 | + baseURL: "http://localhost:8080", |
| 39 | + apiStyle: "tei", |
| 40 | +}); |
| 41 | +console.log(ranked[0]?.id); // paris |
15 | 42 | ``` |
16 | 43 |
|
17 | | -`rerankDocuments(query, docs, config, options?)` validates config with `RerankConfigSchema`, short-circuits empty input with no request, and returns ranked results. Options are optional: `deps` (defaults to global `fetch` and `createDefaultScheduler()`), `retryPolicy`, `registry`, and `signal`. Config takes `baseURL` and `apiStyle`, with optional `model` (used by Cohere and Voyage), `apiKey`, and `timeoutMs`. |
| 44 | +## Where it fits |
18 | 45 |
|
19 | | -A retrieval pipeline reads its reranker endpoint from host config at boot — a half-configured reranker should fail boot, not degrade silently later — then reranks each query's fused candidates, falling back to the caller's own ordering on failure so a reranker outage never fails search: |
| 46 | +[Interchange](https://github.com/faremeter/interchange) runs AI agents with their own identity and permissions. Corbits packages add the parts an agent product needs around it. |
20 | 47 |
|
21 | | -```ts |
22 | | -import { createDefaultScheduler } from "@intx/inference"; |
23 | | -import { |
24 | | - rerankDocuments, |
25 | | - type RerankDoc, |
26 | | - type RerankResult, |
27 | | -} from "@corbits/reranking"; |
| 48 | +- **Runs in:** any process. That can be the Interchange hub (the server that manages tenants and agents), an agent sidecar (the runtime next to each agent), or a plain script. No hub is needed. |
| 49 | +- **Plugs into:** [`@intx/inference`](https://github.com/faremeter/interchange) for retries and error types, and [`@intx/types`](https://github.com/faremeter/interchange). |
| 50 | +- **Pairs with:** [`@corbits/embedding`](https://github.com/corbitsdev/corbits-embedding) to turn text into vectors, and [`@corbits/memory`](https://github.com/corbitsdev/corbits-memory) to store and search them. |
28 | 51 |
|
29 | | -const deps = { fetch, scheduler: createDefaultScheduler() }; |
30 | | - |
31 | | -function requireEnv(name: string): string { |
32 | | - const value = process.env[name]; |
33 | | - if (!value) throw new Error(`${name} is required`); |
34 | | - return value; |
35 | | -} |
36 | | - |
37 | | -const baseURL = requireEnv("RERANK_BASE_URL"); |
38 | | -const model = process.env.RERANK_MODEL; |
39 | | - |
40 | | -export async function rerankFusedCandidates( |
41 | | - query: string, |
42 | | - candidates: readonly RerankDoc[], |
43 | | - onError: (error: unknown) => void, |
44 | | -): Promise<RerankResult[]> { |
45 | | - try { |
46 | | - return await rerankDocuments( |
47 | | - query, |
48 | | - candidates, |
49 | | - { |
50 | | - baseURL, |
51 | | - apiStyle: "tei", |
52 | | - ...(model !== undefined ? { model } : {}), |
53 | | - }, |
54 | | - { deps }, |
55 | | - ); |
56 | | - } catch (error) { |
57 | | - // Reranker unavailable or timed out — report it, then degrade to the |
58 | | - // fused order the caller already computed rather than fail the search. |
59 | | - onError(error); |
60 | | - return candidates.map((doc, i) => ({ |
61 | | - id: doc.id, |
62 | | - score: candidates.length - i, |
63 | | - })); |
64 | | - } |
65 | | -} |
66 | | -``` |
| 52 | +## Reference |
67 | 53 |
|
68 | | -## How it works |
| 54 | +| Export | Description | |
| 55 | +| --------------------------------------------------------------- | ----------------------------------------------------------------- | |
| 56 | +| `rerankDocuments(query, docs, config, options?)` | Scores each doc and returns `{ id, score }`, best first. | |
| 57 | +| `RerankConfigSchema`, `RerankConfig` | Schema and type for `config`. | |
| 58 | +| `RerankOptions` | Type for `options`. | |
| 59 | +| `rerankAdapterRegistry` | The built-in formats: `tei`, `cohere` and `voyage`. | |
| 60 | +| `createRerankAdapterRegistry(adapters)` | Builds a registry with your own formats. | |
| 61 | +| `RerankRequestError` | Thrown when a request fails. | |
| 62 | +| `RerankAdapter`, `RerankRequestBuilder`, `RerankResponseParser` | Types for writing an adapter. | |
| 63 | +| `RerankRequestConfig` | Config an adapter's `buildRequest` receives. | |
| 64 | +| `RerankAdapterRegistry`, `RerankAPIStyle` | Types for registries and built-in style names. | |
| 65 | +| `RerankDoc`, `RerankResult` | Types for docs and results. | |
| 66 | +| `RequestDependencies`, `RetryAfterExtractor` | Types for `options.deps` and `RerankAdapter.extractRetryAfterMs`. | |
69 | 67 |
|
70 | | -Reranking has no single OpenAI-style standard, so this package draws the adapter boundary explicitly: |
| 68 | +### Config |
71 | 69 |
|
72 | | -| `apiStyle` | Endpoint | Request shape | Response shape | |
| 70 | +| Field | Type | Description | |
| 71 | +| ----------- | --------- | ----------------------------------------------------------------------------------- | |
| 72 | +| `baseURL` | `string` | Server root, such as `http://localhost:8080`. | |
| 73 | +| `apiStyle` | `string` | Wire format: `tei`, `cohere` (also Jina) or `voyage`. | |
| 74 | +| `model` | `string?` | Model name. Required by `cohere` (and Jina) and `voyage`. TEI serves its own model. | |
| 75 | +| `apiKey` | `string?` | Sent as a bearer token. | |
| 76 | +| `timeoutMs` | `number?` | Time limit per attempt, so retries can exceed it. Defaults to 30000. | |
| 77 | + |
| 78 | +### Options |
| 79 | + |
| 80 | +| Field | Description | |
| 81 | +| ------------- | ------------------------------------------------------------------------------- | |
| 82 | +| `deps` | `{ fetch, scheduler }`. Defaults to global `fetch` and Interchange's scheduler. | |
| 83 | +| `retryPolicy` | Decides when to retry. Defaults to Interchange's policy. | |
| 84 | +| `registry` | Wire formats to choose from. Defaults to `rerankAdapterRegistry`. | |
| 85 | +| `signal` | Cancels the request. | |
| 86 | + |
| 87 | +### Wire formats |
| 88 | + |
| 89 | +| `apiStyle` | Endpoint | Request | Reply | |
73 | 90 | | ---------- | ------------ | -------------------- | --------------------------------------- | |
74 | 91 | | `tei` | `/rerank` | `{query, texts}` | `[{index, score}]` | |
75 | 92 | | `cohere` | `/v2/rerank` | `{query, documents}` | `{results: [{index, relevance_score}]}` | |
76 | 93 | | `voyage` | `/v1/rerank` | `{query, documents}` | `{data: [{index, relevance_score}]}` | |
77 | 94 |
|
78 | | -Jina's `/rerank` follows the Cohere shape and is served by the `cohere` adapter. |
| 95 | +To add your own format, build a registry and pass it as `registry`. This one serves TEI-shaped replies at `/score`: |
| 96 | + |
| 97 | +```ts |
| 98 | +import { |
| 99 | + createRerankAdapterRegistry, |
| 100 | + rerankAdapterRegistry, |
| 101 | +} from "@corbits/reranking"; |
| 102 | + |
| 103 | +const tei = rerankAdapterRegistry.resolve("tei"); |
| 104 | + |
| 105 | +const registry = createRerankAdapterRegistry({ |
| 106 | + tei, |
| 107 | + house: { |
| 108 | + buildRequest: (query, docs, config) => ({ |
| 109 | + url: `${config.baseURL}/score`, |
| 110 | + headers: { "content-type": "application/json" }, |
| 111 | + body: JSON.stringify({ query, texts: docs.map((doc) => doc.text) }), |
| 112 | + }), |
| 113 | + parseResponse: tei.parseResponse, |
| 114 | + }, |
| 115 | +}); |
| 116 | +``` |
79 | 117 |
|
80 | | -`RerankAdapter` mirrors inference's `ProviderAdapter` with `buildRequest` / `parseResponse` plus optional `extractRetryAfterMs`. Styles resolve through `createRerankAdapterRegistry`, which keeps a private `Map` copy so config-supplied style names resolve safely. Pass a custom `registry` to add a house format alongside the built-ins in `rerankAdapterRegistry`. |
| 118 | +### Errors |
81 | 119 |
|
82 | | -Every protocol addresses documents by position in the request array, and TEI replies unordered. Results map back through `docs[index]`, with an out-of-range index raising so a score always lands on the right document. `RerankDoc.id` carries your stable identifier across that index-based wire, and the final list sorts by score descending. |
| 120 | +- A failed request throws `RerankRequestError`. Its `reason` says what went wrong, and `url` says where. Network errors, timeouts, rate limits and server errors are retried first under the retry policy. |
| 121 | +- A bad config throws arktype's `TraversalError` before any request is sent. |
| 122 | +- With non-empty `docs`, a missing `model` for `cohere` or `voyage` throws `TraversalError`, and an `apiStyle` the registry does not know throws an `Error` that names it. |
| 123 | +- A custom `parseResponse` should throw `ProtocolMismatchError` from `@intx/inference` so the failure becomes a `RerankRequestError`. Anything else it throws is rethrown as is. |
83 | 124 |
|
84 | | -This package raises on failure, leaving fallback policy at the call site. Retrieval pipelines commonly fall back to their fused ordering when the reranker is unavailable and carry on serving. |
| 125 | +An empty `docs` list returns `[]` without a request. To keep search working when the reranker is down, catch the error and keep your original order. |
85 | 126 |
|
86 | | -Every failure mode — transport, HTTP status, or a reply in the wrong shape — raises `RerankRequestError`, carrying the classified `InferenceError` as `reason` plus the request URL. Transport, classification, and retry come from `@intx/inference`: built requests travel the shared `deps.fetch` path with `createDefaultRetryPolicy` guiding backoff. An invalid config raises arktype's `TraversalError` before any request is sent. |
| 127 | +## Using with Interchange |
87 | 128 |
|
88 | | -Interchange retrieval pipelines call `rerankDocuments` after hybrid search to lift the strongest candidates to the top. The shared `@intx/inference` transport keeps rerank retries and error taxonomy consistent with chat and embeddings across the hub. |
| 129 | +To share your host's retry scheduler, pass it as `options.deps.scheduler` along with `fetch`. |
89 | 130 |
|
90 | | -## Development |
| 131 | +## Upgrading from 0.1 |
91 | 132 |
|
92 | | -```bash |
93 | | -bun install |
94 | | -bun test ./src |
95 | | -bunx tsc --noEmit |
96 | | -``` |
| 133 | +- Install `@intx/inference` and `@intx/types` (^0.4.0) yourself. They are now peer dependencies. |
| 134 | +- `ModelRequestError` is renamed to `RerankRequestError`. |
| 135 | +- `rerankAdapters` is no longer exported. Use `rerankAdapterRegistry` or `createRerankAdapterRegistry`. |
| 136 | +- `runJSONRequest`, `extractRetryAfterMs` and `RunRequestOptions` are no longer exported. |
| 137 | +- A missing `model` for Cohere or Voyage now throws `TraversalError` instead of a plain `Error`. |
| 138 | +- Config, wire requests and results are unchanged. |
97 | 139 |
|
98 | 140 | ## License |
99 | 141 |
|
100 | | -LGPL-2.1-only — see [`LICENSE`](LICENSE). |
| 142 | +[LGPL-2.1-only](https://github.com/corbitsdev/corbits-reranking/blob/main/LICENSE) |
0 commit comments