Skip to content

LCORE-2437: Support pgvector in byok_rag enrichment script #1856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
are-ces wants to merge 1 commit into
base: main
Choose a base branch
from
+481 −90
Open

LCORE-2437: Support pgvector in byok_rag enrichment script #1856

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 47 additions & 44 deletions docs/byok_guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,18 +189,33 @@ byok_rag:
score_multiplier: 1.0 # Weight for Inline RAG result ranking (default: 1.0)
```

**`byok_rag` field reference:**
**Common fields (all providers):**

| Field | Required | Default | Description |
|-----------------------|----------|-------------------------------------------|-------------------------------------------------------------------------------------------|
| `rag_id` | Yes | — | Unique identifier for the knowledge source |
| `rag_type` | No | `inline::faiss` | Vector store provider type |
| `rag_type` | No | `inline::faiss` | Vector store provider type (`inline::faiss` or `remote::pgvector`) |
| `embedding_model` | No | `sentence-transformers/all-mpnet-base-v2` | Embedding model identifier or path |
| `embedding_dimension` | No | `768` | Embedding vector dimensionality |
| `vector_db_id` | Yes | — | Vector store ID generated by rag-content (e.g. `vs_8c94967b-81cc-4028-a294-9cfac6fd9ae2`) |
| `db_path` | Yes | — | Path to the vector database file |
| `score_multiplier` | No | `1.0` | Weight for Inline RAG ranking (values > 1.0 boost; < 1.0 reduce) |

**FAISS fields** (`rag_type: inline::faiss`):

| Field | Required | Default | Description |
|-----------|----------|---------|----------------------------------|
| `db_path` | Yes | — | Path to the vector database file |

**pgvector fields** (`rag_type: remote::pgvector`):

| Field | Required | Default | Description |
|------------|----------|--------------------------|---------------------|
| `host` | No | `${env.POSTGRES_HOST}` | PostgreSQL host |
| `port` | No | `${env.POSTGRES_PORT}` | PostgreSQL port |
| `db` | No | `${env.POSTGRES_DATABASE}` | PostgreSQL database |
| `user` | No | `${env.POSTGRES_USER}` | PostgreSQL user |
| `password` | No | `${env.POSTGRES_PASSWORD}` | PostgreSQL password |

**Multiple knowledge sources:**

You can configure multiple BYOK sources. When using Inline RAG, `score_multiplier` adjusts the relative importance of each store's results:
Expand Down Expand Up @@ -283,28 +298,27 @@ byok_rag:
### 2. pgvector (PostgreSQL)
- **Type**: PostgreSQL with pgvector extension
- **Best for**: Large-scale deployments, shared knowledge bases
- **Configuration**: `remote::pgvector`
- **Configuration**: `rag_type: remote::pgvector`
- **Requirements**: PostgreSQL with pgvector extension

> [!NOTE]
> pgvector is not yet supported via `byok_rag` in `lightspeed-stack.yaml` (see [LCORE-2437](https://redhat.atlassian.net/browse/LCORE-2437)).
> It must be configured directly in the Llama Stack configuration file.

```yaml
vector_io:
- provider_id: pgvector-knowledge
provider_type: remote::pgvector
config:
host: localhost
port: 5432
db: knowledge_db
user: lightspeed_user
byok_rag:
- rag_id: pgvector-knowledge
rag_type: remote::pgvector
embedding_model: sentence-transformers/all-mpnet-base-v2
embedding_dimension: 768
vector_db_id: rhdocs
host: ${env.POSTGRES_HOST}
port: ${env.POSTGRES_PORT}
db: ${env.POSTGRES_DATABASE}
user: ${env.POSTGRES_USER}
password: ${env.POSTGRES_PASSWORD}
kvstore:
type: sqlite
db_path: .llama/distributions/pgvector/registry.db
```

> [!NOTE]
> Connection fields (`host`, `port`, `db`, `user`, `password`) default to
> `${env.POSTGRES_*}` environment variable references when omitted.

Comment thread
coderabbitai[bot] marked this conversation as resolved.
**pgvector Table Schema:**
- `id` (text): UUID identifier of the chunk
- `document` (jsonb): JSON containing content and metadata
Expand Down Expand Up @@ -342,13 +356,7 @@ rag:

### Example 2: Multiple Knowledge Sources with pgvector

A configuration combining a local FAISS store (via `byok_rag`) with a remote pgvector store (configured directly in the Llama Stack configuration file):

> [!NOTE]
> pgvector is not yet supported via `byok_rag` in `lightspeed-stack.yaml` (see [LCORE-2437](https://redhat.atlassian.net/browse/LCORE-2437)).
> The pgvector provider must be configured directly in the Llama Stack configuration file.

**`lightspeed-stack.yaml`** — FAISS store and RAG strategy:
A configuration combining a local FAISS store with a remote pgvector store:

```yaml
name: Lightspeed Core Service (LCS)
Expand All @@ -365,34 +373,29 @@ byok_rag:
vector_db_id: vs_e9d8c7b6-43af-4b2d-8e1f-0a9b8c7d6e5f
db_path: /data/vector_dbs/local/faiss_store.db
score_multiplier: 1.0
- rag_id: enterprise-kb
rag_type: remote::pgvector
embedding_model: sentence-transformers/all-mpnet-base-v2
embedding_dimension: 768
vector_db_id: enterprise_docs
host: ${env.POSTGRES_HOST}
port: ${env.POSTGRES_PORT}
db: ${env.POSTGRES_DATABASE}
user: ${env.POSTGRES_USER}
password: ${env.POSTGRES_PASSWORD}

rag:
inline:
- local-docs
- enterprise-kb
tool:
- local-docs
```

**Llama Stack configuration file** — pgvector provider:

```yaml
vector_io:
- provider_id: enterprise-kb
provider_type: remote::pgvector
config:
host: localhost
port: 5432
db: knowledge_db
user: lightspeed_user
password: ${env.POSTGRES_PASSWORD}
kvstore:
type: sqlite
db_path: .llama/distributions/pgvector/registry.db
- enterprise-kb
```

> [!NOTE]
> For pgvector, ensure your PostgreSQL credentials are available via environment variables
> (e.g., `POSTGRES_PASSWORD`).
> (e.g., `POSTGRES_HOST`, `POSTGRES_PASSWORD`).

> [!TIP]
> A complete working example combining BYOK and OKP is available at
Expand Down
78 changes: 73 additions & 5 deletions docs/openapi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11818,7 +11818,7 @@
"type": "string",
"minLength": 1,
"title": "RAG type",
"description": "Type of RAG database.",
"description": "Type of RAG database (e.g. 'inline::faiss', 'remote::pgvector').",
"default": "inline::faiss"
},
"embedding_model": {
Expand All @@ -11842,24 +11842,92 @@
"description": "Vector database identification."
},
"db_path": {
"type": "string",
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "DB path",
"description": "Path to RAG database."
"description": "Path to RAG database. Required for inline::faiss."
},
"score_multiplier": {
"type": "number",
"exclusiveMinimum": 0.0,
"title": "Score multiplier",
"description": "Multiplier applied to relevance scores from this vector store. Used to weight results when querying multiple knowledge sources. Values > 1 boost this store's results; values < 1 reduce them.",
"default": 1.0
},
"host": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "PostgreSQL host",
"description": "PostgreSQL host for remote::pgvector. Defaults to ${env.POSTGRES_HOST} when rag_type is remote::pgvector."
},
"port": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "PostgreSQL port",
"description": "PostgreSQL port for remote::pgvector. Defaults to ${env.POSTGRES_PORT} when rag_type is remote::pgvector."
},
"db": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "PostgreSQL database",
"description": "PostgreSQL database name for remote::pgvector. Defaults to ${env.POSTGRES_DATABASE} when rag_type is remote::pgvector."
},
"user": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "PostgreSQL user",
"description": "PostgreSQL user for remote::pgvector. Defaults to ${env.POSTGRES_USER} when rag_type is remote::pgvector."
},
"password": {
"anyOf": [
{
"type": "string",
"format": "password",
"writeOnly": true
},
{
"type": "null"
}
],
"title": "PostgreSQL password",
"description": "PostgreSQL password for remote::pgvector. Defaults to ${env.POSTGRES_PASSWORD} when rag_type is remote::pgvector."
}
},
"additionalProperties": false,
"type": "object",
"required": [
"rag_id",
"vector_db_id",
"db_path"
"vector_db_id"
],
"title": "ByokRag",
"description": "BYOK (Bring Your Own Knowledge) RAG configuration."
Expand Down
41 changes: 13 additions & 28 deletions docs/rag_guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,6 @@ See the full working [config example](../examples/lightspeed-stack-byok-okp-rag.

This example shows how to configure a remote PostgreSQL database with the [pgvector](https://github.com/pgvector/pgvector) extension for storing embeddings.

> [!NOTE]
> pgvector is not yet supported via `byok_rag` in `lightspeed-stack.yaml` (see [LCORE-2437](https://redhat.atlassian.net/browse/LCORE-2437)).
> It must be configured directly in the Llama Stack configuration file.

> You will need to install PostgreSQL with a matching version to pgvector, then log in with `psql` and enable the extension with:
> ```sql
> CREATE EXTENSION IF NOT EXISTS vector;
Expand All @@ -107,33 +103,22 @@ Each pgvector-backed table follows this schema:
> The `vector_store_id` (e.g. `rhdocs`) is used to point to the table named `vector_store_rhdocs` in the specified database, which stores the vector embeddings.

```yaml
providers:
[...]
vector_io:
- provider_id: pgvector-example
provider_type: remote::pgvector
config:
host: localhost
port: 5432
db: pgvector_example # PostgreSQL database (psql -d pgvector_example)
user: lightspeed # PostgreSQL user
password: password123
kvstore:
type: sqlite
db_path: .llama/distributions/pgvector/pgvector_registry.db
vector_stores:
- embedding_dimension: 768
embedding_model: sentence-transformers/all-mpnet-base-v2
provider_id: pgvector-example
# A unique ID that becomes the PostgreSQL table name, prefixed with 'vector_store_'.
# e.g., 'rhdocs' will create the table 'vector_store_rhdocs'.
# If the table was already created, this value must match the ID used at creation.
vector_store_id: rhdocs
byok_rag:
- rag_id: pgvector-example
rag_type: remote::pgvector
embedding_model: sentence-transformers/all-mpnet-base-v2
embedding_dimension: 768
vector_db_id: rhdocs # becomes PostgreSQL table 'vector_store_rhdocs'
host: ${env.POSTGRES_HOST}
port: ${env.POSTGRES_PORT}
db: ${env.POSTGRES_DATABASE}
user: ${env.POSTGRES_USER}
password: ${env.POSTGRES_PASSWORD}
```

> [!NOTE]
> For pgvector, the PostgreSQL connection details (host, port, database, user, password) are configured
> in the provider configuration. Use environment variables for credentials.
> Connection fields (`host`, `port`, `db`, `user`, `password`) default to `${env.POSTGRES_*}`
> environment variable references when omitted. Use environment variables for credentials.

---

Expand Down
Loading
Loading