Skip to content

Latest commit

 

History

History
394 lines (311 loc) · 12.8 KB

chat.mdx

File metadata and controls

394 lines (311 loc) · 12.8 KB
title Document Q&A
description Chat with documents using text, voice, and RAG-powered retrieval
icon file-lines
**First time here?** Complete the [Setup](/setup) guide first to install GAIA and its dependencies. **Prefer a desktop app?** See [GAIA Chat Desktop](/guides/agent-ui) for the privacy-first GUI with drag-and-drop document Q&A. **Looking for the API?** See the [Agent SDK Reference](/sdk/sdks/chat) for classes, methods, and code examples.

Quick Start

Activate your virtual environment and install GAIA with RAG support:
```bash
uv pip install -e ".[rag]"
```
```python title="simple_chat.py" from gaia.chat.sdk import SimpleChat
chat = SimpleChat()
response = chat.ask("What is Python?")
print(response)

# Follow-up with conversation memory
response = chat.ask("Give me an example")
print(response)
```
```python title="full_chat.py" from gaia.chat.sdk import AgentSDK, AgentConfig
config = AgentConfig(
    show_stats=True,
    max_history_length=6
)
chat = AgentSDK(config)

response = chat.send("Hello! My name is Alex.")
print(response.text)

response = chat.send("What's my name?")
print(response.text)  # Will remember "Alex"
```

CLI Usage

Interactive Mode

Start a conversational chat session:

```bash Basic # Start interactive chat gaia chat ```
# Show performance metrics
gaia chat --stats
- `/resume [id]` - Resume a previous conversation (or list sessions if no id) - `/save` - Save current conversation - `/sessions` - List all saved sessions - `/reset` - Clear conversation and start fresh - `/help` or `/?` - Show help message - `/quit` - Exit the chat session

/quit and /reset save the current session first. If saving fails, the conversation stays open and unchanged so you can fix the storage problem and retry /save, then /quit or /reset. To exit without saving, press Ctrl-C; unsaved changes will be lost. Document indexing and queries also warn if a new document attachment could not be saved to the session, even though the document is available in the current conversation.

Single Query Mode

# One-shot query
gaia chat --query "What is artificial intelligence?"

# With statistics
gaia chat --query "Hello" --show-stats

Document Q&A (RAG)

RAG (Retrieval-Augmented Generation) enables chatting with documents — including PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx), EPUB e-books (.epub), and images (.png, .jpg, .webp, ...) — using semantic search and context retrieval.

CLI with RAG

```bash # Chat with a PDF, Word, PowerPoint, or Excel document gaia chat --index manual.pdf gaia chat --index handbook.docx gaia chat --index slides.pptx gaia chat --index budget.xlsx gaia chat --index screenshot.png ``` ```bash # Chat with multiple documents (PDF, DOCX, PPTX, XLSX supported) gaia chat --index doc1.pdf report.docx slides.pptx ``` ```bash # One-shot query with document gaia chat --index report.pdf --query "Summarize the key findings" ``` ```bash # Auto-index every supported document in a folder, and any new ones dropped in later gaia chat --watch ./docs ``` **Prerequisites:** Voice mode requires the `talk` module: ```bash uv pip install -e ".[talk]" ```
```bash
# Voice with documents
gaia talk --index manual.pdf
```
**Document Indexing Requirements:** Text in images and scans is read by a Vision Language Model (VLM): a tested vision model that is already loaded, then your default chat model when it can read images, which `Gemma-4-E4B-it-GGUF` and `Qwen3.6-35B-A3B-GGUF` both can, and otherwise `Gemma-4-E4B-it-GGUF`. If that model isn't downloaded but another tested one is, such as `Qwen3.6-35B-A3B-GGUF`, GAIA reads images with the one on disk instead of asking you to download it. While chat runs on the NPU, images are not read at all: the NPU model doesn't read them, and GAIA says so instead of loading a second model. `gaia init` already downloads these, unless you set a `default_model` that can't read images: then `gaia init` skips Gemma, so download it yourself: ```bash lemonade pull Gemma-4-E4B-it-GGUF ``` Or download it in the Lemonade Model Manager (`http://localhost:13305`).
  • Image files (.png, .jpg, .jpeg, .webp, .bmp, .gif, .tif, .tiff) are indexed only through the VLM. Without it, indexing the image fails and says how to get the VLM. Index images one at a time: folder indexing skips them.
  • PDF and PPTX files still index without the VLM, but only their text layer. Text inside their images is not indexed.
  • Images on PDF pages that already hold plenty of text are not read. Scans and pages short of text are, which is where nearly all the text in images was found. Set use_vlm=False to skip images altogether.

Interactive RAG Commands

When using gaia chat with documents (via --index flag or /index command), additional commands become available:

Sessions preserve both your conversation history and indexed documents:
- `/resume [id]` - Resume session with conversation and documents restored
- `/save` - Save session including indexed documents
- `/sessions` - List all saved sessions
- `/reset` - Clear conversation and start a new session (indexed documents are preserved)
- `/index ` - Index a document or directory (enables RAG if needed) - `/watch ` - Watch directory for changes and auto-index new files - `/list` - List all currently indexed documents - `/status` - Show RAG system status (indexed files, chunks, memory usage) - `/chunks ` - View indexed chunks for a specific file - `/chunk ` - View specific chunk by ID - `/test ` - Test query retrieval with relevance scores - `/dump ` - Export document and chunks to markdown - `/clear-cache` - Clear RAG cache and force re-indexing - `/search-debug` - Enable detailed search debugging output

RAG Debug Mode

Enable debug mode to see detailed retrieval information:

```bash CLI Debug # CLI with debug gaia chat --index document.pdf --debug ```
# Python SDK with debug — ChatAgent takes a single ChatAgentConfig
from gaia_agent.chat_agent import ChatAgent, ChatAgentConfig

config = ChatAgentConfig(
    rag_documents=['document.pdf'],
    debug=True,
    silent_mode=False,
)
agent = ChatAgent(config)

result = agent.process_query("What is the vision statement?")
print(result)
- Search keys generated by the LLM - Chunks found for each search - Relevance scores - Deduplication statistics - Score distributions

Chunking Strategies

**Default - Fast processing**
```python
config = ChatAgentConfig(
    rag_documents=['document.pdf'],
    chunk_size=500,
    chunk_overlap=50,
)
agent = ChatAgent(config)
```
**More accurate context**
```python
config = ChatAgentConfig(
    rag_documents=['document.pdf'],
    use_llm_chunking=True,
    chunk_size=500,
)
agent = ChatAgent(config)
```

Troubleshooting

```bash uv pip install -e ".[rag]" ``` - Ensure PDF has extractable text (not scanned images) - Check the PDF does not need a password to open — remove it with `qpdf --decrypt --password=YOUR_PASSWORD in.pdf out.pdf` (PDFs with only permission restrictions index fine) - Verify file is not corrupted `enable_rag` is on `AgentSDK`, not `SimpleChat`. Pick one of the settings below:
```python
from gaia.chat.sdk import AgentSDK, AgentConfig

chat = AgentSDK(AgentConfig())

# Faster processing
chat.enable_rag(documents=["doc.pdf"], chunk_size=300, max_chunks=2)

# Better quality
chat.enable_rag(documents=["doc.pdf"], chunk_size=500, max_chunks=5, chunk_overlap=100)

# Memory efficient
chat.enable_rag(documents=["doc.pdf"], chunk_size=400, max_chunks=2)
```

Dynamic Tool Loading

**On by default.** Turn it off under **Settings → Advanced → Dynamic tools**, or with `gaia config set dynamic_tools false`. Either applies to new chats.

A local model has to read every tool's description before it can answer, which is most of the wait on a first reply. With dynamic tool loading a chat starts on the 29 tools people use most, in short form (name, one sentence, and parameters). On a Radeon 8060S with Qwen3.6-35B-A3B that cut what a cold first turn reads from about 21,300 tokens to about 9,500, and roughly halved the time to read it.

When a turn needs something else, the session moves to the full list once, and the list then stays put. The full list is kept ready beside the short one, so a first task starts as quickly as a first "hello". That move happens before the model is asked, when your message names a tool, a picture or a recording, when you paste a skill or ask to save, install, or correct one, when a recalled procedure needs a tool, or when a tool outside the list is a clearly better match than every tool on it. If the agent still lacks something, it calls search_tools, which finds tools and installed skills by what they do.

A session that chats for a while and then needs a rare tool pays for one re-read of the conversation, and both apps show Loading more tools while it happens. A chat reopened after its agent was rebuilt starts on the full list. Loading a skill adds the tools it needs, and unloading it takes them away unless another loaded skill needs them. The agent gets a tool's full documentation from search_tools, or automatically if it calls a tool with the wrong arguments.

This applies to local models, where every token is read on your hardware. Cloud models keep every tool in full. If the embedder is unreachable, the session offers every tool and logs a warning. Precedence is GAIA_DYNAMIC_TOOLS, then an explicit GaiaAgentConfig(dynamic_tools=...), then dynamic_tools in ~/.gaia/config.json. A plain ChatAgent stays off unless you set it. When the environment variable is set, the Settings toggle shows the effective value and disables itself.

The environment variable is read by the process that runs the agent. gaia eval agent drives the agent through the Agent UI backend, so for an eval run set it on the backend, started first, not on the eval command. See the eval guide's Prerequisites for the rest of the setup.

# Terminal 1: start the Agent UI backend with the loader off, to compare to the default
GAIA_DYNAMIC_TOOLS=0 gaia chat --ui

# Terminal 2: run the eval against it
gaia eval agent --category tool_selection

Next Steps

Privacy-first desktop app with drag-and-drop document Q&A Classes, methods, and code examples Add speech recognition and text-to-speech Explore all command-line options Python backend API for the desktop chat application Integrate via OpenAI-compatible API

License

Copyright(C) 2024-2026 Advanced Micro Devices, Inc. All rights reserved.

SPDX-License-Identifier: MIT