<<<<<<< HEAD
A fully local, privacy-preserving Retrieval-Augmented Generation (RAG) system that runs entirely offline on your machine. No cloud services, no API calls, no data leaving your device.
TAI (Trustworthy AI) Assistant enables you to:
- Query your documents using natural language
- Get AI-generated answers with source citations
- Maintain complete privacy with offline operation
- Understand model confidence and explainability
- Python 3.8+ (Python 3.13+ recommended - this project uses 3.13.3)
- 8GB+ RAM (16GB recommended for better performance)
- Windows/Linux/MacOS
- Optional: CUDA-compatible GPU for faster inference
git clone <your-repo-url>
cd Private-Trustworthy-RAG-AssistantThis project uses uv for fast, modern Python package management. Dependencies are managed via pyproject.toml and uv.lock.
Option A: Using UV (Recommended - Already Set Up)
# UV is already configured in this project
# Dependencies are automatically synced via uv.lock
# To add new packages:
uv add package-name
# To sync dependencies:
uv syncOption B: Using Traditional pip + venv
# Create virtual environment
python -m venv venv
# Activate (Windows PowerShell)
venv\Scripts\Activate.ps1
# Or use: venv\Scripts\activate.bat
# Activate (Linux/MacOS)
source venv/bin/activate
# Install dependencies
pip install streamlit torch transformers langchain langchain-community chromadb pandas plotly sentence-transformers# Check Python version
python --version
# Should show: Python 3.13.3 (or your version)
# Check Streamlit
streamlit --version
# List installed packages
pip list# Run the Streamlit UI
streamlit run src/ui/app.pyThe application will open in your browser at http://localhost:8501
Private-Trustworthy-RAG-Assistant/
β
βββ src/
β βββ core/ # Core RAG pipeline
β β βββ __init__.py
β β βββ pipeline.py # Main RAG orchestration
β β
β βββ data/ # Data directories
β β βββ raw/ # Original documents
β β βββ processed/ # Processed documents
β β βββ embeddings/ # Vector embeddings
β β
β βββ db/ # Vector database storage
β β βββ chromadb/ # ChromaDB files
β β
β βββ models/ # Model storage
β β βββ embeddings/ # Embedding models
β β βββ llm/ # Language models
β β
β βββ retrieval/ # Document retrieval
β β βββ __init__.py
β β βββ retriever.py
β β
β βββ generation/ # Answer generation
β β βββ __init__.py
β β βββ generator.py
β β
β βββ ui/ # User interface
β β βββ app.py # Main Streamlit app
β β βββ components/ # UI components
β β βββ query_input.py
β β βββ answer_display.py
β β βββ sources_panel.py
β β βββ sidebar.py
β β
β βββ utils/ # Utilities
β β βββ config.py # Configuration
β β βββ logger.py # Logging setup
β β
β βββ tests/ # Test files
β
βββ .gitignore # Git ignore patterns
βββ .python-version # Python version specification
βββ pyproject.toml # Project metadata and dependencies
βββ uv.lock # Locked dependency versions
βββ main.py # Main entry point
βββ README.md # This file
βββ setup_project_structure.py # Project structure setup script
Edit src/utils/config.py to customize:
CONFIG = {
"models": {
"embedding_model": "gemma", # Embedding model
"llm_model": "flan-t5-base" # Generation model
},
"system": {
"use_gpu": True, # Enable GPU if available
"max_context_tokens": 1024 # Max context length
}
}-
Start the application:
streamlit run src/ui/app.py
-
Enter your question in the search box
-
Review the answer with confidence score
-
Check sources in the right panel
-
Adjust settings in the sidebar as needed
- "What are the main conclusions of the research?"
- "Summarize the methodology used in the study"
- "What limitations are mentioned?"
- "Compare findings across different sections"
- Query Input: Natural language question interface with examples
- Answer Display: Generated responses with confidence scores
- Source Citations: Ranked relevant documents with relevance percentages
- Settings Panel: Adjustable retrieval and generation parameters
- Query History: Recent queries for quick re-submission
- Offline Indicator: Visual confirmation of local-only operation
- β Phase 1: Project Setup & Structure
- β Phase 2: Data & Document Handling
- π Phase 3: Retrieval + Generation Pipeline (In Progress)
- π Phase 4: Explainability & Confidence Scoring (Planned)
- β Phase 5: Local UI (Streamlit) - Current Phase Complete
- π Phase 6: Testing & Optimization (Planned)
- π Phase 7: Documentation & Deployment (Planned)
- 100% Offline: No internet connection required after setup
- Local Storage: All data stays on your machine
- No Telemetry: No usage data collected or transmitted
- Open Source: Fully auditable codebase
- No External APIs: All models run locally
Problem: streamlit: command not found
# Verify Streamlit is installed
pip list | grep streamlit
# If not found, install it
pip install streamlit
# Or with UV
uv add streamlitProblem: ModuleNotFoundError: No module named 'src'
# Make sure you're in the project root
cd Private-Trustworthy-RAG-Assistant
# Run from root directory
streamlit run src/ui/app.pyProblem: Port 8501 already in use
# Use a different port
streamlit run src/ui/app.py --server.port 8502Problem: CUDA not available
# Check PyTorch CUDA support
python -c "import torch; print(torch.cuda.is_available())"
# If False and you have NVIDIA GPU, reinstall PyTorch with CUDA:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118# Create feature branch
git checkout -b feature/your-feature-name
# Make changes and commit
git add .
git commit -m "Description of changes"
# Push to remote
git push origin feature/your-feature-name
# Merge when ready
git checkout main
git merge feature/your-feature-nameWith UV (Recommended):
uv add package-nameWith pip:
pip install package-name
# Then manually update pyproject.toml- Create component file in appropriate directory
- Update
__init__.pyif needed - Add imports to relevant modules
- Update this README with new features
- Add tests in
src/tests/
- Fork the repository
- Create your feature branch
- Commit your changes
- Push to the branch
- Open a Pull Request
Current project uses:
- Python: 3.13.3
- Streamlit: 1.50.0 - Web UI framework
- PyTorch: 2.7.1 - Deep learning framework
- Transformers: 4.54.0 - Hugging Face models
- LangChain: For RAG orchestration
- ChromaDB: Vector database
- Pandas: Data manipulation
See pip list output for complete dependency list.
- Embeddings: Gemma 300D (sentence-transformers)
- Generation: Flan-T5-base (Google)
- Vector DB: ChromaDB
All models downloaded automatically on first run.
Minimum:
- 8GB RAM
- 10GB free disk space
- CPU: 4 cores
Recommended:
- 16GB RAM
- 20GB free disk space
- GPU: NVIDIA with 4GB+ VRAM
- CPU: 8 cores
Current Development System:
- Python 3.13.3
- Windows with PowerShell
- UV package manager
β Completed:
- Project structure setup
- UI components (query input, answer display, sources panel, sidebar)
- Mock data for testing UI without pipeline
- Streamlit app with modular design
- Configuration management
- Logging system
π In Progress:
- RAG pipeline integration (Phase 3)
- Document processing and embedding
π Next Steps:
- Connect UI to actual RAG pipeline
- Implement document upload and processing
- Add confidence scoring
- Enable GPU acceleration
- UI is Ready: The Streamlit interface works with mock data - you can test it immediately
- Pipeline Integration: When Phase 3 is complete, update
process_query()inapp.py - Mock Mode: Current UI uses
get_mock_response()for testing - Component-Based: Each UI element is modular and can be updated independently
- Settings Work: All sidebar settings are stored in session state
For issues or questions:
- Check troubleshooting section above
- Review existing GitHub issues
- Create new issue with details
- Built with Streamlit, LangChain, and Hugging Face Transformers
- Inspired by privacy-first AI principles
- Designed for researchers, students, and privacy-conscious users
A local, private, and trustworthy Retrieval-Augmented Generation (RAG) assistant that runs entirely offline on your machine. This project implements a state-of-the-art RAG pipeline with advanced retrieval techniques, including semantic chunking, query transformation (HyDE + routing), hybrid search (dense + sparse with RRF), and cross-encoder re-ranking.
- 100% Private & Offline: All processing (embedding, retrieval, generation) happens locally. No data leaves your machine.
- Advanced RAG Pipeline:
- Semantic Chunking: Splits text based on semantic similarity using sentence embeddings.
- Query Transformation: Uses HyDE (Hypothetical Document Embeddings) and query routing (definitional/procedural).
- Hybrid Search: Combines dense vector search (ChromaDB with BGE embeddings) and sparse BM25 search, fused with Reciprocal Rank Fusion (RRF).
- Cross-Encoder Re-Ranking: Re-ranks fused results using a lightweight cross-encoder for improved relevance.
- Explainability: Provides source citations, confidence scores, and context preview for transparency.
- Modular & Extensible: Clean separation of concerns with well-defined interfaces.
- Streamlit UI: Intuitive chat interface for interacting with your local knowledge base.
- FastAPI Backend: High-performance asynchronous API for serving the RAG pipeline.
graph TD
A[User Query] --> B(Query Transformer)
B --> C{HyDE Transform}
B --> D{Query Routing}
C --> E[Hybrid Search]
D --> E
E --> F[Dense Retrieval (ChromaDB/BGE)]
E --> G[Sparse Retrieval (BM25)]
F & G --> H[RRF Fusion]
H --> I[Cross-Encoder Re-Ranking]
I --> J[Context Assembly]
J --> K[LLM Generation (Phi-3 GGUF)]
K --> L[Answer with Sources]
- Python 3.11 or higher
- Git
- uv (for fast package installation) or pip
-
Clone the repository:
git clone https://github.com/your-username/Private-Trustworthy-RAG-Assistant.git cd Private-Trustworthy-RAG-Assistant -
Install dependencies (using uv):
uv pip install -r requirements.txt
Or with pip:
pip install -r requirements.txt
-
Download the GGUF model: The pipeline uses the Phi-3-mini-4k-instruct-Q4_K_M.gguf model. Place the downloaded
.gguffile in themodels/directory.mkdir -p models # Download the model and place it in models/ # Example: wget -O models/Phi-3-mini-4k-instruct-q4_k_m.gguf "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4_k_m.gguf"
-
Initialize the embedding model: The BGE embedding model will be downloaded automatically on first run to
local_models_cache/.
uvicorn src.api.main:app --host 0.0.0.0 --port 8000streamlit run src/ui/app.pyOpen your browser to http://localhost:8501 to interact with the chat interface.
- Place your documents (PDF, TXT, MD) in the
docs/directory. - Use the "Upload files" section in the Streamlit sidebar to ingest documents into the knowledge base.
Alternatively, you can run the ingestion script:
python src/data/ingest.py --docs-dir ./docs
Configuration is managed via src/utils/config.py. Key settings include:
- Embedding Model:
BAAI/bge-base-en-v1.5(local) - LLM Model: Phi-3-mini-4k-instruct (GGUF)
- Vector Store: ChromaDB (with FAISS fallback)
- Chunking: Semantic chunking with dynamic threshold
- Retrieval: Hybrid (Dense + Sparse) with RRF
- Re-ranking: Cross-encoder (ms-marco-MiniLM-L-6-v2)
pytestWe use ruff for linting and formatting. To check and fix:
ruff check .
ruff format .This project is licensed under the MIT License - see the LICENSE file for details.
- Phi-3 by Microsoft
- BAAI/bge-base-en-v1.5 for embeddings
- llama-cpp-python for GGUF inference
- ChromaDB for vector storage
- rank-bm25 for sparse retrieval
- sentence-transformers for cross-encoder re-ranking
- Streamlit and FastAPI for the UI and API
This software is for educational and research purposes only. While designed to be private and secure, users are responsible for ensuring compliance with applicable laws and regulations when processing sensitive data.
a1dc2fd (feat: implement phase 3 advanced RAG architecture with FastAPI, GGUF, and Hybrid Search)