Free LLM Router 🚀

Bring your own LLM API keys and route OpenAI Chat Completions, OpenAI Responses/Codex, and Claude Code through one gateway with a beautiful, self-hosted dashboard.

Free LLM Router handles provider failover, request deduplication, analytics, rate-limit cooldowns, and more—making it easier, cheaper, and more reliable to use AI in your applications.

📖 What is this product?

Imagine you have several different AI assistants (like different brands of smart speakers or chat apps) that you pay for separately. Each one has its own strengths—some are fast, some are cheap, some are good at specific tasks like coding or understanding images.

Free LLM Router is like a smart switchboard that lets you use ALL of these AI assistants through ONE simple connection, without having to manage each service separately.

How it works

1. You sign in once — Log into the dashboard with your Clerk account.
2. You add cloud keys or pair Ollama — Connect hosted AI services, private models on a computer you control, or both.
3. You get one master key — The system gives you a special code (starting with flm_) that you use in your apps.
4. Your apps talk to the router — Instead of your app talking directly to each AI service, it talks to this router.
5. The router chooses the best AI — Behind the scenes, it picks an eligible cloud or local model based on capabilities, policy, speed, availability, health, quotas, and local routing gates.

---

✨ Key Features

• One Connection, Many AIs: Talk to OpenRouter, Groq, Anthropic, OpenAI, DeepSeek, Together AI, and more through one simple API.
• Automatic Failover & Circuit Breakers: If an AI service is having issues, the router silently switches to another one.
• Rate-Limit & Quota Protection: Automatically honors Retry-After headers and tracks your free-tier limits to avoid unexpected charges.
• Request Deduplication: Coalesces identical in-flight requests and briefly reuses successful responses, saving provider quota.
• Model Aliases & Capabilities: Create virtual models (like vision-router) that only route to providers with specific capabilities.
• Comprehensive Dashboard: Manage cloud and local providers with matching card flows, test routed, exact cloud, or exact Local LLM prompts in the Playground, and inspect detailed request analytics and performance timings.
• Full Compatibility: Works out of the box as an OpenAI-compatible endpoint, Codex CLI gateway, and Claude Code gateway.
• Private Local Ollama Nodes: Pair Ollama through a protected loopback agent and ngrok tunnel, inspect connections in Providers, manage routing gates, limits, lifecycle, and model capabilities under Settings → Router & Policies → Local LLM Properties, and test exact local models in the Playground.

---

🚀 Getting Started

Prerequisites

• Node.js (version 20 or higher)
• Git
• A Clerk account (for authentication)

1. Installation

git clone https://github.com/Pr0fess0rOP/free-llm-router.git
cd free-llm-router
npm install

2. Configuration

Create a .env file in the root directory:

# Required for Clerk authentication
CLERK_SECRET_KEY=sk_test_your_clerk_secret_key
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_test_your_clerk_publishable_key

# Recommended when deployed behind Vercel, a reverse proxy, or a custom domain
CLERK_AUTHORIZED_PARTIES=https://your-dashboard.example.com

# Strongly recommended for production (encrypts stored API keys)
# Generate with: openssl rand -base64 32 | tr '+/' '-_' | tr -d '='
ACCOUNT_ENCRYPTION_KEY=your_32_byte_base64url_secret

3. Start the Server

npm run dev

The dashboard will start at http://localhost:8787.

4. Create a Router & Connect Models

1. Open http://localhost:8787 and sign in.
2. Create your first router to generate your flm_... router key.
3. Go to the Providers page and add your API keys for services like Groq or OpenRouter.
4. Optionally open Providers → Local LLMs and pair an Ollama computer.
5. Head to the Playground to test a routed request, exact hosted model, or exact Local LLM immediately.

For detailed setup instructions, see Getting Started.

To connect Ollama running on your own computer, see Connect a private Ollama node.

---

📚 Documentation

Detailed documentation has been moved to the docs/ folder to keep this README concise:

• Detailed Manual & Feature Guide - Comprehensive explanation of all features, routing logic, and architecture.
• Architecture & Flow - Deep dive into request flows and system design.
• Simple Explanation - A non-technical overview of the product.
• Getting Started - Step-by-step setup and testing guide.
• Connect a Local LLM - Pair, configure, route to, test, and remove a private Ollama node.
• Local-node Security - Network boundaries, credentials, signed requests, and agent limits.
• Local-node Troubleshooting - Diagnose Ollama, ngrok, heartbeat, routing, and credential problems.
• Changelog - Release-by-release feature and behavior changes.

---

🤝 Contributing

We welcome contributions! Please see our Contributing Guidelines to learn how to get started, run tests, and submit pull requests.

🛡️ Security

Please review our Security Policy for information on reporting vulnerabilities and our security practices.

💬 Support & Community

• Code of Conduct: Please read our Code of Conduct.
• Need Help? See our Support Guide on how to get assistance or report issues.

---

📄 License

This project is licensed under the MIT License.