Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions llm.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# OmniRoute

> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 36+ AI providers — all through a single OpenAI-compatible endpoint.

## Overview

OmniRoute solves the problem of managing multiple AI provider subscriptions, quotas, and rate limits. It sits between your AI-powered tools (IDE agents, CLI tools) and AI providers, routing requests intelligently through a 4-tier fallback system: Subscription → API Key → Cheap → Free.

**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.

## Tech Stack

- **Runtime:** Node.js >= 18

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There is an inconsistency regarding the required Node.js version. This file specifies >= 18, while the README.md (line 917) states Node.js 20+. These should be aligned to avoid confusion during environment setup.

- **Runtime:** Node.js >= 20

- **Framework:** Next.js 16 (App Router) with TypeScript

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Next.js 16 is not yet released (the current stable version is 15). This appears to be a typo and should be corrected to reflect the actual version used in the project.

- **Framework:** Next.js 15 (App Router) with TypeScript

- **Database:** SQLite via better-sqlite3 (local, zero-config)
- **State management:** Zustand (client), lowdb (server JSON persistence)
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth

Copilot AI Feb 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package used is "bcryptjs" not "bcrypt". While they are functionally similar, the actual dependency in package.json is "bcryptjs": "^3.0.3".

Suggested change
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
- **Auth:** OAuth 2.0 (PKCE) for providers, bcryptjs for local user auth

Copilot uses AI. Check for mistakes.
- **Background jobs:** Custom token health check scheduler
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)

## Project Structure

```
/
├── src/ # Main application source
│ ├── app/ # Next.js App Router pages and API routes
│ │ ├── (dashboard)/ # Dashboard UI pages (providers, combos, analytics, logs, etc.)
│ │ ├── api/ # REST API endpoints
│ │ │ ├── v1/ # OpenAI-compatible API (chat, models, embeddings, images, audio)
│ │ │ ├── oauth/ # OAuth flows per provider (authorize, exchange, callback)
│ │ │ ├── providers/ # Provider CRUD and batch testing
│ │ │ ├── models/ # Dashboard model listing and aliases
│ │ │ ├── combos/ # Combo CRUD (multi-model fallback chains)
│ │ │ └── ... # Other endpoints (usage, logs, health, settings, etc.)
│ │ └── login/ # Login page
│ ├── domain/ # Domain types and business logic interfaces
│ ├── lib/ # Core libraries
│ │ ├── db/ # SQLite database layer (providers, combos, prompts, logs)
│ │ ├── oauth/ # OAuth providers, services, and utilities
│ │ │ ├── providers/ # Provider-specific OAuth configs (GitHub, Google, Claude, etc.)
│ │ │ ├── services/ # Provider-specific token exchange logic
│ │ │ └── utils/ # PKCE, callback server, token helpers
│ │ ├── cloudSync.ts # Cloud sync via Cloudflare Workers
│ │ ├── tokenHealthCheck.ts # Background OAuth token refresh scheduler
│ │ └── localDb.ts # Unified database access layer
│ ├── shared/ # Shared utilities, components, and constants
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, etc.)
│ │ ├── constants/ # Provider definitions, model lists, pricing
│ │ └── utils/ # Helpers (auth, CORS, error codes, machine ID)
│ ├── sse/ # SSE proxy pipeline
│ │ ├── services/ # Auth resolution, format translation, response handling
│ │ └── middleware/ # Rate limiting, circuit breaker, caching, idempotency
│ ├── store/ # Zustand client-side stores
│ ├── types/ # TypeScript type definitions
│ ├── proxy.ts # Main proxy request handler
│ └── server-init.ts # Server initialization (DB, health checks)
├── open-sse/ # Standalone SSE server (npm workspace)
│ ├── config/ # Model registries (embedding, image, audio, rerank, moderation)
│ ├── handlers/ # Request handlers per API type
│ └── translators/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses)

Copilot AI Feb 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The directory name should be translator/ (singular) instead of translators/ (plural). The actual directory in the codebase is open-sse/translator/.

Suggested change
│ └── translators/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses)
│ └── translator/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses)

Copilot uses AI. Check for mistakes.
├── tests/ # Test suites
│ ├── unit/ # Unit tests (32+ test files)
│ └── integration/ # Integration tests
├── docs/ # Documentation and screenshots
├── bin/ # CLI entry points (omniroute, reset-password)
└── .env.example # Environment variable template
```

## Key Architectural Decisions

1. **OpenAI-compatible API surface:** All incoming requests follow the OpenAI API format (`/v1/chat/completions`, `/v1/models`, etc.). This makes OmniRoute a drop-in replacement for any tool that supports custom OpenAI endpoints.

2. **Provider abstraction via format translators:** Each AI provider (Claude, Gemini, etc.) has a translator in `open-sse/translators/` that converts between the OpenAI format and the provider's native format. This happens transparently.

Copilot AI Feb 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path should reference open-sse/translator/ (singular) instead of open-sse/translators/ (plural). The actual directory in the codebase is open-sse/translator/.

Copilot uses AI. Check for mistakes.

3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider are supported for multi-account rotation.

4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 6 strategies: fill-first, round-robin, power-of-two-choices, random, least-used, cost-optimized.

5. **SSE proxy pipeline (`src/sse/`):** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.

6. **SQLite for persistence:** All state (providers, combos, logs, settings) is stored in a single SQLite database file at `data/omniroute.db`. This keeps the app self-contained and zero-config.

7. **OAuth with PKCE:** OAuth flows use PKCE for security. A local callback server (`src/lib/oauth/utils/server.ts`) handles the redirect. Token refresh is handled by a background job (`tokenHealthCheck.ts`).

## Main Flows

### Proxy Request Flow
1. Client sends OpenAI-format request to `/v1/chat/completions`
2. API key validation (`src/shared/utils/apiAuth.ts`)
3. Model resolution: direct model or combo lookup
4. For combos: iterate through models in fallback order
5. Auth resolution: get credentials for the target provider
6. Format translation: OpenAI → provider native format
7. Upstream request with circuit breaker and rate limiting
8. Response translation: provider → OpenAI format
9. SSE streaming back to client

### OAuth Flow
1. Dashboard initiates `/api/oauth/[provider]/authorize`
2. User completes OAuth login in browser
3. Callback hits `/api/oauth/[provider]/exchange`
4. Tokens stored as a provider connection in SQLite
5. Background job refreshes tokens before expiry

### Model Listing
- `/api/models` — Dashboard endpoint, lists all defined models with aliases
- `/v1/models` — OpenAI-compatible endpoint, lists only models from active providers

## Important Notes for LLMs

1. **Two model endpoints exist:** `/api/models` (dashboard, all models) and `/v1/models` (OpenAI-compatible, active only). Don't confuse them.

2. **Provider IDs vs aliases:** Providers have both an ID (`claude`, `github`) and a short alias (`cc`, `gh`). Models are referenced as `alias/model-name` (e.g., `cc/claude-opus-4-6`).

3. **The `open-sse/` directory is a separate npm workspace** with its own config, handlers, and translators. It handles the actual SSE streaming and format translation.

4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.

Copilot AI Feb 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The environment variable name is incorrect. According to .env.example, the variable is INITIAL_PASSWORD, not ADMIN_PASSWORD. Also, API_KEY is not a single environment variable in .env.example - instead there are provider-specific API keys like DEEPSEEK_API_KEY, GROQ_API_KEY, etc., and API_KEY_SECRET for encryption.

Suggested change
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars include: `PORT`, `NEXT_PUBLIC_BASE_URL`, `INITIAL_PASSWORD`, `API_KEY_SECRET`, and provider-specific keys like `DEEPSEEK_API_KEY`, `GROQ_API_KEY`, etc.

Copilot uses AI. Check for mistakes.

5. **Database migrations:** SQLite schema is managed inline in `src/lib/db/core.ts` and `src/lib/db/providers.ts`. No migration framework — schema changes are applied on startup.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The statement 'No migration framework' is inaccurate. The codebase implements a versioned migration system using a migrationRunner and tracks applied migrations in the _omniroute_migrations table (as seen in src/lib/db/core.ts).

5. **Database migrations:** SQLite schema is managed via a custom migration runner in `src/lib/db/migrationRunner.ts`. Schema changes are applied on startup and tracked in the `_omniroute_migrations` table.

Copilot AI Feb 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This statement is incorrect. The codebase does have a migration framework. There is a src/lib/db/migrationRunner.ts file that manages versioned SQL migrations from src/lib/db/migrations/ directory. The migration system reads numbered SQL files (e.g., 001_initial_schema.sql) and tracks applied migrations in the _omniroute_migrations table. The claim "No migration framework" contradicts the actual implementation.

Suggested change
5. **Database migrations:** SQLite schema is managed inline in `src/lib/db/core.ts` and `src/lib/db/providers.ts`. No migration framework — schema changes are applied on startup.
5. **Database migrations:** SQLite schema is managed via a migration framework in `src/lib/db/migrationRunner.ts`, which applies versioned SQL files from `src/lib/db/migrations/` and tracks applied migrations in the `_omniroute_migrations` table.

Copilot uses AI. Check for mistakes.

6. **Tests use Node.js built-in test runner:** Run `npm test` or `node --test tests/unit/*.test.mjs`. Playwright is used for E2E tests.

7. **The proxy pipeline is in `src/sse/`**, not in `src/app/api/v1/`. The API routes in `src/app/api/v1/` delegate to the SSE server running on a separate Express instance.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The description of the proxy pipeline delegating to a 'separate Express instance' is incorrect based on the provided code. The API routes in src/app/api/v1/ import and call the SSE handlers directly as functions within the Next.js process.

7. **The proxy pipeline is in `src/sse/`**, not in `src/app/api/v1/`. The API routes in `src/app/api/v1/` call the SSE handlers directly from the `open-sse` workspace within the Next.js process.

Copilot AI Feb 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The claim about "SSE server running on a separate Express instance" is inaccurate. The SSE proxy pipeline in src/sse/ is integrated within the Next.js application and delegates directly from the API routes (e.g., src/app/api/v1/chat/completions/route.ts calls handleChat from src/sse/handlers/chat.ts). There is no separate Express server instance for SSE handling - it's all within the Next.js app.

Suggested change
7. **The proxy pipeline is in `src/sse/`**, not in `src/app/api/v1/`. The API routes in `src/app/api/v1/` delegate to the SSE server running on a separate Express instance.
7. **The proxy pipeline is in `src/sse/`**, not in `src/app/api/v1/`. The API routes in `src/app/api/v1/` call into the SSE handlers in `src/sse/` within the same Next.js application (no separate Express server).

Copilot uses AI. Check for mistakes.

## Links

- Repository: https://github.com/diegosouzapw/OmniRoute
- Website: https://omniroute.online
- npm: https://www.npmjs.com/package/omniroute
- Docker Hub: https://hub.docker.com/r/diegosouzapw/omniroute
- Documentation: See `/docs/` directory
Loading