An OpenAI-compatible LLM gateway you own and run.

Route one endpoint to 40+ providers, issue virtual keys, enforce budgets, and track usage.

Docs · otari.ai · Launch blog · Discord

Otari sits between your applications and model providers. It authenticates requests, resolves provider credentials, enforces budgets before dispatch, and records usage afterwards. Provider calls go through any-llm.

Run Otari standalone with your own database and provider keys, or connect a data plane to otari.ai.

Why Otari

• One OpenAI and Anthropic-compatible endpoint for many providers
• Provider credentials stay behind the gateway
• Revocable API keys with user, workspace, and model scope
• Budget checks before spend and usage records after settlement
• Local routing policies for failover, weighting, and learned selection
• Optional code execution, web search, MCP, guardrails, and file understanding

Quickstart

Run an ephemeral standalone gateway with Docker:

docker run --rm -p 8000:8000 \
  -e OTARI_MASTER_KEY=SET_A_MASTER_KEY \
  -e OPENAI_API_KEY=YOUR_OPENAI_KEY \
  -e OTARI_CONFIG_YAML='default_pricing: true' \
  mzdotai/otari:latest \
  otari serve

On the first empty database, Otari creates an API key and prints it once:

No API keys found. Created bootstrap key for first run. Save this key now:
gw-...

Send a request with that key:

curl http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer gw-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai:gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hello."}]
  }'

OpenAI clients work by setting base_url to http://localhost:8000/v1.

This container uses SQLite inside the container and is deleted when it stops. Use the Compose setup below for persistent data.

Run the full stack

git clone https://github.com/mozilla-ai/otari
cd otari
cp config.example.yml config.yml
# Set a master key, provider credentials, and pricing in config.yml.
docker compose pull
docker compose up -d

Compose runs Otari with PostgreSQL. Optional profiles add the code sandbox, web search, and guardrail services:

docker compose --profile code-exec --profile web-search --profile guardrails up -d

The dashboard is served at http://localhost:8000/. To store provider keys through the dashboard, set OTARI_SECRET_KEY to a Fernet key generated by otari gen-secret-key.

One-click deployment templates are available for Render and Railway.

Runtime modes

Mode	Purpose
Standalone	One process serves management and inference using local storage.
Hosted	A multi-tenant control plane; inference runs on connected gateways.
Hybrid	A data-plane gateway resolves credentials and reports usage to otari.ai.

When OTARI_MODE is unset, OTARI_AI_TOKEN selects hybrid mode; otherwise Otari defaults to standalone. See Runtime modes.

API and dashboard

The core completion routes are:

• POST /v1/chat/completions
• POST /v1/messages
• POST /v1/responses

Standalone also serves the broader OpenAI-compatible and management APIs. The running server publishes Swagger UI at /docs and OpenAPI at /openapi.json. See API reference.

The dashboard manages providers, models, routing, tools, keys, members, budgets, settings, activity, and usage. Its navigation adapts to the deployment mode and the signed-in person's authority. See Admin dashboard.

Development

git clone https://github.com/mozilla-ai/otari
cd otari
uv sync --dev
cp config.example.yml config.yml
uv run otari serve --config config.yml

For local development without PostgreSQL, change database_url to sqlite+aiosqlite:///./otari.db.

Common checks:

make test
make lint
make typecheck
pnpm --dir web run lint

See CONTRIBUTING.md before opening a pull request.

Documentation

• Quickstart
• Deployment
• Configuration
• Models
• Routing
• Access control
• Built-in tools
• SDK and agent integrations
• Architecture

License

Apache 2.0. See LICENSE.