@@ -5,7 +5,7 @@ All notable changes to OpenVidia will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [2.1.0] - 2026-08-01
 
 ### Added
 - **Thinking toggle** — `auto` / `on` / `off` next to the active model. A
@@ -37,6 +37,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - CONTRIBUTING.md guide for new contributors
 - SECURITY.md with vulnerability disclosure process
 - Error logging improvements for better debugging
+- **Graph Engine** — `openvidia.graph_engine`: a `Hub` with mailboxes, `run_agent`
+  with `max_iterations` / `budget_tokens` / `timeout_s` caps, spawn-and-kill
+  subagent tools with a depth bound, and `generate_and_verify` generator-verifier
+  loops with explicit rubrics (`APPROVED` / `REJECTED`). 10 regression tests in
+  `tests/test_graph_engine.py`
+- **Per-key × per-model scoring** — `KeyModelHealth` learns a composite score
+  (success rate + median time to first token) from live traffic; candidate
+  selection now prefers the best key for the requested model
+- **Adaptive RPM ceiling** — each key's sliding window learns the real ceiling
+  from `Retry-After` headers on 429s and never spends the full 28 RPM budget
+  against a tighter upstream limit
+- **Multi-endpoint NVIDIA routing** — extra upstream endpoints via
+  `endpoints.json` or `OPENVIDIA_UPSTREAM_ENDPOINTS`; 60s blacklist on ≥500
+  errors, automatic recovery on success
+- **Multi-node sync via Redis (opt-in)** — cooldowns, invalid keys, pool
+  throttling and model circuit breakers shared across instances over pub/sub
+  (`redis_config.json` / `OPENVIDIA_REDIS_URL`, extra `openvidia[redis]`)
+- **Embedding cache** — in-memory cache for `/v1/embeddings` (TTL 300s, SHA-256
+  keyed by model + input); hit/miss counters on `/ops/health`
+- **Free-tier provider fallback** — when every NVIDIA key is exhausted, the
+  catch-all fails over to OpenAI-compatible providers declared in
+  `providers.json` (unresolved keys skip the provider silently)
+- **Ops endpoints** — `/ops/keys` and `/ops/health`, protected by the
+  control-plane token, exposing pool, models, endpoints, embedding cache and
+  Redis state
+- Jcode as a first-class CLI target (`openvidia setup` configures it)
+- **Smart key rotation** — rotation ordered by in-flight cost, per-key RPM and
+  consecutive failures, so concurrent traffic spreads across the pool instead
+  of piling onto key[0]
+- **Network-error circuit breaker** — three consecutive network errors stop
+  rotation (503) instead of burning every key; 504 timeouts cool the key for
+  30s without consuming the attempt budget
 
 ### Changed
 - Compaction serves a cached summary plus every later message verbatim while
@@ -48,6 +80,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Concurrent requests on one conversation share a single summarize
 - Version bumped from 2.0.0 to 1.0.0 (first stable release)
 - Test suite uses pytest with async support
+- Key selection is model-aware: candidates are ordered by a per-key × per-model
+  score learned from live traffic (see Added)
+- The test suite ships 297 regression tests (proxy, shims, compaction, graph
+  engine, delivery features)
 
 ### Removed
 - **The pinned default model.** `DEFAULT_MODEL = "deepseek-ai/deepseek-v4-pro"`
 
@@ -6,7 +6,7 @@
 ![Ruff](https://img.shields.io/badge/Ruff-passed-261230?logo=ruff&logoColor=white)
 ![Stars](https://img.shields.io/github/stars/ciroautuori/openvidia?style=social)
 ![Last Commit](https://img.shields.io/github/last-commit/ciroautuori/openvidia)
-![Version](https://img.shields.io/badge/version-2.0.0-green)
+![Version](https://img.shields.io/badge/version-2.1.0-green)
 
 **Multi-key proxy for NVIDIA NIM with a native desktop dashboard.**
 
@@ -123,10 +123,16 @@ NVIDIA's free NIM tier limits each API key to ~40 RPM. Aggressive bursts trigger
 - **Per-key cooldown timers** — on real 429s, trusts `Retry-After` exactly (no multiplication); distinguishes transient worker saturation (`ResourceExhausted`) from true RPM limits and never burns key cooldowns for the former
 - **Network-error circuit breaker** — `ConnectTimeout`/`ConnectError` don't trigger key cooldown (the key is fine). After 3 consecutive network errors across different keys, rotation stops entirely and returns 503 (the network is down, no key can help)
 - **504 model timeouts don't burn keys** — gateway timeouts cool the key for 30s but don't consume the attempt budget, since the model (not the key) is the bottleneck
-- **Sliding-window RPM limiting** — keeps each key under 28 RPM (safe margin below 40)
+- **Sliding-window RPM limiting** — keeps each key under 28 RPM (safe margin below 40); the ceiling self-tunes down from real `Retry-After` headers, so the pool never spends its full budget against a tighter upstream limit
 - **Health checks** — revives keys whose cooldowns have expired, but skips 401/403 invalid keys (dead keys stay dead)
 - **No silent model substitution** — the model you select is the model that answers; if it fails on every key you are told so, never handed output from a different model
 - **Auto-compaction** — summarizes long histories so requests never fail on context overflow ([details](#auto-compaction))
+- **Per-key × per-model scoring** — rotation is ordered by a composite score (success rate + median time to first token) learned per key and per model from live traffic: the best key for the requested model goes first
+- **Multi-endpoint routing** — extra NVIDIA endpoints (`endpoints.json` or `OPENVIDIA_UPSTREAM_ENDPOINTS`), a 60s blacklist on ≥500 errors, automatic recovery
+- **Multi-node sync via Redis (opt-in)** — cooldowns, invalid keys, pool throttling and model circuit breakers shared across instances ([details](#resilience--scale))
+- **Embedding cache** — `/v1/embeddings` hits are served from memory (TTL 300s) without touching upstream quota
+- **Free-tier provider fallback** — when every NVIDIA key is exhausted, requests fail over to OpenAI-compatible providers declared in `providers.json` ([details](#resilience--scale))
+- **Graph engine** — programmatic multi-agent orchestration (Hub, run_agent, generator-verifier loops) on top of the proxy ([details](#graph-engine))
 
 ---
 
@@ -507,6 +513,9 @@ update-desktop-database ~/.local/share/applications/
 | `timeouts.json` | Upstream timeouts (optional — see [Slow models](#slow-models)) |
 | `model_limits.json` | Context windows the proxy learned by itself — never edit by hand |
 | `model_options.json` | Reasoning toggle + the payload used to express it (see [Thinking](#thinking-reasoning-toggle)) |
+| `endpoints.json` | Extra NVIDIA upstream endpoints (optional — see [Resilience & Scale](#resilience--scale)) |
+| `redis_config.json` | Redis URL for multi-node sync (optional — `OPENVIDIA_REDIS_URL` also works) |
+| `providers.json` | Free-tier fallback providers (optional — name, base URL, env var for the key, model map) |
 
 Add keys via the dashboard (**Keys** section) or edit `keys.json`:
 
@@ -663,6 +672,96 @@ DEFAULT_COOLDOWN = 30.0   # Network errors, unknown 5xx
 | `POST` | `/api/restart` | Zero-downtime restart (spawn new, kill old) |
 | `GET` | `/api/logs/stream` | SSE log stream (real-time) |
 
+### Ops (control plane — token protected)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/ops/keys` | Per-key status: cooldown, RPM + learned ceiling, per-model score, in-flight |
+| `GET` | `/ops/health` | Pool, per-model health, endpoints, embedding cache, Redis sync, recent logs |
+
+---
+
+## Graph Engine
+
+Programmatic multi-agent orchestration on top of the proxy, following the
+[graph-engine coordination patterns](https://www.anthropic.com/engineering/graph-engine-coordination):
+a single `Hub` owns mailboxes and tasks, agents run with hard caps, and workers
+can be spawned, messaged and killed from inside the loop. Runs on any model the
+proxy serves.
+
+```python
+import asyncio
+from openvidia import Hub, Provider, ToolSpec, run_agent
+
+async def add(args, ctx):
+    return str(args["a"] + args["b"])
+
+async def main():
+    hub = Hub(default_provider=Provider())  # http://127.0.0.1:1919/v1 — the proxy above
+    try:
+        result = await run_agent(
+            hub, "math",
+            system="You are a calculator. Always use the add tool.",
+            first_turn="What is 5 + 2?",
+            tools=[ToolSpec(
+                name="add", description="Sum two integers",
+                handler=add,
+                parameters={"type": "object", "properties": {
+                    "a": {"type": "integer"}, "b": {"type": "integer"}},
+                    "required": ["a", "b"]},
+            )],
+        )
+        print(result)
+    finally:
+        await hub.close()
+```
+
+Highlights: `run_agent` caps every run with `max_iterations`, `budget_tokens`
+and `timeout_s`; `graph_tools(hub, worker_factory=...)` provides
+`get_status`, `send_message`, `wait_for_message`, `create_subagents` (depth-bound:
+workers cannot spawn workers) and `kill_subagents`; `generate_and_verify` runs
+generator → verifier loops with explicit rubrics until `APPROVED` or
+`max_rounds`; providers are swappable per worker (a `Provider` or a
+`Callable[[str], Provider]`). 10 regression tests ship in `tests/test_graph_engine.py`.
+
+---
+
+## Resilience & Scale
+
+Beyond the core pool, OpenVidia is built to survive upstream trouble and to
+scale across machines.
+
+- **Multi-endpoint routing** — by default requests go to
+  `integrate.api.nvidia.com`; add endpoints via `endpoints.json` or
+  `OPENVIDIA_UPSTREAM_ENDPOINTS` (comma-separated). A status ≥500 blacklists an
+  endpoint for 60s; a success revives it. Deterministic client errors never
+  retry on another endpoint.
+- **Multi-node sync (Redis, opt-in)** — one proxy per machine, shared state:
+  cooldowns, invalid keys, pool throttling and model circuit breakers propagate
+  over pub/sub so a 429 learned on node A protects node B. Enable with
+  `OPENVIDIA_REDIS_URL` or `redis_config.json` (`{"url": "..."}`); install the
+  extra with `pip install "openvidia[redis]"`.
+- **Embedding cache** — repeated `/v1/embeddings` requests (same model + input)
+  are answered from an in-memory cache (TTL 300s) keyed by SHA-256; hit/miss
+  counters are exposed on `/ops/health`.
+- **Free-tier provider fallback** — when every NVIDIA key is exhausted, the
+  catch-all fails over to OpenAI-compatible providers listed in
+  `providers.json`:
+
+  ```json
+  [
+    {
+      "name": "glm",
+      "base_url": "https://api.example.com/v1",
+      "api_key_env": "GLM_API_KEY",
+      "models": { "deepseek-ai/deepseek-v4-flash": "glm-5" }
+    }
+  ]
+  ```
+
+  A provider without its `api_key_env` set in the environment is skipped
+  silently; model names are rewritten through the `models` map.
+
 ---
 
 ## Tech Stack
@@ -675,6 +774,7 @@ DEFAULT_COOLDOWN = 30.0   # Network errors, unknown 5xx
 - **[tomlkit](https://github.com/sdispater/tomlkit)** — TOML config editing (preserves comments)
 - **Vanilla HTML/CSS/JS** — zero frontend build, no node_modules
 - **Python 3.12+** — single process, no external services
+- **[redis](https://redis.io/)** (optional) — multi-node state sync via `redis-py`
 
 ---
 
 
@@ -0,0 +1,28 @@
+# Graph Engine: mappa wayfinder
+
+Label: `wayfinder:map` — tracker local-markdown (GitHub issues disabilitati sul repo).
+
+## Destination
+
+Modulo `openvidia/graph_engine.py` (~400 righe, async) che implementa il Graph Loop su DeepSeek locale (localhost:1919): Hub + run_agent + tool spawn/status/collect/kill + generate_and_verify, con test pytest verdi e ruff pulito.
+
+## Notes
+
+- domain: python async, httpx, orchestrazione agenti (pattern Anthropic: orchestrator-subagent, generator-verifier, contesto fresco, depth bound)
+- skill: /grilling, /domain-modeling, /tdd
+- OVERRIDE: effort in **full-delivery** — i ticket si risolvono nella sessione stessa, non solo decisioni. Chart minimale, poi implementazione completa.
+
+## Decisions so far
+
+- [wf-001 full-delivery approved](tickets/wf-001-full-delivery.md) — l'utente ha richiesto mappa + implementazione completa nella stessa sessione ("E POI PARTI FULL DELIVERY").
+- [wf-002 API pubblica graph_engine](tickets/wf-002-API-pubblica-graph-engine.md) — `openvidia/graph_engine.py` implementato e verificato: Hub + run_agent + graph_tools + generate_and_verify, 10/10 test, ruff pulito, export da `__init__.py`.
+
+## Not yet specified
+
+- Provider alternativi (non-DeepSeek) via env/CLI
+- Persistenza sessioni graph su disco
+
+## Out of scope
+
+- Multi-nodo Redis / HTTP2 pool upstream (roadmap separata del proxy)
+- Frontend dashboard del graph engine
@@ -0,0 +1,11 @@
+# wf-001 — full-delivery approved
+
+Label: `wayfinder:task` · HITL (deciso dall'utente in sessione) · **CLOSED**
+
+## Question
+
+L'utente carica wayfinder e dice "E POI PARTI FULL DELIVERY": l'effort Graph Engine deve essere sia mappato sia implementato per intero nella stessa sessione.
+
+## Resolution
+
+Approvato. La mappa (docs/wayfinder/map.md) è chart minima, la costruzione del grafo parte subito: graph_engine.py + tests + verify pytest/ruff.
@@ -0,0 +1,13 @@
+# wf-002 — API pubblica graph_engine
+
+Label: `wayfinder:task` · AFK · **CLOSED**
+
+blocked-by: wf-001
+
+## Question
+
+Quale public API espone `openvidia/graph_engine.py`? Design già deciso in sessione: `Hub` (register/new_name/close/snapshot), `run_agent`, `generate_and_verify`, `ToolSpec`/`WorkerSpec`/`Provider` dataclass, `graph_tools` (get_status, send_message, wait_for_message, create_subagents, kill_subagents), export da `__init__.py`.
+
+## Resolution
+
+Implementata e verificata (10/10 test, ruff pulito): `openvidia/graph_engine.py` con `Provider` (chat async OpenAI-style), `ToolSpec`, `WorkerSpec`, `Hub` (status/mailbox/task/events), `run_agent` (loop con max_iterations, budget_tokens, timeout, crash su risposta vuota, tool error recovery), `graph_tools` (closure con factory per spawn/kill, depth bound: i worker non spawnano), `generate_and_verify` (rubric esplicita, APPROVED/REJECTED, feedback nel round successivo). Export public da `openvidia/__init__.py`. Test: `tests/test_graph_engine.py` (10 test, ScriptedProvider).
@@ -6,3 +6,25 @@
 """
 
 from __future__ import annotations
+
+from openvidia.graph_engine import (
+    AgentCtx,
+    Hub,
+    Provider,
+    ToolSpec,
+    WorkerSpec,
+    generate_and_verify,
+    graph_tools,
+    run_agent,
+)
+
+__all__ = [
+    "AgentCtx",
+    "Hub",
+    "Provider",
+    "ToolSpec",
+    "WorkerSpec",
+    "generate_and_verify",
+    "graph_tools",
+    "run_agent",
+]
@@ -382,6 +382,7 @@ async def _stream_anthropic(
 
     state.stats.success += 1
     state.stats.record_key_usage(used_key, ok=True)
+    state.record_key_model_result(used_key, chat_payload.get("model", ""), ok=True)
     if used_idx is not None:
         state.log_cb(f"✔ key[{used_idx}] OK")
 
@@ -701,6 +702,7 @@ async def handle_anthropic_messages(
 
     state.stats.success += 1
     state.stats.record_key_usage(used_key, ok=True)
+    state.record_key_model_result(used_key, chat_payload.get("model", ""), ok=True)
     if used_idx is not None:
         state.log_cb(f"✔ key[{used_idx}] OK")
 
 
@@ -465,3 +465,44 @@ def jcode_config_path() -> Path:
     Jcode uses a fixed location (not XDG), the same on every platform.
     """
     return Path.home() / ".jcode" / "config.toml"
+
+
+def upstream_endpoints() -> list[str]:
+    """Upstream endpoints for the router: env beats file beats default.
+
+    La lista si mantiene ordinata: il primo endpoint è il preferito, gli
+    altri sono fallback per il failover.
+    """
+    env = os.environ.get("OPENVIDIA_UPSTREAM_ENDPOINTS", "").strip()
+    if env:
+        endpoints = [e.strip().rstrip("/") + "/" for e in env.split(",") if e.strip()]
+        if endpoints:
+            return endpoints
+    try:
+        data = json.loads((config_dir() / "endpoints.json").read_text())
+    except (FileNotFoundError, json.JSONDecodeError, OSError, TypeError):
+        return [UPSTREAM_BASE]
+    if not isinstance(data, dict):
+        return [UPSTREAM_BASE]
+    raw = data.get("endpoints")
+    if not isinstance(raw, list):
+        return [UPSTREAM_BASE]
+    endpoints = [e for e in raw if isinstance(e, str) and e.strip()]
+    if not endpoints:
+        return [UPSTREAM_BASE]
+    return [e.strip().rstrip("/") + "/" for e in endpoints]
+
+
+def redis_url() -> str:
+    """Redis URL for cross-node sync; empty string means standalone mode."""
+    env = os.environ.get("OPENVIDIA_REDIS_URL", "").strip()
+    if env:
+        return env
+    try:
+        data = json.loads((config_dir() / "redis_config.json").read_text())
+    except (FileNotFoundError, json.JSONDecodeError, OSError, TypeError):
+        return ""
+    if not isinstance(data, dict):
+        return ""
+    url = data.get("url", "")
+    return url if isinstance(url, str) else ""