state-taxonomy-and-inventory.md

Router and Dashboard State Taxonomy and Inventory

This document is the canonical inventory for restart-sensitive router and dashboard state.

Use it to answer three questions before adding or changing a stateful feature:

• what kind of state is this
• who owns it
• what durability and recovery behavior users should expect

State Taxonomy

ephemeral_request_state

• Per-request or per-connection working state that may live only in memory.
• Examples: extproc request context, in-flight streaming buffers, active SSE or WebSocket client maps, short-lived retry state.
• Losing this state on process restart is acceptable.

restart_safe_local_state

• State that should survive a local process restart in one workspace or one node, but does not need to be shared across replicas by default.
• Examples: local-dev bootstrap outputs under .vllm-sr/, the local auth SQLite file, Evaluation Plane run bundles, generated runtime config snapshots, and local artifact directories.
• This is acceptable for local development and single-node tooling, but it is not a substitute for product durability in multi-user or multi-replica deployments.

shared_durable_workflow_state

• State that users or operators expect to survive restart and remain authoritative across processes, containers, or replicas.
• Examples: user accounts, audit logs, workflow jobs, campaign state, chat history if presented as a product surface, vector-store metadata, file registries, response history, replay history, model-selection learning state.
• This state should live behind a server-owned storage contract, not browser storage or process memory.

audit_analytics_telemetry

• Append-oriented facts about progress, lifecycle, usage, or recovery that operators need for debugging, audits, or reporting.
• Examples: typed workflow events, startup progress records, replay aggregates, deploy history, durable status transitions.
• This is distinct from request-local logs and must not depend on log scraping.

derived_projection_state

• A query-oriented projection derived from a canonical source of truth.
• This state may be rebuilt, but it should still be persisted if operators or the dashboard rely on it.
• Examples: normalized tables for deployed models, signals, decisions, plugins, DSL snapshots, topology views, or current active config version.

Source Of Truth Rules

• Keep canonical router intent in YAML and DSL, not in a second mutable primary database model.
• If the dashboard needs fast querying, filtering, audit, or joins, create a persisted projection from the active YAML or DSL rather than introducing dual-primary writes.
• Treat .vllm-sr/ and dashboard-data/ as local-dev adapters unless a feature is explicitly documented as single-workspace only.
• Keep live connection registries in memory, but move user-visible entities and workflow progress behind durable records.

Current Inventory

Surface	Primary owner	Current backend / default	Current durability class	Restart behavior today	Scale risk	Recommended direction
Response API stored responses and conversations	router runtime, src/semantic-router/pkg/responsestore/**	Default redis; optional memory for local dev only	shared_durable_workflow_state	Response and conversation history survives restart when using the default Redis backend. The memory backend emits a startup warning and loses all data on restart.	Replica-local only when memory is explicitly selected; Redis backend is shared across replicas	Keep metadata and conversation chain in a durable server-owned store by default for product use. Prefer relational storage for metadata and queryability; keep large payloads in blob/object storage only if needed later.
Router replay records	router runtime, src/semantic-router/pkg/routerreplay/**, src/semantic-router/pkg/extproc/router_replay_setup.go	Disabled by default with process-local memory; optional postgres, redis, milvus, or qdrant when explicitly enabled	audit_analytics_telemetry; durability follows the explicitly selected backend	The default keeps no cross-restart history. PostgreSQL, Redis, Milvus, and Qdrant can preserve records when explicitly configured; the memory backend loses all records on restart.	Durable backends provide shared state but must be provisioned and monitored explicitly.	Keep metadata and replay records in a durable server-owned store for production use. Prefer PostgreSQL for long-term audit retention, Redis for lightweight deployments, and Milvus/Qdrant only when semantic replay search is explicitly needed.
Semantic cache entries	router runtime, src/semantic-router/pkg/cache/**	Default memory; optional Redis, Milvus, hybrid	ephemeral_request_state in local dev; shared cache in scaled deploys	Restart flushes cache; replicas do not share hot entries by default	Cold-start latency, inconsistent cache hit rates, and uneven behavior across replicas	Keep this as cache, not a database table. Prefer Redis or hybrid shared backends for scaled deployments; document memory backend as local/dev or single-node only.
RAG retrieval result cache	router runtime, src/semantic-router/pkg/extproc/req_filter_rag_cache.go	Process-wide singleton in-memory LRU with TTL	ephemeral_request_state	Restart flushes cache; cache is global per process, not per tenant or replica	Hidden shared mutable state, no observability, no durability, and no multi-replica coherence	Keep as optional cache only. Move to a pluggable shared cache backend if this becomes performance-critical, or document as local process optimization.
Agentic memory vectors	router runtime via routerruntime.Registry, src/semantic-router/pkg/memory/**	Disabled by default; vector content leans on Milvus config when enabled	shared_durable_workflow_state when enabled	Depends on backend choice; not enabled by default. InitWithRuntime now resolves startup memory state only from the shared runtime registry; nil-registry callers still use the process-wide fallback.	Product semantics remain ambiguous between experimental memory and supported user data	Keep vector embeddings in Milvus or another vector store, but pair them with explicit metadata and lifecycle ownership in a durable server-owned contract.
Vector store collection registry	router runtime, src/semantic-router/pkg/vectorstore/manager.go	Pluggable MetadataRegistry: in-memory (default) or Postgres	shared_durable_workflow_state when metadata_store: postgres; otherwise ephemeral	With metadata_store: memory, collection metadata disappears on restart even if backend collections remain. With postgres, metadata survives restarts. API servers with a runtime registry resolve the vector manager, embedder, and ingestion pipeline only from the registry once present; process-wide globals are limited to nil-registry fallback callers. ExtProc vectorstore RAG resolves request-time manager/embedder dependencies only from the router runtime registry.	Set vector_store.metadata_store: postgres for durable registry; the CLI local runtime provisions Postgres and fills metadata_postgres defaults for this mode. A warning is emitted when metadata_store is memory with a durable backend (Milvus/Valkey/Qdrant).	Postgres registry tables: vector_store_registry, file_registry. Loaded on startup via LoadFromRegistry.
Vector store file registry	router runtime, src/semantic-router/pkg/vectorstore/filestore.go	File bytes on local disk; file metadata backed by MetadataRegistry	Mixed restart_safe_local_state and shared_durable_workflow_state	With metadata_store: memory, files may remain on disk while metadata vanishes on restart. With postgres, file metadata is durable. API servers with a runtime registry resolve the file store only from the registry once present; process-wide globals are limited to nil-registry fallback callers.	Set vector_store.metadata_store: postgres for restart-safe file metadata. File bytes remain on local disk.	Shares the same MetadataRegistry as the collection registry.
Startup readiness and model download progress	router runtime, src/semantic-router/pkg/startupstatus/status.go, redis_writer.go, src/semantic-router/pkg/apiserver/route_startup_status.go	Default file; recommended redis through startup_status.store_backend for production. Router exposes GET /startup-status and readiness consumes the same state resolver.	audit_analytics_telemetry with shared_durable_workflow_state when Redis backend is configured	Status survives restart and is shared across replicas when using the Redis backend. The file backend emits a startup warning and is not visible to the dashboard in containerized deployments. The API server starts before model downloads so /startup-status is available during the entire boot sequence, and /ready now reads the same Redis/file startup state.	File backend is replica-local and path-dependent; Redis backend is shared across replicas. Dashboard reads from /startup-status API first, falling back to file path.	Keep Redis as the recommended backend for containerized and multi-replica deployments. The file backend remains as a local-dev fallback. Status resolution uses only the /startup-status API and the file path.
Router Learning adaptation and protection state	router runtime via global.router.learning.adaptation, global.router.learning.protection, src/semantic-router/pkg/extproc/router_learning_*.go, and routerruntime.LearningRuntime	In-process runtime state for the first implementation; outcomes update model-targeted experience through the runtime registry	ephemeral_request_state for single-replica Router Learning states	Protection state preserves conversation/session continuity inside one router process. Adaptation state records model experience, reliability, latency, cache, cost, and outcome evidence in process. Missing state fails open and is recorded in learning diagnostics.	Multi-replica deployments need sticky routing for session/conversation protection; non-sticky replicas split online state and model experience.	Keep request-time reads local. For production multi-replica learning, add a designed shared-state seam with strict timeout, fail-open, and local fallback semantics.
Legacy Elo, RL-driven, and GMTRouter selector state	legacy selector implementations under src/semantic-router/pkg/selection/**	Historical local JSON or in-memory selector stores	Legacy only	Public decision.algorithm.type: elo, rl_driven, and gmtrouter config is removed from the clean Router Learning API and rejected by config loading. Reusable model-choice concepts belong in Router Learning adaptation, model experience, outcomes, and offline recipe learning.	Keeping these as public selector-owned stores would split learning state across algorithms and replicas.	Do not add new product behavior to legacy selector-local stores. Migrate reusable rating, reward, and personalization concepts into typed adaptation experience or offline recipe-learning artifacts.
Tools database	router integrations, global.integrations.tools, config/tools_db.json	JSON file path by default	restart_safe_local_state	Survives only as workspace file	Multi-user editing, audit, and HA are weak	If dashboard editing becomes first-class, add a durable metadata store or projection for tools; keep JSON as import/export and local-dev source.
Dashboard auth users, roles, permissions, audit logs	dashboard backend auth, dashboard/backend/auth/store.go; Helm persistence in deploy/helm/semantic-router/templates/dashboard-pvc.yaml	SQLite at ./data/auth.db by default; Helm can mount dashboard-local state at /app/data/auth.db when dashboard.persistence.enabled=true	restart_safe_local_state	Restart-safe in one workspace or one mounted PVC; not shared across replicas. The Helm chart now fails template rendering when dashboard.replicaCount > 1 because the current store is not an HA store.	Adequate for local stacks and one dashboard replica with a persistent volume; weak for HA or multi-instance deployments	Keep SQLite for local dev and single-replica Helm deployments. Add a relational production storage seam before allowing multi-replica Dashboard auth.
Dashboard auth session token	dashboard backend auth handlers and SQLite store, dashboard/frontend/src/utils/authFetch.ts, dashboard/frontend/src/contexts/AuthContext.tsx, dashboard/frontend/src/contexts/authSession.ts; Helm env wiring in deploy/helm/semantic-router/templates/dashboard-deployment.yaml	Login/bootstrap return a JWT response, set an HttpOnly vsr_session cookie, and persist a server-side session id for newly issued tokens; frontend stores only a bounded local token for token-based transports; Helm persistence wires DASHBOARD_AUTH_DB_PATH into the mounted dashboard-local PVC	restart_safe_local_state in the current SQLite auth store; intended shared_durable_workflow_state for production multi-replica deployments	Malformed or oversized local tokens are dropped before they can be mirrored into headers, cookies, or query-token transports; backend bearer, cookie, and query token inputs are normalized and bounded before JWT parsing; newly issued token session ids are checked against the server-side session record; logout clears local state, calls POST /api/auth/logout, revokes the current server-side session when present, and expires the server cookie; auth-store startup prunes expired or revoked sessions older than the retention window; provider startup probes /api/auth/me with same-origin credentials so a valid HttpOnly cookie can recover the user without readable local token material; Helm production values now persist the SQLite file while blocking unsupported multi-replica dashboard settings	Production multi-replica session-store policy is not complete; token-based protected transports still use bounded localStorage; tokens without a session id remain accepted until session-only auth policy is complete	Continue moving toward server-owned session contracts for production deployments. Current browser storage is bounded and sanitized, server cookie issue/clear includes explicit expiry semantics, backend token intake is bounded, newly issued sessions can be revoked, old inactive session rows are pruned, reload can recover from the server cookie, and Helm can preserve the local session DB across pod restarts; local token storage and SQLite-only session storage still need retirement or tighter scoping.
Evaluation Plane runs, evidence, and reports	dashboard backend dashboard/backend/evaluationplane/**; Python worker src/vllm-sr/cli/evaluation/**	Atomic local run bundles plus content-addressed objects and private lifecycle policy/audit records under EVALUATION_DATA_DIR	restart_safe_local_state for one workspace; immutable evaluation evidence after finalization	Manifests, control events, normalized evidence, reports, lineage, checksums, server-derived ownership, retention, holds, bounded quotas, checkpointed lifecycle chain anchors, active audit records, and live-resource creation bindings survive restart; corrupt lifecycle state is quarantined or fails startup closed	The filesystem store, process semaphore, lifecycle coordinator, and quotas are local to one dashboard replica; compacted lifecycle event bodies are not a queryable local archive; multi-replica execution still needs a shared queue plus object/metadata storage	Keep the local bundle and lifecycle contract as the portable single-replica evidence format. Preserve owner/admin authorization, hold/reference safety, quota accounting, checkpoint verification, and dry-run/apply collection when adding object storage and a relational run index. A separately designed immutable archive sink is required before claiming long-term event-level audit retention.
Evaluation progress fanout and cancellation	dashboard backend dashboard/backend/evaluationplane/service_execution.go; SSE handler dashboard/backend/handlers/evaluation_plane_events.go	Durable control-events.jsonl with in-memory subscribers and process cancel functions	restart_safe_local_state for event history; ephemeral_request_state for live connections and cancel handles	Clients can replay durable control events with Last-Event-ID; active streams and cancel handles vanish on restart, and interrupted runs become failed	Multi-replica subscribers and cancellation are not coordinated	Keep connection registries local for the single-replica implementation. Add a shared queue/pub-sub and lease protocol together with shared run storage rather than sharing only the SSE layer.
ML pipeline jobs	dashboard backend ML pipeline, dashboard/backend/mlpipeline/runner.go, dashboard/backend/workflowstore/**, dashboard/backend/handlers/mlpipeline.go	SQLite workflow.sqlite (default ./data/workflow.sqlite) for jobs and typed progress rows; SSE client maps in handler	shared_durable_workflow_state for job rows; ephemeral_request_state for live SSE	Job status and progress events survive dashboard restart; in-flight subprocess work is still lost without a worker resume story	Same SQLite file must be mounted for multi-replica dashboards	Keep subprocess execution separate; optional future: external job runner.
OpenClaw container registry	dashboard backend OpenClaw, dashboard/backend/handlers/openclaw.go, CLI src/vllm-sr/cli/container_services.py	JSON file under OpenClaw data dir	restart_safe_local_state	Registry survives in one workspace, not shared across dashboards	Container control depends on local file convention and workspace mounts	Keep this as local-dev adapter for vllm-sr serve, but introduce a server-owned registry for multi-user dashboard control.
OpenClaw teams and workers	dashboard backend OpenClaw, dashboard/backend/handlers/openclaw.go, openclaw_teams.go, openclaw_workers.go, dashboard/backend/workflowstore/**	Rows in shared workflow.sqlite; one-time JSON import from containers.json / teams.json when tables are empty	shared_durable_workflow_state (restart-safe local SQLite)	Entities survive process restart when the DB file is preserved	HA requires shared DB backend (future seam)	Same as evaluation auth: SQLite for single-node; add relational seam for multi-replica.
OpenClaw rooms and chat messages	dashboard backend OpenClaw, dashboard/backend/handlers/openclaw_rooms.go, dashboard/backend/workflowstore/**	Room rows and append-only message rows in workflow.sqlite; one-time JSON import from rooms.json / room-messages/*.json; SSE/WS client maps in memory	shared_durable_workflow_state for persisted chat; ephemeral_request_state for live transport	Messages no longer rewrite whole JSON files; history survives restart with the DB	Very large rooms may need pagination APIs	Add list cursors / limits on message APIs as usage grows.
Dashboard chat history and queued tasks in Playground-style surfaces	dashboard frontend, dashboard/frontend/src/hooks/useConversationStorage.ts, dashboard/frontend/src/hooks/conversationStorage.ts, dashboard/frontend/src/hooks/usePlaygroundQueue.ts, dashboard/frontend/src/components/ChatComponent.tsx	Browser localStorage with local caps for saved conversations and queued playground tasks	Ambiguous today	Persists only in one browser; malformed conversation and queued-task records are dropped on restore, duplicate conversation IDs are normalized, and persistence is bounded before hydration and writes	No cross-device continuity, no audit, and no server recovery	Decide explicitly: either mark this demo-only and ephemeral, or move conversation and task queue state server-side if it is a supported product surface.
Config backups, generated runtime config, DSL snapshots	CLI and dashboard, src/vllm-sr/cli/commands/runtime_support.py, dashboard/backend/handlers/config_backups.go, dashboard/backend/handlers/runtime_config_sync.go, src/semantic-router/pkg/apiserver/kb_persistence.go	Files under .vllm-sr/, including generated runtime config and KB bootstrap markers	restart_safe_local_state and derived_projection_state	Survive in one workspace; generated KB bootstrap markers are treated as local restart state, corrupted marker YAML is ignored, marker rewrites use same-directory atomic replace, API startup with a runtime registry now reads only registry-published config instead of adopting config.Get() from another runtime, registry-backed classification refresh no longer republishes service config through config.Get(), and managed KB mutations update server-local/runtime-registry config without replacing the process-wide config cache	Files blur source-of-truth boundaries and are hard to audit in multi-user setups	Keep YAML and DSL as canonical intent. Add durable version/audit tables plus read-model projections for current models, signals, decisions, plugins, and DSL text. Do not make DB a second mutable primary writer yet.
Active deployed models, signals, decisions, plugins, DSL parse results	shared config contract, dashboard topology/config APIs, src/semantic-router/pkg/config/**, src/semantic-router/pkg/dsl/**	Derived from YAML/DSL at runtime; not stored as durable projection	Missing derived_projection_state	Recomputed ad hoc from files and live parse paths; runtime-registry-backed API startup, classification-service resolution, config refresh, and managed KB config mutation no longer fall back to or rewrite process-wide config before registry publication, ExtProc initial router construction, gRPC startup, and config-source watcher selection resolve from active router/runtime config before the process-wide fallback, empty-registry ExtProc construction parses the explicit config file instead of adopting a conflicting Kubernetes global, empty-registry ExtProc server config resolution also avoids adopting a conflicting process-wide config before runtime publication, registry-backed ExtProc file reload publishes through the runtime registry without rewriting config.Get(), and ExtProc modality/system-prompt request filters plus their shared request-time helper use router/runtime-owned config before process-wide fallback	Hard to query, audit, diff, or expose consistently across dashboard and future APIs	Add a persisted projection tied to deployed config version. Suggested projections: active config version, DSL snapshot, models, signals, decisions, plugins, and validation diagnostics.

Default Memory-Backed Surfaces To Treat As High Risk

• global.stores.semantic_cache.backend_type = memory
• global.stores.vector_store.backend_type = memory in dashboard defaults when enabled
• RAG cache_results in src/semantic-router/pkg/config/rag_plugin.go
• vector-store metadata in src/semantic-router/pkg/vectorstore/manager.go
• vector-store file registry in src/semantic-router/pkg/vectorstore/filestore.go
• model-selection learning state without configured storage in src/semantic-router/pkg/selection/{elo.go,rl_driven.go,gmtrouter.go}; rl_driven and gmtrouter decision-scoped storage_path now reaches runtime selector construction, and API servers with a runtime registry no longer use selector globals before runtime publication, but shared-store parity remains open

What Should Go To A Database First

• User accounts, roles, permissions, and audit logs
• Evaluation, ML pipeline, and model-research job metadata and typed progress events
• OpenClaw teams, workers, rooms, and room messages
• Router-visible metadata for vector stores and uploaded files
• Response API conversations and response metadata if they are exposed as supported product features
• Config version history, active config projection, deployed model/signal/decision/plugin projection, and DSL snapshots

What Should Prefer Shared Cache Or Specialized Storage Instead Of A Database

• Semantic cache entries: shared cache such as Redis or the existing hybrid cache path
• RAG retrieval cache: shared cache if retained at all
• Vector embeddings and memory embeddings: vector backends such as Milvus or Llama Stack
• Large binary artifacts: local file/object storage with durable metadata in a database

Execution Order

1. Publish and keep this inventory current.
2. Make router metadata and replay or response history restart-safe where the product already exposes those surfaces.
3. Move dashboard workflow state off in-memory maps and browser-only storage into server-owned durable records.
4. Add a persisted deployed-config projection so dashboard and future APIs stop reparsing YAML and DSL for every query path.
5. Add restart and recovery coverage in E2E for at least one router state surface and one dashboard workflow. (Response API restart-recovery E2E test added in e2e/testcases/response_api_restart_recovery.go, registered in Redis profile.)