Centralize read models and refactor admin web flows

Author: siriusctrl

README.md

@@ -115,6 +115,7 @@ pnpm coverage   # Coverage with CI-enforced thresholds
 |----------|-------------|
 | [Trading Model](docs/trading-model.md) | Current simulation semantics for spot markets, perp markets, funding, settlement, and liquidation |
 | [Architecture](docs/architecture.md) | System design, package responsibilities, worker model, persistence, timeline and SSE architecture |
+| [Refactor Roadmap](docs/refactor-roadmap.md) | Current simplification targets, read-model cleanup plan, worker cleanup plan, and future reconciler evolution |
 | [API Reference](docs/api-reference.md) | Current REST and SSE surfaces, timeline event types, admin endpoints, and runtime configuration |
 | [Admin Guide](docs/admin-guide.md) | Dashboard workflows, admin order placement, timelines, liquidation monitoring, and operator APIs |
 | [Trading Agent](docs/trading-agent.md) | How to build an autonomous trading agent against the current API and event model |

docs/admin-guide.md

@@ -35,7 +35,7 @@ The overview screen shows:
 - total balance, market value, unrealized PnL, and equity across users
 - per-user cards with balances, equity, and top holdings
 - market-level summary data across all tracked positions
-- equity trend charts backed by periodic admin snapshots
+- equity trend charts backed by the background equity snapshotter worker
 
 #### Agent detail
 
@@ -117,6 +117,7 @@ Important rules:
 ## Timeline Semantics
 
 Admin timelines use the same merged event builder as user timelines.
+The event record shape is shared through `@unimarket/core`, so the dashboard and API stay on one timeline contract.
 
 Current timeline event types:
 - `order`
@@ -177,7 +178,7 @@ For operators, that means the activity feed can now show:
 
 ## Operational Notes
 
-- The overview page records equity snapshots in the background. The chart becomes more useful over time.
+- `GET /api/admin/overview` is read-only. Equity snapshots are recorded by the background equity snapshotter worker.
 - Admin order placement does not bypass trading constraints or risk rules.
 - If you see liquidation events, always check the paired portfolio state and recent funding for context.
 - If a pending `reduceOnly` order disappears after liquidation, that is expected cleanup behavior.

docs/api-reference.md

@@ -35,6 +35,7 @@ Notes:
 - liquidation timeline entries are sourced from the dedicated `liquidations` audit table
 - the backing filled liquidation order is hidden from timeline results to avoid duplicate entries
 - Polymarket timeline items try to resolve human-readable market names and outcomes
+- the timeline record shape is shared with the dashboard through `@unimarket/core`
 
 ## Trading
 
@@ -172,6 +173,10 @@ The full operator workflow is documented in [Admin Guide](admin-guide.md). The m
 
 All admin endpoints require `Authorization: Bearer <ADMIN_API_KEY>`.
 
+Admin read-model notes:
+- `GET /api/admin/overview` is read-only and does not write equity snapshots as a side effect
+- `GET /api/admin/equity-history` is backed by the background equity snapshotter worker
+
 Admin order-placement notes:
 - `POST /api/admin/users/:id/orders` accepts the same payload shape as `POST /api/orders`
 - optional `accountId` must match the target user's default account
@@ -196,6 +201,7 @@ Relevant runtime settings:
 - `SETTLE_INTERVAL_MS`
 - `FUNDING_INTERVAL_MS`
 - `LIQUIDATION_INTERVAL_MS`
+- `EQUITY_SNAPSHOT_INTERVAL_MS`
 - `MAINTENANCE_MARGIN_RATIO`
 - `DEFAULT_TAKER_FEE_RATE`
 - `${MARKET}_TAKER_FEE_RATE`

docs/architecture.md

@@ -36,7 +36,7 @@
 │  └────────────────────────────────────────────────────────────────┘  │
 │                                                                      │
 │  Background workers: reconciler · settler · funding collector ·     │
-│  liquidator                                                         │
+│  liquidator · equity snapshotter                                    │
 └──────────────────────────────────────────────────────────────────────┘
 ```
 
@@ -70,12 +70,14 @@ unimarket/
 │   ├── api/
 │   │   └── src/
 │   │       ├── routes/        # HTTP entrypoints
-│   │       ├── services/      # Shared API orchestration (e.g. order placement)
+│   │       ├── services/      # Shared API orchestration + read models
 │   │       ├── db/            # Schema and SQLite setup
 │   │       ├── reconciler.ts  # Pending limit worker
 │   │       ├── settler.ts     # Resolution worker
 │   │       ├── funding-collector.ts
 │   │       ├── liquidator.ts
+│   │       ├── equity-snapshotter.ts
+│   │       ├── periodic-worker.ts
 │   │       ├── timeline.ts    # Unified audit timeline builder
 │   │       ├── events.ts      # SSE event bus + event types
 │   │       └── index.ts       # API bootstrap
@@ -131,6 +133,7 @@ It handles:
 - request validation
 - idempotency
 - shared order-placement and order-cancellation orchestration for routes and workers
+- shared portfolio and overview read-model builders
 - persistence
 - worker scheduling
 - SSE event emission
@@ -144,6 +147,7 @@ The `web` package is the operator dashboard.
 
 It is intentionally thin:
 - reads from REST endpoints
+- centralizes authenticated admin requests in a small API client layer
 - renders portfolio, market, and timeline state
 - writes through documented admin endpoints
 - does not reimplement trading logic in the browser
@@ -211,7 +215,7 @@ A few design choices matter here.
 
 ## Background Workers
 
-The server process starts four workers after database migration.
+The server process starts five workers after database migration.
 
 ### Reconciler
 
@@ -250,6 +254,15 @@ Why it exists:
 - keeps the risk model explicit and testable
 - surfaces liquidation as a first-class event instead of a hidden side effect
 
+### Equity Snapshotter
+
+Purpose:
+- record periodic account-equity snapshots for operator history charts
+
+Why it exists:
+- keeps `GET /api/admin/overview` read-only
+- makes snapshot cadence an explicit background policy instead of a dashboard side effect
+
 ## Timeline and Event Architecture
 
 The system exposes two audit surfaces.
@@ -267,6 +280,8 @@ Current timeline event types:
 
 The account timeline and admin timeline both use the same builder so operators and end users see consistent event semantics.
 
+The timeline record contract is shared through `@unimarket/core`, so the API and web dashboard do not maintain separate event-shape definitions.
+
 ### SSE
 
 SSE is the real-time feed.

docs/refactor-roadmap.md

@@ -0,0 +1,250 @@
+# Refactor Roadmap
+
+This document tracks the main simplification and cleanup opportunities in the codebase after the current order-lifecycle and admin-dashboard refactors.
+
+It is intentionally pragmatic:
+
+- focus on reducing duplicate orchestration,
+- keep market-agnostic behavior intact,
+- preserve auditability and permission boundaries,
+- avoid speculative abstractions that do not remove real complexity.
+
+## Current Direction
+
+The codebase has already made two important moves:
+
+- order placement now runs through shared services instead of separate user/admin execution paths
+- order cancellation now also runs through shared services instead of separate route/worker code
+
+The next simplifications should continue in the same direction:
+
+- keep routes thin
+- keep workers focused on scheduling and trigger decisions
+- keep read models and write paths centralized
+
+## Recommended Refactor Sequence
+
+### 1. Shared Portfolio Read Models
+
+Priority: high
+Status: completed
+
+Problem:
+
+- user portfolio and admin single-user portfolio both enrich positions, quotes, and pending orders
+- admin overview also contains a third read-model path that re-derives similar account and position summaries
+
+Current duplication exists across:
+
+- `packages/api/src/routes/account.ts`
+- `packages/api/src/routes/admin.ts`
+
+Why this matters:
+
+- portfolio fields can drift between user and admin views
+- quote handling and perp enrichment rules are being maintained in more than one place
+- small additions such as `maintenanceMargin`, `accumulatedFunding`, or future audit fields require repeated edits
+
+Implemented shape:
+
+- `packages/api/src/services/portfolio-read.ts`
+- `packages/api/src/services/admin-overview.ts`
+
+These builders now centralize:
+
+- quote fetching
+- perp state enrichment
+- funding aggregation
+- open-order shaping
+- account-level totals
+
+### 2. Move Snapshot Writes Out Of `GET /api/admin/overview`
+
+Priority: high
+Status: completed
+
+Problem:
+
+- the admin overview route performs asynchronous `equity_snapshots` writes after building the response
+
+Current location:
+
+- `packages/api/src/routes/admin.ts`
+
+Why this matters:
+
+- a read endpoint is performing background writes
+- failure handling is hidden in route-local `void (...)` fire-and-forget code
+- snapshot cadence policy is tied to a dashboard read path instead of an explicit worker or service
+
+Implemented shape:
+
+- `/api/admin/overview` is read-only
+- snapshot generation runs through `packages/api/src/equity-snapshotter.ts`
+- snapshot cadence is controlled by `EQUITY_SNAPSHOT_INTERVAL_MS`
+
+### 3. Shared Worker Scaffold
+
+Priority: medium
+Status: completed
+
+Problem:
+
+- reconciler, settler, funding collector, and liquidator all repeat the same interval/locking/logging structure
+
+Current locations:
+
+- `packages/api/src/reconciler.ts`
+- `packages/api/src/settler.ts`
+- `packages/api/src/funding-collector.ts`
+- `packages/api/src/liquidator.ts`
+
+Why this matters:
+
+- interval parsing, running guards, startup logging, and stop handlers are duplicated
+- worker ergonomics are inconsistent by file over time
+
+Implemented shape:
+
+- `packages/api/src/periodic-worker.ts`
+- reconciler, settler, funding collector, liquidator, and equity snapshotter all use the shared scaffold
+
+### 4. Shared Timeline Contract Types
+
+Priority: medium
+Status: completed
+
+Problem:
+
+- the web dashboard re-declares the timeline event contract instead of consuming a shared source
+
+Current duplication exists across:
+
+- `packages/api/src/timeline.ts`
+- `packages/api/src/events.ts`
+- `packages/web/src/lib/useAgentTimeline.ts`
+- `packages/web/src/components/ActivityFeed.tsx`
+
+Why this matters:
+
+- adding or renaming event fields requires touching both API and web manually
+- UI assumptions can drift from the actual backend timeline payload
+
+Implemented shape:
+
+- timeline event record types live in `@unimarket/core`
+- API timeline builders and dashboard timeline hooks consume the shared contract
+
+### 5. Break Up `TradePage`
+
+Priority: medium
+Status: completed
+
+Problem:
+
+- the admin trade console currently owns market loading, agent loading, quote refresh, search, portfolio fetch, order form state, and trader creation in one page component
+
+Current location:
+
+- `packages/web/src/pages/TradePage.tsx`
+
+Why this matters:
+
+- the file is large and mixes multiple concerns
+- repeated auth-failure handling is embedded in page-level effects
+- API response types are redeclared inline instead of being shared or centralized
+
+Implemented shape:
+
+- `packages/web/src/pages/TradePage.tsx` now acts as an orchestration page
+- trade-specific UI lives under `packages/web/src/components/trade/`
+- networking and response types no longer live inline inside the page file
+
+### 6. Shared Admin API Client Helpers
+
+Priority: medium
+Status: completed
+
+Problem:
+
+- admin web code still performs repeated `fetch + auth header + auth failure handling + response parsing` logic
+
+Current locations:
+
+- `packages/web/src/pages/TradePage.tsx`
+- other admin-facing web modules
+
+Implemented shape:
+
+- `packages/web/src/lib/admin-api.ts`
+- admin hooks and pages now share:
+  - auth header injection
+  - `401/403` logout handling
+  - JSON error extraction
+  - typed response parsing
+
+## Future Reconciler Evolution
+
+Priority: not short-term
+
+The current reconciler model is intentionally simple:
+
+- global scan of `pending` limit orders
+- one quote fetch per `market:symbol`
+- executable-side check using `buy -> ask`, `sell -> bid`
+- once crossed, the whole order fills
+
+This is the right short-term model for paper trading because it is easy to explain and easy to audit.
+
+### What may be added later
+
+If the platform later wants more realistic limit-order simulation, the reconciler may evolve toward:
+
+- depth-aware matching for markets with `orderbook`
+- price-time priority instead of simple traversal order
+- partial fills
+- explicit residual pending quantity
+
+### What should not happen
+
+The platform should not introduce a half-step model such as:
+
+- “sometimes liquidity matters”
+- but without orderbook depth
+- and without partial fills
+- and without price-time priority
+
+That would be more complex than the current model without being meaningfully more correct.
+
+### Recommended future boundary
+
+If this work is taken on later:
+
+- keep the current simple reconciler as the default path
+- enable depth-aware behavior only for markets that expose `orderbook`
+- preserve a clear, documented distinction between:
+  - trigger price
+  - execution price
+  - filled quantity
+  - remaining pending quantity
+
+This is intentionally a future path, not an immediate roadmap item.
+
+## What To Avoid
+
+These changes would likely add churn without removing enough complexity:
+
+- merging user and admin HTTP routes into one path
+- pushing permission logic down into core trading functions
+- adding market-specific branches outside adapters and capability checks
+- introducing a generic abstraction before there are at least two real call sites
+
+## Decision Rule
+
+Before taking on any simplification, ask:
+
+1. Does it remove an actual duplicate business rule or orchestration path?
+2. Does it preserve auditability and permission boundaries?
+3. Does it make the next feature cheaper without hiding current behavior?
+
+If the answer is not clearly yes, the refactor probably is not worth doing yet.

docs/trading-model.md

@@ -108,6 +108,12 @@ The reconciler runs in the background and:
 
 This keeps limit-order behavior deterministic without introducing a full matching engine.
 
+Future evolution:
+
+- the short-term contract remains whole-order fills once the executable price crosses
+- a future depth-aware model may be added for markets with `orderbook`
+- if that happens, it should use price-time priority and partial fills rather than a simplified FIFO approximation
+
 ## Trading Constraints
 
 Every market may expose symbol-level constraints through `getTradingConstraints(symbol)`.