Merge pull request #20 from pankaj4u4m/spec

Spec

Author: pankaj4u4m

GEMINI.md

@@ -3,7 +3,7 @@
 ## General Agent Guidelines
 - **No Shortcuts:** Never assume the structure of data or APIs. Always verify the source of truth (e.g., `api.ts` for frontend, `schemas.py` or `models.py` for backend) before implementation.
 - **Strict Verification:** Always run the project's build, lint, and type-check commands after making changes. A task is not complete until it passes all validation steps.
-- **Atelier Integration:** Use `mcp_atelier_trace` after every significant fix to record the outcome and help the system learn.
+- **Atelier Integration:** Use `mcp_atelier_record` after every significant fix to record the outcome and help the system learn.
 
 ## Frontend Engineering Standards (TypeScript/React)
 

Makefile

@@ -75,7 +75,6 @@ typecheck: ## Run mypy strict type-checking
 	uv run mypy --strict $(PY_PATHS)
 
 verify: lint format-check typecheck test ## Verify code, runtime smoke tests, and agent integrations
-	bash scripts/verify_atelier_mcp_stdio.sh
 	bash scripts/verify_atelier_service.sh
 	bash scripts/verify_atelier_postgres.sh
 	bash scripts/verify_agent_clis.sh

QUICK_REFERENCE.md

@@ -12,7 +12,7 @@ _For agents/automation, see [AGENT_README.md](AGENT_README.md) instead._
 
 - **Starting a coding task?** → Use `/atelier:context` or pick `atelier:code` agent
 - **Same error failed 2+ times?** → Use `/atelier:rescue`
-- **Done with the task?** → Use `/atelier:trace`
+- **Done with the task?** → Use `/atelier:record`
 - **Just reading code?** → Use `atelier:explore` agent (read-only)
 - **Reviewing someone's PR?** → Use `atelier:review` agent (no editing)
 
@@ -27,7 +27,7 @@ Core Loop
 ─────────────────────────────────────────────────
 /atelier:context             Gather task context and procedures
 /atelier:implement           Execute task (optional: /atelier:rescue on failure)
-/atelier:trace               Record outcome (files, commands, errors, results)
+/atelier:record              Record outcome (files, commands, errors, results)
 ```
 
 Intelligence
@@ -50,7 +50,7 @@ Operations
 
 - `/atelier:context` — **Step 1: Gather Context**. Retrieve procedures, facts, and rules before starting.
 - **Step 2: Implement** — Execute the task. Use `/atelier:rescue` if stuck on the same error twice.
-- `/atelier:trace` — **Step 3: Record Trace**. Save what happened for the reasoning store once done.
+- `/atelier:record` — **Step 3: Record**. Save what happened for the reasoning store once done.
 
 ### Intelligence Skills (as needed)
 
@@ -73,7 +73,7 @@ Operations
 
 ```
 atelier:code       Main coding agent
-├─ Loop: context → plan → implement → rescue → verify → trace
+├─ Loop: context → plan → implement → rescue → verify → record
 ├─ Tools: All (editing + MCP + shell)
 └─ Hard rules: No secrets, no plan contradictions
 
@@ -98,10 +98,10 @@ atelier:repair     Repair specialist (on repeated failures)
 ```
 CORE WORKFLOW
 ─────────────────────────────────────────────────
-task             Fetch ReasonBlocks, memory, ledger, and environment context
+context           Fetch ReasonBlocks, memory, ledger, and environment context
 route              Dispatch route decide/verify operations
 rescue             Get recovery procedure
-trace       Save observable outcome for learning
+record      Save observable outcome for learning
 verify             Verify high-risk domain before success
 
 CONTEXT + MEMORY                                           [Atelier augmentation]
@@ -163,7 +163,7 @@ ID  Title                              Domain
 ```
 atelier context --task "..." --domain beseam.shopify.publish --files src/...
 atelier rescue --task "..." --error "..." --files ... --recent-actions "tried X" "tried Y"
-atelier trace record --agent atelier:code --domain ... --status success --files-touched [...]
+atelier runs record --agent atelier:code --domain ... --status success --files-touched [...]
 atelier verify rubric_shopify_publish --checks '{"check_1": true, ...}'
 atelier block list --domain beseam.shopify.publish
 atelier eval list
@@ -253,7 +253,7 @@ DO NOT:
 ❌ Ignore high-severity Atelier warnings
 ❌ Invent plan steps that contradict ReasonBlocks
 ❌ Store secrets, API keys, tokens in traces
-❌ Call record_trace without observable facts
+❌ Call record without observable facts
 ❌ Edit code after review agent reports "block"
 ❌ Approve a rubric_gate "block" verdict
 ❌ Re-propose same hypothesis twice in repair
@@ -270,7 +270,7 @@ Q: Same error just failed twice
 A: Use /atelier:rescue
 
 Q: I'm done with the task
-A: Use /atelier:trace
+A: Use /atelier:record
 
 Q: I need to review someone's patch
 A: Use atelier:review agent

README.md

@@ -48,23 +48,21 @@ Check the installed runtime:
 ```bash
 atelier --version
 atelier-mcp --version
-atelier servicectl status
-atelier stack status
+atelier background status
 ```
 
 → User install guide: [docs/installation.md](docs/installation.md)
 
 ## What Runs After Install
 
-The installed product always gives you **CLI + MCP + background processing**.
+The installed product gives you **CLI + MCP + Background Services**.
 
-If Docker is available and `ATELIER_NO_STACK=1` is not set, the installer also
-tries to start the optional visualization stack on ports `3125` and `8787`.
+Atelier automatically registers itself with your OS background manager (**systemd** on Linux, **launchd** on macOS). This ensures it starts on boot and restarts automatically if it crashes.
 
 - `atelier ...` works with no HTTP server.
 - `atelier-mcp` works with no HTTP server.
-- `atelier servicectl ...` manages offline/background work.
-- `atelier stack ...` remains optional at runtime even if the installer started it for convenience.
+- `atelier background ...` manages the OS-level services (controller + stack).
+- `atelier stack ...` manages the visualization containers manually.
 
 Pure CLI mode still emits Atelier telemetry events unless you disable it with `atelier telemetry off` or `ATELIER_TELEMETRY=0`.
 
@@ -74,67 +72,72 @@ Atelier follows a simple **3-step process** for reliable coding:
 
 1. **Context**: Retrieve procedures and facts before starting.
 2. **Implement**: Execute the task (with optional rescue/route).
-3. **Trace**: Record the outcome once done.
+3. **Record**: Record the outcome once done.
 
 Passive tracking works automatically. Active context features require `ATELIER_DEV_MODE=1`.
 
 ```bash
 # Fetch context for an agent task (Requires Dev Mode)
-atelier context \
-  --task "Fix generated output that drifts back after refresh" \
-  --domain source.truth \
-  --file src/content/generate.py
+atelier tools call context --dev --args '{
+  "task": "Fix generated output that drifts back after refresh",
+  "domain": "source.truth",
+  "files": ["src/content/generate.py"]
+}' --json
 
 # Verify required checks after a task completes (Requires Dev Mode)
-echo '{"canonical_identifier_used": true, "pre_change_state_captured": true, "read_after_write_completed": true}' \
-  | atelier verify rubric_state_change_safety
+atelier tools call verify --dev --args '{
+  "rubric_id": "rubric_state_change_safety",
+  "checks": {
+    "canonical_identifier_used": true,
+    "pre_change_state_captured": true,
+    "read_after_write_completed": true
+  }
+}' --json
 ```
 
 Common runtime commands:
 
 ```bash
-atelier servicectl status
+atelier background status
 atelier worker list
-atelier trace list
-atelier search "read after write verification"
+atelier runs list
+atelier tools call search --dev --args '{"query":"read after write verification"}' --json
 ```
 
 → Installed quickstart: [docs/quickstart.md](docs/quickstart.md)
 
 ## Optional UI Stack
 
-The UI is optional. If the installer already started it for you, you can leave it
-running, inspect it, or stop it. If Docker was unavailable during install, start
-it manually when you want visualization or a browser-based view of the runtime.
+The UI is optional. The installer registers it as a background service by default if Docker is available. You can manage it alongside the controller:
 
 ```bash
-atelier stack start
-atelier stack status
-atelier stack stop
+# View logs for the visualization stack
+atelier background logs stack
+
+# Restart the entire environment
+atelier background restart
 ```
 
 Then open:
 
 - frontend: [http://localhost:3125](http://localhost:3125)
 - service API: [http://localhost:8787](http://localhost:8787)
 
-Useful stack commands:
-
-```bash
-atelier stack status
-atelier stack logs
-atelier stack stop
-```
+## Background Processing & Auto-Update
 
-## Background Processing
+Atelier includes an offline processing controller that runs as a background service. It handles session imports, maintenance jobs, and **automatic code updates**.
 
-`servicectl` is the installed offline processing controller. It runs detached and periodically enqueues and processes maintenance work. It works on the default SQLite install and on Postgres-backed deployments.
+The controller periodically checks your git repository for updates. When found, it automatically pulls the latest code, syncs dependencies, and restarts the services.
 
 ```bash
-atelier servicectl status
-atelier servicectl logs
-atelier servicectl stop
-atelier servicectl start
+# Check service health and auto-update status
+atelier background status
+
+# View background logs (from journald/launchd)
+atelier background logs controller
+
+# Manually trigger a restart
+atelier background restart
 ```
 
 You can also queue and process work manually:

bin/atelier-status

@@ -155,14 +155,37 @@ run_files = sorted(
 )
 
 # ── Extract Live Savings and DB Cost ──────────────────────────────────────
-savings_map = {}
+# savings_map: total cost_saved_usd per session (all levers combined)
+# routing_map: model_routing savings per session
+# compaction_map: session_compaction savings per session
+savings_map    = {}
+routing_map    = {}
+compaction_map = {}
+routing_total    = 0.0
+compaction_total = 0.0
+
+def _lever_bucket(lever: str) -> str:
+    l = (lever or "").lower()
+    if "routing" in l:   return "routing"
+    if "compact" in l:   return "compaction"
+    return "other"
+
 try:
     with open(os.path.join(ROOT, "live_savings_events.jsonl")) as f:
         for line in f:
             d = json.loads(line)
             rid = d.get("session_id")
+            cost = float(d.get("cost_saved_usd", 0.0) or 0.0)
+            lever = str(d.get("lever") or d.get("tool_name") or "")
+            bucket = _lever_bucket(lever)
             if rid:
-                savings_map[rid] = savings_map.get(rid, 0.0) + float(d.get("cost_saved_usd", 0.0))
+                savings_map[rid] = savings_map.get(rid, 0.0) + cost
+                if bucket == "routing":
+                    routing_map[rid] = routing_map.get(rid, 0.0) + cost
+                    routing_total += cost
+                elif bucket == "compaction":
+                    compaction_map[rid] = compaction_map.get(rid, 0.0) + cost
+                    compaction_total += cost
 except Exception:
     pass
 
@@ -258,16 +281,26 @@ if LINE_MODE:
     if cost_v == 0 and "input_tokens" in snap:
          # Heuristic for imported traces
          cost_v = (snap.get("input_tokens",0)*3 + snap.get("output_tokens",0)*15) / 1000000.0
-    
-    saved_v  = savings_map.get(session_id, 0.0)
+
+    saved_v      = savings_map.get(session_id, 0.0)
+    routing_v    = routing_map.get(session_id, 0.0)
+    compaction_v = compaction_map.get(session_id, 0.0)
+
+    saved_seg = ""
+    if saved_v > 0:
+        line_breakdown = []
+        if compaction_v > 0: line_breakdown.append(f"compact={usd(compaction_v)}")
+        if routing_v    > 0: line_breakdown.append(f"routing={usd(routing_v)}")
+        breakdown_suffix = f" ({', '.join(line_breakdown)})" if line_breakdown else ""
+        saved_seg = f" {SEP} {GREEN}saved={usd(saved_v)}{breakdown_suffix}{RESET}"
 
     line = (
         f"{BADGE} {BRAND}run {session_id[:8]}{RESET} {SEP} {DIM}{agent}{RESET} {SEP} "
         f"{domain} {SEP} {task} {SEP} {status_color(status)} "
         f"{SEP} ev={events} err={errors} blk={blockers}"
         f" {SEP} files={files_n} tools={tools_n}"
         + (f" {SEP} cost={usd(cost_v)}" if cost_v > 0 else "")
-        + (f" {SEP} {GREEN}saved={usd(saved_v)}{RESET}" if saved_v > 0 else "")
+        + saved_seg
         + (f" {SEP} {dur_str}" if dur_str else "")
         + f" {SEP} {DIM}{age_str}{RESET}"
     )
@@ -364,9 +397,13 @@ box_line(
     f"{DIM}tools={k(total_tools)}  files={total_files}  errs={total_errors}{RESET}"
 )
 if total_cost > 0 or saved_usd > 0:
+    breakdown_parts = []
+    if compaction_total > 0: breakdown_parts.append(f"compact {usd(compaction_total)}")
+    if routing_total    > 0: breakdown_parts.append(f"routing {usd(routing_total)}")
+    breakdown_str = f"  {DIM}({' · '.join(breakdown_parts)}){RESET}" if breakdown_parts else ""
     box_line(
         f"{DIM}cost{RESET} {usd(total_cost)}  "
-        + (f"{GREEN}saved{RESET} {usd(saved_usd)}" if saved_usd > 0 else "")
+        + (f"{GREEN}saved{RESET} {usd(saved_usd)}{breakdown_str}" if saved_usd > 0 else "")
         + (f"  {DIM}tokens{RESET} {k(total_tokens)}" if total_tokens else "")
     )
 
@@ -391,17 +428,26 @@ for d in all_run_entries[:N_RUNS]:
     if cost_v == 0 and "input_tokens" in d:
          cost_v = (d.get("input_tokens",0)*3 + d.get("output_tokens",0)*15) / 1000000.0
 
-    saved_v = savings_map.get(session_id, 0.0)
+    saved_v      = savings_map.get(session_id, 0.0)
+    routing_v    = routing_map.get(session_id, 0.0)
+    compaction_v = compaction_map.get(session_id, 0.0)
 
     # Professional 2-line layout
     # [icon] Task ................. ID
     dots = "." * max(1, (W - len(re.sub(r'\033\[[^m]*m', '', task)) - 16))
     box_line(f" {status_icon(status)}  {BOLD}{task}{RESET} {DIM}{dots} {session_id[:8]}{RESET}")
-    
+
     metrics = []
-    if cost_v > 0:  metrics.append(f"cost={usd(cost_v)}")
-    if saved_v > 0: metrics.append(f"{GREEN}saved={usd(saved_v)}{RESET}")
-    if dur_str:     metrics.append(dur_str)
+    if cost_v > 0:
+        metrics.append(f"cost={usd(cost_v)}")
+    if saved_v > 0:
+        run_breakdown = []
+        if compaction_v > 0: run_breakdown.append(f"c={usd(compaction_v)}")
+        if routing_v    > 0: run_breakdown.append(f"r={usd(routing_v)}")
+        run_breakdown_str = f" {DIM}({' '.join(run_breakdown)}){RESET}" if run_breakdown else ""
+        metrics.append(f"{GREEN}saved={usd(saved_v)}{run_breakdown_str}")
+    if dur_str:
+        metrics.append(dur_str)
     metrics_str = f" {SEP} ".join(metrics)
     
     meta_line = f"    {DIM}{age_str}{RESET} {SEP} {agent} {SEP} {domain}"

docs-archive/internal/engineering/telemetry-implementation-plan.md

@@ -76,7 +76,7 @@ src/atelier/gateway/adapters/
   cli.py                   # ADD: SIGINT/SIGTERM handlers, command lifecycle events
   mcp_server.py            # ADD: session start/end, tool_called wrapper events
   http_api.py              # ADD: request middleware emitting api_request events
-  wrappers.py              # ADD: lifecycle hooks for atelier-task / atelier-context / etc.
+  wrappers.py              # ADD: lifecycle hooks for atelier-context / etc.
 
 src/atelier/core/runtime/
   engine.py                # ADD: lift loop_detection / monitor signals to product events

docs-archive/sdk/cli.md

@@ -25,7 +25,7 @@ Developer-mode commands require `ATELIER_DEV_MODE=1`:
 - `atelier edit`
 - `atelier route`
 
-Older docs may refer to `atelier lint`, `atelier check-plan`, `atelier task`, or
+Older docs may refer to `atelier lint`, `atelier check-plan`, `atelier context`, or
 `atelier pack`. Those are not the current public CLI surface.
 
 ## Common Commands