Genesis: Versioned Build Plan — Safety-Ordered, Waterfall

Status: V3 Feature Complete | Last updated: 2026-04-12

V3 is feature complete. All nine build phases (0-9) are done. The phase-ordered waterfall served its purpose — safety-first scaffolding for a system that now runs autonomously. Future work is scoped as V4 or V5, not as additional V3 phases. This document is now historical reference for the build sequence and design rationale.

Ordering principle: Build what's safest, most testable, and least likely to break first. Each phase depends on the previous phases working correctly. No phase should be started until its dependencies are verified.

Versioning principle: Each version is a complete, independently valuable system — not a half-built one. Later versions benefit from operational data generated by earlier ones. Features that fundamentally require operational data to avoid producing garbage belong in the version that will have that data.

Master design reference: genesis-v3-autonomous-behavior-design.md remains the full architectural reference. This document says WHAT to build, WHEN, and in WHICH VERSION. The master doc says WHY and HOW each component works.

Runtime architecture: docs/plans/2026-03-07-agentic-runtime-design.md defines the dual-runtime model: Claude Code as intelligence layer, Agent Zero as infrastructure. Phases 4+ (Perception onward) shift reflection engine ownership to CC background sessions. Task execution phases (8-9) use CC as orchestrator, not AZ's main agent.

Identity reference: genesis-v3-vision.md defines who Genesis is and what it aspires to be. Every implementation decision should be consistent with that document.

Version Overview

V3: The Complete Copilot          V4: The Self-Tuning Copilot       V5: The Autonomous Copilot
─────────────────────────         ──────────────────────────────    ──────────────────────────────
Perceives, remembers,             Same copilot, measurably          Proposes changes to itself,
learns, communicates              better. Meta-prompting,           anticipates needs, earns
(basic), has fixed                calibration loops, richer         higher autonomy, learns
boundaries. Ships with            outreach. Built on V3 data.       how to learn. Needs months
conservative defaults.            Features activate as data         of V4 data.
                                  accumulates (feature flags).

Knowledge Base: Parallel track — architecturally independent, can ship alongside any version.

Version	Capability Plateau	Data Requirement
V3	Complete working copilot	None — ships with conservative fixed defaults
V4	Self-tuning copilot	~1-2 months of V3 operational data
V5	Autonomous copilot	~3-6 months of V4 operational data

Why three versions? They align with three genuine capability plateaus. More versions create artificial boundaries. Fewer create a dumping ground where 2-week-data features mix with 6-month-data features.

V3: The Complete Copilot

10 phases, ordered by safety. Each phase depends on its predecessors. V3 is a complete, shippable system with conservative fixed defaults.

V3 Phase Dependency Map

Phase 0: Data Foundation ✅
    ├── Phase 1: Awareness Loop ✅ ────────┐
    ├── Phase 2: Compute Routing ✅ ───────┤
    │                                      │
    │   Phase 3: Surplus Infrastructure ✅ ─┤  (needs Phase 0 + 2)
    │                                      │
    │                    Phase 4: Perception ✅
    │                         │
    │                    Phase 5: Memory Operations ✅
    │                         │
    │                    Phase 6: Learning Fundamentals ✅
    │                         │
    │                    Phase 7: Simple Deep Reflection ✅
    │                         │
    │                    Phase 8: Basic Outreach ✅
    │                         │
    │                    Phase 9: Basic Autonomy ✅ (needs Phase 6 + 8)

Parallel tracks: Phases 1, 2, and 3 can be built in parallel (all depend only on Phase 0). Phase 4 can start once Phases 1 and 2 are done. The critical sequential path is Phases 4→5→6→7→8→9.

V3 Phase 0: Data Foundation ✅ COMPLETE

Risk: LOW — Pure schemas and CRUD. No LLM calls. No user-facing behavior.

Status: COMPLETE (2026-03-02). 13 tables, 13 CRUD modules, Qdrant wrapper (1024-dim), 4 MCP server stubs (24 tools), 170 tests. Committed on main.

What to Build

Component	Schema/Table	Design Doc Reference
Memory storage	Episodic, semantic, procedural memory tables	§4 MCP Servers → memory-mcp
Observation storage	Observations with utility tracking (`retrieved_count`, `influenced_action`)	§Loop Taxonomy → Tier 4
Execution traces	Trace table per Execution Trace Schema. Include `initiated_by` (user / awareness_loop / surplus / reflection) and `request_delivery_delta` (structured: original_request, discoveries, adjusted_scope, scope_communicated, delta classification, discovery attribution). See §Self-Learning Loop step 3.	§Task Execution Architecture
Surplus staging	`surplus_insights` (content, source_type, model, drive_alignment, confidence, TTL, promoted_to)	§Cognitive Surplus
Signal weights	Signal source weights with adaptation bounds	§Awareness Loop → Signal-Weighted Trigger
Capability gaps	Gap tracking (description, frequency, first/last seen, feasibility, revisit_after)	§Open Question #10
Procedural memory	Procedures with confidence, invocation_count, success_rate, version	§Procedural Memory Design
User model cache	Structured user model JSON store	§Open Question #2
Speculative claims	Hypothesis table (claim, speculative flag, evidence_count, expiry)	§Open Question #13
Autonomy state	Per-category autonomy level tracking	§Autonomy Hierarchy
Outreach history	Outreach log with engagement tracking fields	§Proactive Outreach
Daily brainstorm log	Brainstorm outputs (upgrade-user, upgrade-self) with promotion status	§Cognitive Surplus

Workspace Files

Initialize workspace files alongside database schemas:

Cognitive state summary (replaces JOURNAL.md) — Fixed-size regenerated summary (~600 tokens) of active context + pending actions. Stored in cognitive_state DB table, not a file. Written by: Deep reflection (Phase 7). Read by: every fresh 20-30B+ context. Narrative continuity comes from this + memory-mcp episodic queries, not from an append-only journal. See §Cognitive State Summary in design doc.

All schemas include GROUNDWORK fields for V4/V5 features. Build the full schema now — some fields stay empty until later versions populate them.

MCP Server Interfaces

Define tool interfaces for all 4 servers. Implementation can be stubs initially.

memory-mcp: store, retrieve (hybrid: embedding + activation), link, update_activation, list_by_type
recon-mcp: store_finding, query_findings, schedule_job, list_scheduled
- Build note: All HTTP fetches in recon-mcp should include the Accept: text/markdown header. Cloudflare-enabled sites (~20%+ of the web) will return clean Markdown instead of HTML, yielding ~80% token reduction. The x-markdown-tokens response header allows pre-checking content size before fetching the full body. Non-Cloudflare sites ignore the header and return HTML normally. Zero implementation cost — one header on every outbound request.
- Implementation candidate — Scrapling: Evaluate D4Vinci/Scrapling for recon-mcp's web monitoring/scraping. Key differentiators: adaptive element tracking that survives site redesigns (intelligent similarity-based relocation), TLS fingerprint impersonation, Cloudflare bypass via StealthyFetcher, and a Scrapy-like spider framework with pause/resume checkpoints. Offers three fetcher tiers (Fetcher for fast HTTP, StealthyFetcher for anti-detection, DynamicFetcher for full Playwright browser). Evaluate at implementation time against Playwright-only approach — Scrapling may be better for non-interactive scraping while Playwright remains necessary for interactive browser tasks.
health-mcp: report_metric, query_health, get_error_rates, list_alerts
outreach-mcp: queue_message, get_pending, record_engagement, get_channel_stats, list_channels

Verification

Every table: create, read, update, delete
Schema supports every query pattern in the design doc
MCP tool interfaces accept and return expected types
Foreign key relationships are correct

V3 Phase 1: The Metronome (Awareness Loop) ✅ COMPLETE

Risk: LOW — Purely programmatic. No LLM, no user-facing output. Dependencies: Phase 0.

Status: COMPLETE (2026-03-03). Signal collector, composite scorer, depth classifier, critical-bypass, 5-minute tick scheduler with APScheduler. 117 tests (287 cumulative). Committed on main.

What to Build

5-minute tick scheduler with hybrid event-driven + calendar guardrails
Signal collector: inbox, health-mcp, monitor, recon-mcp, calendar events
Composite urgency score calculator (§Awareness Loop → Signal-Weighted Trigger)
Depth classifier: score → Micro / Light / Deep
Critical event bypass: urgent signals skip tick
Depth Escalation Protocol: Awareness Loop stays sole coordinator; Reflection Engine sets escalation flags; critical override bypasses tick
Fixed signal weights from the design doc (NOT adaptive — adaptation is V4)
Three-category scheduling: Awareness Loop coordinates all three categories of scheduled work (§Three Categories of Scheduled Work in design doc):
- Event-driven reflection: adaptive, signal-triggered, calendar floors as safety nets
- Genesis's own rhythms: internal cadence timers (morning report, calibration cycles)
- User-scheduled crons: recon-mcp + future cron infrastructure
- Categories are distinct — different governing principles, same coordinator

Verification

Known signal + known weight → expected composite urgency score
Score thresholds correctly classify to right depth
Critical events bypass the tick
Calendar guardrails enforce min/max intervals

V3 Phase 2: Compute Routing ✅ COMPLETE

Risk: LOW — Infrastructure plumbing. No judgment calls. Dependencies: Phase 0.

Status: COMPLETE (2026-03-04). Full genesis.routing package: types, config loader, circuit breaker (state machine + registry), retry with backoff, cost tracker, dead-letter queue, degradation tracker (L0-L5), router with fallback chains. YAML config at config/model_routing.yaml (13 providers, 23 call sites). 93 routing tests (380 cumulative). Committed on main.

What AZ Already Provides

Agent Zero has provider abstraction via LiteLLM (unified_call() in models.py) and a RateLimiter (token bucket) per model. Genesis does NOT rebuild these — it layers on top:

AZ provides	Genesis adds
`unified_call()` — single LLM call path	Pre-call routing (fallback chain selection)
`RateLimiter` per model	Circuit breakers per provider (health-mcp)
LiteLLM's per-call cost tracking	Cost aggregation in SQLite (budget enforcement)
Two model slots (chat/utility)	28-call-site routing registry with fallback chains

What to Build

Genesis wraps the call sites that invoke unified_call(), adding routing and tracking. It does NOT replace unified_call() itself.

Call-site routing layer: Wraps each Genesis LLM call site with primary → fallback chain logic from the model routing registry. On failure, try next model in chain. On success, record which model handled it.
Model availability detection: health-mcp polls provider endpoints (local GPU, free APIs, paid APIs). This is Genesis-built — AZ has no equivalent.
Automatic fallback: provider unreachable → next in fallback chain
Re-routing: provider recovers → eligible for routing again
Model tracking: which model handled each call (cost/quality analysis)
Cost accounting: per-call cost (from LiteLLM), aggregated in SQLite cost_events table, enforced against budgets table
Model routing registry: defines all 28 call sites with primary models, fallback chains, free compute sources, paid alternatives, and output validation contracts (see docs/architecture/genesis-v3-model-routing-registry.md)

Compute Hierarchy

3B local (CPU)     → Embeddings, light extraction ONLY. No surplus, no reasoning.
20-30B local (GPU) → Micro/Light reflection, extraction, surplus tasks. PRIMARY workhorse.
Free APIs          → DEFAULT FALLBACK: Mistral, Groq, Gemini (privacy-ordered).
Sonnet-class       → Deep reflection, judgment calls, quality gates.
Opus-class         → Strategic reflection, identity proposals (V5).

3B Model Constraints

CPU only, must stay responsive for embeddings/extractions. CANNOT do: reflection, classification, surplus tasks, meta-prompting, or anything requiring reasoning/judgment.

Verification

See genesis-v3-resilience-patterns.md for tactical resilience design (backoff parameters, circuit breaker thresholds, degradation levels, dead-letter staging).

See genesis-v3-resilience-architecture.md for system-level resilience: composite state machine (cloud/memory/embedding/CC axes), deferred work queue with staleness policies, embedding backlog recovery, CC budget management, recovery orchestration, and out-of-band status file. Implemented in src/genesis/resilience/ (7 modules).

V3 Phase 3: Surplus Infrastructure ✅ COMPLETE

Risk: LOW-MODERATE — Scheduling and staging. Moved early to leverage free compute ASAP. Dependencies: Phase 0, Phase 2.

Status (2026-03-13): Surplus queue, staging area, deferred work infrastructure, and scheduler all implemented and tested.

What to Build

Surplus task queue with priority model:
- Priority considers: drive weights (fixed), recency of last audit, user activity patterns
- Task types: self-improvement, user-value-ideation, system-optimization
Idle cycle detection: when no user interaction is active and compute is available
Compute preference (not enforcement):
- Surplus exists to capture free/cheap compute that would otherwise go to waste. Its routing chains default to free models first (local 30B, free-tier APIs).
- This is a design preference, not a hard constraint. Genesis can override if it makes a case to the user, but that's an edge case — if a task genuinely needs paid compute, it probably belongs in regular operation, not surplus. Promote it.
- No never_pays walls, no ComputeTier.NEVER rejection. Just chains ordered with free models first.
Surplus output staging area: ALL outputs go to staging, never directly to production
- Outputs await promotion by next reflection review or user approval
Local machine uptime tracking (config-based schedule, not learned — learning is V4)
Daily brainstorming sessions (mandatory, at least 2/day):
1. "How can I upgrade the user?" — structured Light reflection on recent interactions, user model, and pending tasks. What opportunities is the user missing? What could be done better? What knowledge gap could be filled? → staging area
2. "How can I upgrade myself?" — structured Light reflection on recent system performance, procedure effectiveness, observation utility. What's working? What's not? What should I try differently? → staging area
- In V3, these use static prompt templates (Light depth, 20-30B or Gemini free). V4 upgrades them to meta-prompted sessions for higher quality.
- Outputs go to staging. Promoted by next Deep reflection review or user review.
- Each session writes a brainstorm observation to memory-mcp (outcomes + key ideas)
- Run on free compute. If free compute is unavailable, these are the LAST tasks to skip.

Why This Phase Is Early (Phase 3, Not Phase 6)

The surplus infrastructure is just a table, a queue, and a scheduler — trivially safe. Moving it early means that from the moment Perception (Phase 4) exists, surplus can immediately run extra micro/light reflections during idle cycles on free compute. As more phases come online, surplus tasks get more sophisticated:

Phase 4+: Surplus micro/light reflections on idle signals
Phase 6+: Procedure auditing, memory scanning
V4: Meta-prompted creative brainstorming, ongoing background dialogue

The daily brainstorming sessions run from day 1 of V3 with static prompts. They're simple Light reflections — low risk, high learning value. The sophisticated meta-prompted versions come in V4, but the infrastructure and habit are established now.

Verification

Surplus tasks only execute on free/cheap compute
Cost-frequency rule enforced (free=always, threshold=never)
Staging area stores without promoting to production
Daily brainstorm sessions fire reliably (exactly 2/day minimum)
Brainstorm sessions write to brainstorm_log (memory-mcp integration in Phase 5)
Idle detection identifies available compute windows

V3 Phase 4: Perception (Micro/Light Reflection)

Risk: MODERATE — First LLM calls, but low stakes. Dependencies: Phase 1, Phase 2.

Known Gap: Signal Collectors (as of 2026-03-17)

8 of 11 awareness signal collectors are stubs returning hardcoded 0.0. This means the awareness loop runs but triage receives near-zero signal data, causing it to almost never escalate to micro/light/deep reflection. The system looks "idle" when it's actually blind.

Stub collectors (return 0.0): ConversationCollector, TaskQualityCollector, OutreachEngagementCollector, ReconFindingsCollector, BudgetCollector, StrategicTimerCollector. (MemoryBacklogCollector removed 2026-04-11 — the retrieval-coverage metric was being misread as reflection urgency by the Deep depth scorer.)

Real collectors: CCSessions (queries DB), ErrorSpike (queries circuit breakers), CriticalFailure (queries circuit breakers).

Impact: The perception pipeline (micro/light reflection) exists and works when triggered, but is almost never triggered because signals stay near zero. Fixing the collectors to query real data sources is required to make the perception pipeline actually perceive.

What to Build

Micro reflection: routine ticks, low urgency. Quick structured extraction:
- Tag signals, extract entities/topics, produce structured JSON summary
- Model: 20-30B local or Gemini Flash fallback
Light reflection: moderate urgency or accumulated micro signals:
- Situation assessment, memory queries, drive-relevant interpretation, basic recommendation
- Model: 20-30B local or Gemini Flash fallback
Prompt templates for each depth (parameterized, with rotating prompt pool to prevent mode collapse)
- Convention: All prompts at Light depth and above must include explicit chain-of-thought scaffolding ("Think through step by step: ..."). CoT significantly improves reasoning quality on judgment-heavy tasks (situation assessment, salience evaluation) at negligible token cost. Micro prompts should NOT use CoT — they are extraction tasks, not reasoning tasks.
Structured output parsing: validate LLM output against schema, retry on malformed
Pre-Execution Assessment prompt pattern in the main agent's system prompt:
- LLM judgment call: "Does this request make sense? Better way? Missing info?"
- Draws on: user model, memory, active context, procedural memory, open questions
- Decision space: proceed / proceed with note / clarify / challenge / suggest alternative
- Most requests pass through instantly (near-zero latency for clear requests)
- Pushback is philosophically mandated, not signal-dependent
- See §Pre-Execution Assessment in design doc
Identity context loading: SOUL.md + user.md loaded into every 20-30B+ prompt
- SOUL.md: src/genesis/identity/SOUL.md — who Genesis is (~1100 tokens, static)
- user.md: src/genesis/identity/user.md — user's self-description (seed + overrides). Draft after Phase 4 implementation. Pull from v2 USER.md: timezone (EST), communication preferences ("brief first, detailed as context deepens", "don't offer next steps by default"), philosophical/first-principles thinking style, "critic when the user locks onto one track" instruction. Omit: v2-specific tool references, v2 action policy (superseded by L1-L4 autonomy), relationship section (superseded by SOUL.md).
- Context assembly must position identity docs early (high attention region)
Context assembly principles (learned from v2 + design discussion):
- v2 injected ~2200 tokens of identity files on every prompt (SOUL+USER+AGENTS+POLICY+MEMORY). v3 is more surgical: SOUL.md (~1100 tokens) + user.md (<300 tokens) + cognitive state (~600 tokens) = ~2000 tokens always loaded. Everything else on-demand via memory.
- Quality is non-negotiable; cost is managed through routing and pacing, never through context degradation. ContextAssembler's job is to assemble the best possible context for the requested depth. It does NOT truncate, budget-gate, or degrade inputs to save tokens. Cost control belongs to the Router (cheaper models first) and the Surplus Scheduler (pacing — how often we reflect, not how well).
- If assembled context significantly exceeds what's typical for a depth, that's a signal the depth classifier should have escalated — not a reason to truncate. Log it as a depth-mismatch signal for future classifier tuning (V4).
- Per-depth context scope (not budget): Micro gets identity + signals. Light adds user profile + cognitive state + relevant memories. Deep/Strategic get richer context. This is about relevance, not cost — don't load irrelevant context, but never cut relevant context to save tokens.
- "Context Triangulation" from v2: don't load full documents when a targeted extract suffices. The question is "what does this reflection need?" not "how much can we afford?"
Cognitive state summary infrastructure:
- cognitive_state table (schema + CRUD): stores regenerated summary of active context + pending actions (~600 tokens, regenerated after Deep reflections)
- Rendered and loaded into 20-30B+ prompts alongside SOUL.md and user model
- At Phase 4, generation is stubbed (manual or triggered by first Deep reflection in Phase 7). The table and rendering pipeline are built now so they're ready.
- See §Cognitive State Summary in design doc

Deferred to Phase 4b (Surplus Wiring)

Once the Reflection Engine is stable, a follow-up build wires surplus (Phase 3) to trigger extra micro/light reflections during idle cycles on free compute. This is a separate build step — Phase 4 proves the reflection pipeline works; Phase 4b connects it to surplus for opportunistic use.

Wire SurplusExecutor to call Reflection Engine for micro tasks (Light moved to CC pipeline)
Surplus queue gets new task types: surplus_micro_reflection, surplus_light_reflection
Idle-cycle reflections use the same prompt templates and output contracts as tick-driven ones
Free compute only — surplus reflections never spend money

Deferred: Observation Sweep (#28)

Call site #28 (Observation sweep — scan environment for noteworthy changes) is deferred from Phase 4. It depends on having things to observe (recon findings, filesystem changes, scheduled job results) which aren't fully wired yet. Place in Phase 5 or later when memory operations and recon-mcp provide observable data.

Verification

Testing Strategy for LLM-Dependent Components

Phase 4 introduces the first LLM calls. LLM outputs are non-deterministic, which requires a different testing approach than the pure data-layer tests in Phase 0.

Unit tests (no real LLM):

Mock/stub the LLM call. Return canned responses.
Test that the surrounding logic (prompt assembly, output parsing, retry on malformed, schema validation) works correctly.
Test that bad LLM output (missing fields, wrong types, empty response) triggers proper error handling and retry.
These run fast, are deterministic, and catch regressions in plumbing.

Golden test sets (semi-deterministic):

Curated set of interaction summaries with expected triage depth assignments.
Run against the real 3B model with temperature=0 (or lowest available).
Assert that depth assignment falls within an acceptable range (e.g., expected depth ±1), not exact match. The model's judgment may shift across versions.
Maintain ~20-30 golden examples covering each depth level.
Run as a separate test suite (pytest -m integration), not with unit tests.

Distribution tests (aggregate behavior):

Feed a batch of ~100 varied interactions through triage.
Assert distribution properties: "at least 40% are Depth 0 or 1" (trivial interactions should be common), "no more than 10% are Depth 3+" (deep reflections should be rare).
This catches systematic bias (model always assigns high depth = expensive).

Real LLM integration tests:

Separate test suite, run manually or in CI with --run-integration flag.
Tests actual model endpoints (Ollama 3B, 20-30B if available).
Slower, costs real compute, but catches model-specific issues.
Not required to pass for every commit — run before releases.

Key principle: Test the system's behavior in aggregate and at boundaries, not the exact text the LLM produces. If you're asserting exact strings from an LLM, you're testing the wrong thing.

V3 Phase 5: Memory Operations

Risk: MODERATE — Extension of existing memory system. Dependencies: Phase 0, Phase 4.

What to Build

Activation scoring: base_score × recency_factor × access_frequency × connectivity_factor
Hybrid retrieval: embedding similarity + activation score for ranking
Observation storage with utility tracking fields (retrieved_count, influenced_action)
Lightweight memory linking at storage time: similarity search → link related memories
Memory-mcp full implementation: replace Phase 0 stubs with real operations
GROUNDWORK for Knowledge Base: multi-collection Qdrant support, source parameter on retrieval, source_type tagging (memory vs. reference)
Open question storage: observations tagged open_question with domain and originating context. Participate in normal memory retrieval — surface when relevant information arrives. No special matching infrastructure needed; the memory system's existing semantic retrieval handles connection-making. See §Open Questions and Persistent Curiosity in design doc.
User model evolution: v2 had a static USER.md (~300 tokens) injected every prompt. v3's user model is richer — it lives in the user model cache (Phase 0 schema) and gets synthesized by Light reflection (#11). Phase 5 builds the retrieval side: when Genesis needs user preferences (e.g., "does the user prefer brief or detailed here?"), it pulls from the user model store, not a static file. user.md is the seed; the user model is the living version. v2 items to seed: timezone, communication style, autonomy preferences, thinking style, relationship expectations.

AZ Memory System Internals (Discovered 2026-03-04)

Deep-dive into AZ's memory implementation reveals the exact surface area for the FAISS→Qdrant swap:

Current implementation:

Memory class (python/helpers/memory.py, ~570 lines) uses MyFaiss — a custom FAISS subclass with InMemoryDocstore. Three areas: MAIN, FRAGMENTS, SOLUTIONS.
VectorDB class (python/helpers/vector_db.py, ~150 lines) wraps FAISS for knowledge/document queries. Used by document_query.py.
MemoryConsolidator (python/helpers/memory_consolidation.py, ~780 lines) — merges/deduplicates memories using LLM judgment. Must continue working post-swap.
Storage: ~/agent-zero/usr/memory/<subdir>/ with index.faiss and serialized docstore. Currently has live data in default/.

Memory dashboard API contract (python/api/memory_dashboard.py):

Actions: search, delete, bulk_delete, update, get_memory_subdirs, get_current_memory_subdir
Calls Memory class methods — confirmed it does NOT call FAISS directly
Frontend formats: id, area, timestamp, content_full, knowledge_source, source_file, file_type, consolidation_action, tags, metadata

Public API to preserve (methods the dashboard and extensions call):

Memory.get() / Memory.get_by_subdir() — factory
search_similarity_threshold(query, limit, threshold, filter)
delete_documents_by_ids(ids)
update_documents(docs)
get_document_by_id(id)
db.get_all_docs() — returns dict of all documents (used for unfiltered listing)

Migration consideration: Existing FAISS data needs one-time migration to Qdrant. Build a migration script that reads existing FAISS indices, re-embeds if needed (dimension mismatch: FAISS may use different embeddings than our 1024-dim qwen3), and inserts into Qdrant collections.

genesis-memory plugin status: Currently just a plugin.yaml with no code. The actual swap implementation goes here.

Verification

Activation scores produce reasonable rankings
Hybrid retrieval differs from pure embedding similarity
New memories link to related existing ones
Utility tracking fields increment on retrieval and action
Memory dashboard renders identically after FAISS→Qdrant swap — V3 built on Qdrant from day one, no FAISS layer
Memory consolidation works with Qdrant backend
[DEFERRED:V4] Existing FAISS memories migrated without data loss — no FAISS data to migrate; V3 is greenfield on Qdrant

V3 Phase 6: Learning Fundamentals

Risk: MODERATE-HIGH — First feedback-dependent behavior. Bad classification compounds. Dependencies: Phase 4, Phase 5.

What to Build

Post-interaction outcome classification:
- approach_failure → change behavior
- workaround_success → store both failed path AND successful workaround as procedural memory. The workaround becomes the primary approach for future identical tasks. This is a positive outcome — Genesis is now better at this task type.
- capability_gap → log, don't false-learn. Requires workaround search exhaustion (2-3 alternative approaches attempted or budget made further search uneconomical).
- external_blocker → user-rectifiable / future feasibility (revisit_after) / permanent. Same workaround search exhaustion requirement as capability_gap. The bar for "permanent constraint" is high — prefer revisit_after when uncertain.
- success → reinforce
Engagement signal extraction (fixed per-channel heuristics):
- Reply speed/length, reactions, follow-ups, explicit feedback
- Channel-specific: WhatsApp reactions, email opens, web UI duration
Basic procedural memory:
- Store with confidence, retrieve top-match, version tracking
- Failure modes stored with conditions (not bare strings) — "fails WHEN X" not just "fails." Include transient flag for conditions that may not persist.
- Attempted workarounds (both successful and failed) stored on parent procedure with specific conditions. Failed workarounds are NOT blanket "never try this" signals.
- Retrieval framing MUST surface failure conditions and workaround history alongside success data — nuance stripped at retrieval makes nuance in storage worthless.
- No confidence decay in V3 (deferred to V4 — needs data to tune decay rate)
Real signal collectors (Phase 1 stubs → real data):
- BudgetCollector, ErrorSpikeCollector, CriticalFailureCollector, TaskQualityCollector — produce programmatic signals (computed metrics with thresholds), not raw data for the LLM to parse. Follows Trackio pattern: domain knowledge embedded in collector code, LLM receives pre-interpreted signals. (MemoryBacklogCollector removed 2026-04-11 — see note in Phase 1 stub list above.)
- ConversationCollector, OutreachEngagementCollector, ReconFindingsCollector, StrategicTimerCollector — may remain stubs until their data sources exist (Phase 8).
Null hypothesis with fixed maturity milestones (DATA VOLUME, not time):
- <50 procedures = early (extract aggressively)
- 50-200 = growing (moderate thresholds)
- 200+ = mature (conservative, only novel patterns)
Basic speculative claim quarantine (safety feature, not data-dependent):
- Claims tagged speculative: true with TTL expiry
- Quarantined from future context retrieval
- Expired with no evidence → archived
- Confirmation cycle (evidence counting, cross-referencing) deferred to V4
Retrospective triage (Step 0 of Self-Learning Loop — runs on EVERY interaction):
- Programmatic pre-filter: < ~100 tokens + zero tool calls → depth 0 (skip)
- Everything else → 3B SLM classification (few-shot prompt + calibration rules)
- Depth assigned by characteristics (complexity, blockers, effort, stakes), NOT by interaction type. A cron job or browser session can be depth 4; a trivially completed formal task might only be depth 1.
- 3B prompt includes curated examples and calibration rules (initially hand-crafted, later maintained by the daily triage calibration cycle)
- V3: all calibration is prompt-level (no fine-tuning)
Signal weight tiers (bootstrap defaults — V3 uses fixed tiers, V4 calibrates):
- Tier 1 (strong): direct user corrections, explicit feedback, rejected deliverables
- Tier 2 (moderate): clear task success/failure, outreach engagement
- Tier 3 (weak): behavioral inference, silence, override outcomes
- Critical constraint: weak signals must NOT erode philosophical commitments (pushback)
- See §Signal Weight Tiers in design doc
Daily triage calibration cycle (companion job to Morning Report):
- 20-30B model reviews sampled triage decisions from last 24h
- Under-classification audit: reads actual chat logs for depth-0 decisions to check whether anything worth capturing was missed
- Over-classification audit: checks whether depth 2+ retrospective outputs were subsequently used
- Memory pattern review for salience shifts
- Outputs updated few-shot examples + calibration rules for the 3B's triage prompt
- Memory signals reach the 3B through the 30B's digested rules, NOT directly
Retrospective observations: Interactions at depth 2+ that produce notable outcomes write a retrospective observation to memory-mcp — what happened, root cause classification, lessons extracted. Depth 2 = notable outcome (discovery, correction, or surprise). Depth 3+ = formal task outcome.

Adaptive Obstacle Resolution (Procedural)

Genesis's philosophical commitment is "failure is not an option" — there is almost always a workaround, the question is how much effort and creativity to spend finding it. This applies to ALL obstacles: web fetching 403s, API rate limits, model unavailability, tool failures, permission errors.

V3 scope: Static fallback chains for known obstacle types (e.g., web fetch: direct → search cache → archive.org → headless browser). Hard-coded ordering, but structured as procedure-like data so V4's learning system can take over ranking.

Why V3 is static: The adaptive system (procedures that learn which workarounds work for which obstacles and update rankings) requires Phase 6's outcome classification and procedural memory. V3 establishes the infrastructure; V4 makes it adaptive.

V4 target: Each obstacle type becomes a learned procedure with a ranked list of resolution methods. Genesis records which methods work for which contexts, updates rankings as methods stop working or new ones emerge, and proactively researches new tools/methods via surplus compute. The procedure's confidence scores reflect real-world effectiveness, not static assumptions.

Design principle: Every tool in the fallback chain is valid — the question is ordering by path of least resistance. A headless browser is overkill for most fetches but valid when simpler methods fail. The learning system maintains this ranking, not a developer.

Tool Capability Discovery

The tool_registry table (Phase 0) stores tool metadata but is currently unpopulated. Phase 6 populates it with real capability data and writes the routing logic that enables content-type-based auto-routing.

What to build:

Populate tool_registry with capability metadata for all available tools (MCP tools, Gemini API, Firecrawl, CC native tools, etc.). Each entry includes: content types handled, access requirements, cost tier, reliability.
Content-type routing function: given a resource (URL, file, content type), query tool_registry for tools that can handle it, return ranked options.
Integration with the Adaptive Obstacle Resolution fallback chains: when the primary tool fails, the routing function suggests the next capable tool.
Cross-model routing table: YouTube → Gemini, web pages → Firecrawl → any LLM, images → Claude vision, PDFs → Claude/Gemini, code repos → CC native. See 2026-03-08-research-insights-and-followups.md Section 3 for the full capability routing table.
capability_gaps table (Phase 0) updated when a content type has no capable tool — feeds the proactive improvement surfacing in morning reports (Phase 8).

Why Phase 6: Tool capability discovery is learning infrastructure. Genesis needs to know what its tools can do before it can autonomously resolve obstacles or evaluate external sources. The static fallback chains (Adaptive Obstacle Resolution above) provide the V3 structure; this populates them with real data.

Sub-Agent Memory Harvesting

When Genesis dispatches CC sub-agent sessions (deep reflection, task execution, surplus), incidental learnings are currently lost. Phase 6 implements two of the three harvesting mechanisms (third in Phase 7):

Mechanism 1 — Structured debrief (Phase 6): CC session system prompt requires a learnings section in final output. Genesis parses this after session completion and feeds findings into memory operations (store as observations, update/create procedures). Prompt engineering addition to session-type CLAUDE.md content.

Mechanism 2 — Auto-memory harvest (Phase 6): After CC session completion, Genesis reads the session's auto-memory directory and ingests relevant items into the Genesis memory store. Richer than structured debrief — captures things the LLM found worth remembering even if not in explicit output.

Mechanism 3 — Cross-run context injection (Phase 7): When launching a CC session, include relevant memories from previous runs of similar tasks. Uses Phase 5 hybrid retrieval (query by task description, inject top-k relevant memories into system prompt). Requires Phase 6 stored learnings.

See 2026-03-08-research-insights-and-followups.md Section 11 for full design.

Inbox Monitor Service

A peripheral service (like the surplus scheduler) that monitors a user-configured folder for new content. This is NOT part of the awareness loop — it's an external task runner with its own schedule. The awareness loop is cognition; this is a hand.

What to build:

InboxMonitor class with its own APScheduler instance (like SurplusScheduler)
Configurable: folder path, check interval, response directory, enabled/disabled
On each check: scan folder for new/modified files (mtime or content hash tracking)
Item classification: link (→ research task), note (→ observe and store), ambiguous (→ queue question for user via outreach pipeline)
Dispatches research tasks to the surplus queue (links → evaluate skill)
Writes response files as Obsidian-compatible markdown to a configurable response subdirectory (e.g., _genesis/) within the watched folder
Pending clarification items queued to outreach/message_queue — NOT processed until user responds through foreground channel
User-configurable via dashboard or config file (path, interval, on/off)

Architecture constraints:

The awareness loop does NOT know about Obsidian or the inbox folder
The inbox monitor feeds INTO existing infrastructure (surplus queue, message_queue, outreach pipeline) — it does not bypass them
Response files written atomically (temp file + rename) to avoid sync conflicts
Processed items tracked by content hash to avoid re-processing

Why Phase 6: Requires outcome classification (is this a link? a note? ambiguous?) and procedural memory (learn which inbox patterns are research vs noise). The surplus scheduler infrastructure (Phase 3) provides the scheduler pattern. CC session dispatch (Phase 7 session_config) handles the actual evaluation work.

Minimum viable version (post-Phase 6): Monitor folder, classify items, dispatch link evaluation to CC sessions, write markdown responses. No tag-based routing, no proactive research, no graph integration (those are V4).

See docs/plans/2026-03-09-inbox-monitor-plan.md for implementation plan.

Status: IMPLEMENTED (2026-03-10) Design updated: LLM-first classification (no heuristic pre-classification layer). The LLM reads raw content and decides per-item whether to research, note, or question. Batching: max 5 items per CC session, overflow creates additional sessions. State tracked in inbox_items DB table (not JSON file). System prompt at src/genesis/identity/INBOX_EVALUATE.md. AZ extension _55_genesis_inbox.py. 51 new tests (1067 cumulative).

Skill Wiring for Genesis Sessions

Skill conventions: All Genesis skills MUST follow the conventions in docs/reference/genesis-skill-conventions.md — description format, writing style, size limits, progressive disclosure, Genesis-specific frontmatter fields. Read that doc before creating or modifying any skill.

AZ's skill system discovers SKILL.md files from plugin directories. Genesis skills (e.g., RESEARCH_EVALUATION.md) must be wired into the AZ plugin's skill directory for AZ-runtime discovery.

What to build:

Copy/symlink Genesis skills from src/genesis/skills/ into usr/plugins/genesis/skills/<skill-name>/SKILL.md with proper frontmatter (name, description, triggers matching Genesis's use cases).
Ensure AZ agent can skills_tool:load Genesis skills when relevant signals trigger them (e.g., new tool discovered → load research-evaluation skill).
Skills are living documents — procedural learning (this phase) can update skill files when Genesis learns better approaches.
Adopt SKILL.md structure from Superpowers pattern: progressive disclosure (metadata always loaded, SKILL.md body on trigger, references/ on demand).
The evaluate command (.claude/commands/evaluate.md) should be restructured as a Genesis plugin skill (src/genesis/skills/evaluate/SKILL.md + references/) for use in CC background sessions. Interactive CC command stays as-is.

Why Phase 6: Skills need to be discoverable before Genesis can autonomously use them for evaluation or research. Phase 6's procedural learning also enables skill self-updating — Genesis observes "this evaluation approach worked better" and updates the skill file.

Skills inventory exercise:
- Before building individual skills, enumerate ALL skills Genesis needs across all consumers: CC background sessions (evaluate, research, reflect, retrospective, debug), foreground conversation (morning-report, outreach), autonomous operation (task-planning, verification, self-assessment, obstacle-resolution), inbox monitor (classify, research).
- LangChain demonstrated 25%→95% performance improvement from domain-specific skills. Skills are not optional — they are critical infrastructure.
- Design each skill following the industry-standard SKILL.md pattern with progressive disclosure (advertise ~100 tokens, load <5000 tokens, read resources on demand).

Perception Template Externalization

Phase 4's PromptBuilder has 6 reflection templates (3 micro, 3 light) hardcoded as Python strings. GL-1 proved that externalizing prompts to CAPS markdown files works better — easier to edit, version, audit, and test.

Near-term cleanup: Externalize the 6 templates to src/genesis/identity/ as CAPS markdown files (e.g., MICRO_TEMPLATE_HEALTH.md, LIGHT_TEMPLATE_PATTERN.md). The PromptBuilder loads them the same way CCReflectionBridge loads reflection prompts — with fallback to hardcoded strings if files are missing.

Why: Transparency — the user should be able to see and edit everything that shapes how Genesis thinks. Hardcoded strings hide behavior in code that only developers can audit. CAPS markdown files make it visible to anyone.

Phase 4 Follow-Up: Reflection Template Expansion

Phase 4 ships with 3 micro templates and 3 light templates. By Phase 6, outcome classification data reveals which templates produce useful observations vs noise. Use this data to:

Identify blind spots (signal patterns none of the 3 templates catch well)
Identify redundancy (templates that overlap >40% of the time)
Add templates to fill demonstrated gaps — target 5-7 micro once stabilized
Do NOT add templates on a schedule. Only add when data justifies it.

Critical Risk

This is the highest-leverage failure point. Bad classification → bad lessons → systematic drift. Mitigation: null hypothesis = conservative by default. Monitor capability_gap vs approach_failure ratio — heavy skew = classification probably wrong.

Verification

V3 Phase 7: Simple Deep Reflection ✅ COMPLETE

Risk: MODERATE — LLM call at Sonnet-class, but single structured prompt (no multi-model orchestration). Dependencies: Phase 4, Phase 5, Phase 6.

Status (2026-03-13): Reflection scheduler, context gatherer, output router, learning stability monitor, CC session bridge — all implemented and tested.

What to Build

Single structured Deep reflection (Sonnet-class):
- Triggered by Awareness Loop when composite urgency crosses Deep threshold
- Only runs jobs with pending work (adaptive, NOT calendar-driven)
- Covers when applicable:
  - Memory consolidation (if backlog of observations)
  - Cost reconciliation (if spend changed since last check)
  - Lessons extraction (if new retrospectives available)
  - Recon triage (if new findings accumulated)
  - Review surplus staging area — promote or discard brainstorm outputs
  - Skill effectiveness review — analyze skill-tagged session outcomes, trigger refiner for underperforming skills (see Skill Evolution section)
  - Regenerate cognitive state summary — model already has full picture in context; final step produces ~600-token summary of active context + pending actions, written to cognitive_state table
- Reads current cognitive state summary for continuity ("what was I focused on?")
- "Dream Cycle 2.0" — same jobs as current dream cycle, smarter trigger, single model call
Explicit memory consolidation in Deep reflection prompt:
- Deep reflection prompt (REFLECTION_DEEP.md) must include explicit consolidation directives: deduplicate related memories, merge overlapping observations, restructure stale links, flag contradictions. Current prompt focuses on pattern recognition only.
- "Dreaming" pattern: periodic LLM pass that reorganizes memory, not just reads it.
- This is the v3 equivalent of the v2 dream cycle's memory cleanup jobs, but LLM-driven rather than programmatic.
Mandatory weekly self-assessment (Sunday, configurable):
- Runs as a Deep reflection job even if no other Deep triggers are pending
- Evaluates 6 dimensions: reflection quality, procedure effectiveness, outreach calibration, learning velocity, resource efficiency, blind spots
- Each dimension has a concrete data source (no vague self-congratulation)
- Output: structured assessment → memory-mcp (episodic, tagged self_assessment)
- See §Weekly Self-Assessment in design doc
Weekly quality calibration (companion job to weekly self-assessment):
- More capable model samples recent task outputs and assesses:
  - Quality gate strictness: were passes justified?
  - Pushback frequency: did Genesis challenge when a thoughtful person would have?
  - Quality drift: are standards slipping compared to earlier periods?
  - Signal weight compliance: did weak signals erode philosophical commitments?
  - Procedure effectiveness: do tasks where learned procedures were applied succeed more than baseline? Track per-procedure success rate trend.
  - Learning velocity sanity: is the rate of new procedures/observations reasonable? Contradictory observations at high rates = something wrong.
- Output: quality calibration observations in memory-mcp, tagged quality_drift if drift detected. Retrieved during future Pre-Execution Assessments as counterweight.
- See §Quality Calibration Cycle in design doc
Learning stability monitoring (integrated into quality calibration + deep reflection):
- Problem: same failure mode as Test-Time Training at neural level — unconstrained updates from learning can degrade system performance rather than improve it. Procedures learned from recent-but-atypical interactions can override sound general knowledge. Observations can accumulate contradictions without resolution.
- Procedure quarantine: when weekly quality calibration detects a procedure's success rate declining (applied 3+ times, success rate below 40%), quarantine it: set a quarantined flag that excludes it from retrieval. Still stored, can be rehabilitated if circumstances change. Quarantine is a deep reflection output.
- Contradiction detection: deep reflection memory consolidation explicitly checks for observations that contradict each other. When found: resolve (keep the one with stronger evidence), merge (synthesize a nuanced replacement), or flag for user review (when evidence is ambiguous). Don't let contradictions accumulate silently.
- Learning regression signal: if weekly self-assessment's "procedure effectiveness" dimension trends downward for 2+ consecutive weeks, emit a learning.regression event via event bus AND include it as a cognitive state item. This makes the regression visible to every subsequent reflection and pre-execution assessment.
- Consolidation-as-defense: deep reflection memory consolidation isn't just organization — it's the primary defense against memory pollution. Dedup, merge, and prune are safety operations, not housekeeping. Treat them with that gravity.
Research output documentation structure:
- When Genesis performs Deep reflection, strategic evaluation, or autonomous research, it produces structured written findings. These need a defined storage location and organization scheme — Genesis can't just dump everything into a flat memory store.
- Research during Phase 7 planning: evaluate options (dated files in docs/plans/, memory-mcp tagged entries, cognitive state journal, or a hybrid). Decide on a structure that supports both retrieval (memory system) and human audit (file system).
- This is a design question to resolve during Phase 7 implementation, NOT to defer. The decision affects how all subsequent phases store their outputs.
CC session skill loading (session_config.py):
- Phase 7 is when background CC sessions become the primary mechanism for Deep reflection. These sessions need skills injected based on session type.
- session_config.py generates per-session-type configs: MCP servers, hooks, CLAUDE.md content, and skill references. Reflection sessions doing strategic evaluation get the research-evaluation skill. Task sessions get verification and debugging skills.
- Mechanism: --append-system-prompt or --system-prompt flag in CCInvoker with skill content included for that session type.
- See 2026-03-08-research-insights-and-followups.md Section 13 for the full session type × capability matrix.
Cross-run context injection (sub-agent memory harvesting mechanism 3):
- When launching a CC session for a task, include relevant memories from previous runs of similar tasks. Uses Phase 5 hybrid retrieval (query by task description, inject top-k relevant memories into system prompt).
- Requires: Phase 6 stored learnings (structured debrief + auto-memory harvest must already be populating the memory store).
- See 2026-03-08-research-insights-and-followups.md Section 11.
Skill evolution system (skills as learning artifacts, not static config):
- Conventions baseline: docs/reference/genesis-skill-conventions.md defines the structural and stylistic standards all skills must follow. The SkillRefiner must produce proposals that comply with these conventions (description format, imperative writing style, size limits, progressive disclosure structure).
- Design invariant: Skills are not static configuration — they are learning artifacts governed by the same lifecycle as procedures (creation → usage → measurement → refinement → quarantine/retirement). The learning system and skill system are a single connected loop, not two disconnected ones. skill_tags on CCInvocation (added Phase 6) provide the data pipeline; this phase builds the analysis and action layers.
- SkillEffectivenessAnalyzer (genesis.learning.skills.effectiveness):
  - Computes per-skill health metrics from cc_sessions (skill_tags in metadata) + outcome data from observations/debriefs. No new tables — computed on-the-fly.
  - SkillReport: usage_count, success_rate, baseline_success_rate (from untagged sessions doing similar work — answers "does this skill actually help?", per the LangChain 25%→95% evidence), failure_patterns, trend (IMPROVING/STABLE/DECLINING), tools_used vs tools_declared (mismatch = evolution signal), sessions_since_last_refined.
  - Trend computed from rolling window (last 10 uses vs prior 10).
- Skill typing — each skill classified as uplift (extends what Genesis can do), workflow (describes how to do what it already can), or hybrid. Stored in SKILL.md frontmatter. Matters for evolution: uplift failures suggest capability gaps (need new tools/integrations); workflow failures suggest bad instructions (need prompt rewrites).
- SkillRefiner (genesis.learning.skills.refiner):
  - LLM call: takes SkillReport + current SKILL.md → SkillProposal (proposed content, rationale, change_size classification, confidence, failure_patterns_addressed).
  - Change size: MINOR (description tweaks, examples, clarifying steps, <10 lines) | MODERATE (workflow steps, new sections, thresholds) | MAJOR (full restructure, new skill creation, retirement).
  - Progressive disclosure enforcement: if a skill grows past ~500 lines through evolution, propose restructuring into SKILL.md + references/ subdirectory rather than appending more content (Microsoft three-stage model: advertise → load → read).
  - Classification is LLM-determined, sanity-checked by diff line count.
- SkillApplicator (genesis.learning.skills.applicator):
  - Applies or stages based on change size × autonomy level:
```
Autonomy L2 (V3 default):  MINOR auto-applies, MODERATE+ staged
Autonomy L3 (earned):      MINOR + MODERATE auto-applies, MAJOR staged
Autonomy L4 (earned):      All auto-apply
```
  - Auto-applied: write to SKILL.md, log as observation tagged skill_evolution.
  - Staged: write as observation tagged skill_proposal, include full diff + rationale.
  - Validation step for MODERATE+ proposals: second LLM call evaluates the proposal against the failure patterns it claims to address. GROUNDWORK(skill-ab-testing): V4 replaces this with actual A/B eval runs comparing old vs new skill on similar prompts.
  - All changes tracked in observations — the memory system IS the audit trail.
- Deep reflection integration: "skill effectiveness review" added as a deep reflection job. Runs when new outcome data exists for skill-tagged sessions since last review, or when a skill's success rate drops below threshold. Produces SkillReports for active skills, triggers refiner for underperformers. Subject to same learning stability monitoring (quarantine, contradiction detection, regression signals).
- GROUNDWORK(skill-autonomy-graduation): Autonomy graduation for skill evolution. Uses existing autonomy_state table — new category skill_evolution, starts at L2. Elevation criteria (V4 activates, V3 stores):
  - User approval rate of staged proposals > 80% over 20+ proposals
  - Zero regressions from auto-applied changes over 30-day window
  - No quarantined skills that were auto-modified
- GROUNDWORK(skill-ab-testing): V4 adds actual A/B testing — run two skill versions on similar tasks, compare outcomes. Replaces the second-opinion LLM validation.
- GROUNDWORK(skill-auto-creation): V4 adds new skill creation from scratch — pattern recognition across many sessions identifies repeated workflows worth crystallizing.
- GROUNDWORK(skill-manifest): V4 adds structured capability manifests (HAND.toml → SKILL.toml pattern from OpenFang) — explicit tool/permission/metric declarations per skill. V3 tracks tools_used empirically; V4 makes declarations enforceable.
- Not in V3: skill retirement (needs enough data), skill A/B testing (needs eval infrastructure), new skill creation from scratch (needs pattern recognition across many sessions), meta-skill learning (V5 — learning how to write better skills).
No meta-prompting — uses a comprehensive static prompt. V4 replaces this with the 3-step meta-prompting protocol for higher quality.
No Strategic reflection — weekly MANAGER / monthly DIRECTOR reviews are V4.
No fresh-eyes review on Deep outputs — V4 adds this for high-stakes outputs.

Why This Exists (Not Just Micro/Light)

Without Deep reflection, V3 has no equivalent of the current nightly dream cycle. Micro/Light handle real-time signal processing; Deep handles the periodic "step back and think about the bigger picture" that consolidation, lesson extraction, and surplus review require. A single Sonnet-class call with a comprehensive prompt is a massive upgrade over 13 fixed-interval cron jobs that run every night regardless of whether there's anything to process.

Verification

V3 Phase 8: Basic Outreach ✅ COMPLETE

Risk: HIGH — User-facing. Trust damage from bad outreach is hard to reverse. Dependencies: Phase 3, Phase 6, Phase 7.

Status (2026-03-13): Governance gate, fresh-eyes review, outreach pipeline, morning report, engagement tracker, calibration infrastructure, outreach MCP, recon MCP — all implemented and tested. Channel registration wired. Morning report failures observable via event bus.

Dashboard & UI (2026-03-13): Genesis dashboard at /genesis with 4 panels (system health, pending actions, activity feed, config files). Provider activity tracker (ProviderActivityTracker) gives embedding call counts, latency, cache hit rates, error rates — exposed via /api/genesis/provider-activity and health_status() MCP tool. EmbeddingProvider instrumented with activity tracker + event bus events for fallback/failure. AZ UI rebranded with Genesis nav button. Outreach API blueprint wired into bootstrap. See dashboard design spec.

Pulled forward (pre-Phase-8): Health MCP tool implementations (health_status, health_errors, health_alerts) and the Neural Monitor dashboard were built ahead of Phase 8. The shared HealthDataService backend serves both the human dashboard (Flask API at /api/genesis/health, dashboard at /genesis/monitor) and the health MCP tools (Genesis self-awareness). See neural monitor design spec. Runtime now exposes circuit_breakers, cost_tracker, dead_letter_queue, deferred_work_queue, cc_budget_tracker as properties.

Post-Phase-8 audit (2026-03-13): Fixed 5 routing bugs (call site ID mismatch in retrospective triage, fresh_eyes method name, test references, Ollama model name). Changed circuit breaker to trip on PERMANENT errors (broken providers now show red, not green). Enriched HealthDataService with awareness heartbeat, queue ages, CC session durations, outreach rates, disk space, and Ollama model validation. New health alerts for tick overdue, stale dead letters, disk low, and Ollama model mismatch. Deferred signals documented in docs/plans/health-signal-gaps-deferred.md.

Knowledge autonomy reference: genesis-knowledge-autonomy.md defines the principle that Genesis should handle all knowledge ingestion, organization, retrieval, and proactive integration without the user managing infrastructure. Phase 8 is where multi-channel ingestion, retrieval confidence gating, and ingestion feedback land. Read that doc when starting this phase.

What to Build

Outreach pipeline: staging → governance check → channel selection → timing → delivery → tracking
Governance gate (deterministic, before every outreach):
- Within autonomy permissions?
- Passes salience threshold? (fixed threshold in V3)
- Timing appropriate? (quiet hours from config)
- Similar outreach sent recently? (dedup)
- Budget check for paid channels
Outreach categories in V3: Blocker and Alert only
- Finding/Insight/Opportunity categories deferred to V4 (need calibrated user model)
Exactly 1/day surplus-driven outreach from day 1 of autonomous behavior:
- Sourced from daily brainstorm staging area (Phase 3)
- Labeled as surplus-generated
- Governance-gated and engagement-tracked
- This is how the system learns to be proactive — it can't grow without opportunities
Engagement tracking per outreach:
- Delivered, opened, replied to, reply sentiment, action taken, ignored
- Fed back to Learning Loop (Phase 6)
Fixed channel preferences (config-driven, not learned — channel learning is V4)
Fresh-eyes review on outreach before sending: cross-model check on the 1/day surplus outreach
Daily morning report:
- Trigger: first idle cycle after configured morning time (default: 7:00 AM), or first interaction of day
- NOT a template checklist — Genesis decides what's worth saying based on cognitive state summary, overnight activity, system state, and pending items
- Question seam: may include 1-2 questions from recent self-reflection — the most useful thing Genesis is currently uncertain about. Not a questionnaire, not a backlog. If unanswered, the outreach attempt expires but the question persists in memory as an open observation (see §Open Questions in design doc).
- Delivered via the outreach pipeline (same governance, same channel selection)
- Engagement-tracked — system learns what users find useful in morning reports
- Model: Light depth (20-30B / Gemini free) — daily surplus task, not premium
- V3: static prompt template. V4: meta-prompted for adaptive content selection.
- See §Daily Morning Report in design doc

Approval Dashboard UI

The task queue, surplus staging area, and outreach approval workflow need a user-facing UI. Rather than building a separate "Mission Control" application, Genesis builds this as an AZ dashboard panel — same Alpine.js, same Flask API pattern, same auth as the existing memory dashboard.

The dashboard shows:

Pending proposals — surplus insights, outreach drafts, task suggestions awaiting user approval
In-progress tasks — background work Genesis is currently executing
Completed work — recent results with outcomes and cost
Approval actions — approve, reject, defer, modify scope

Identity & Prompt File Audit Page

All CAPS markdown files in src/genesis/identity/ that shape LLM behavior (SOUL.md, USER.md, REFLECTION_DEEP.md, REFLECTION_STRATEGIC.md, future perception templates) should be visible and editable from the dashboard.

Why: These files define how Genesis thinks. Hiding them in the codebase breeds opacity — the user should be able to audit and modify anything that isn't genuinely hardcoded. This is the difference between a system the user trusts and a black box. If a behavior feels wrong, the user should be able to trace it to a specific file and change it.

Implementation: A dashboard panel that discovers all .md files in the identity directory, displays their content with syntax highlighting, and allows inline editing with save. Changes are written to disk immediately (no git commit required — the user can manage version control separately).

Convention: CAPS filenames (e.g., SOUL.md, not soul.md) signal "user-editable, not internal implementation." Only constants that MUST be hardcoded (schema definitions, protocol values, safety constraints) live in Python code.

This is a V3 Phase 8 UI deliverable. The queue infrastructure is Phase 3 (surplus). The approval logic is Phase 9 (autonomy). The dashboard ties them together for the user.

See docs/reference/agent-zero-architecture-deep-dive.md → Memory System → Memory Dashboard for the pattern Genesis follows (Alpine.js + single Flask endpoint + action-based API).

CC Chat Widget

The Genesis dashboard includes a CC-backed chat widget — the web UI equivalent of the Telegram relay. Calls ConversationLoop.handle_message() via a new Flask API endpoint (genesis_chat). Same session persistence, morning reset, and intent parsing as Telegram.

Why not rewire AZ's existing chat: AZ's web UI chat is deeply coupled to AgentContext — chat history, context management, tool call visualization, sub-agent rendering. Replacing that backend with CC would mean reimplementing most of the UI plumbing. Building a separate Genesis chat widget is cleaner, has zero rebase liability, and naturally grows as phases land.

Convergence: AZ's chat tab becomes a debug/legacy tool as CC handles all intelligent conversation. The Genesis dashboard is the primary user-facing UI.

AZ UI Architecture Findings (Discovered 2026-03-04)

Deep-dive into AZ's web UI reveals the implementation approach for this dashboard:

Approach: Option B + C — Standalone Genesis dashboard modal (visibility) PLUS Genesis config via MCP tools and chat (seamless on-the-fly changes). The dashboard gives the user a place to see everything; chat-based config lets them change things without leaving the conversation.

Why NOT a new settings tab:

AZ's 6 settings tabs are hardcoded in settings.html — no plugin extensibility
Settings fields are hardcoded in individual HTML templates
Backend Settings TypedDict is a fixed schema in settings.py
Adding a tab requires patching AZ core files (rebase liability)

What IS extensible (use these):

Modal system (window.openModal(path)) — load any component as a modal
Component loader (<x-component path="...">) — lazy-loads HTML + JS
Alpine.js stores (createStore("name", model)) — reactive state management
API handlers — add new Flask endpoints for Genesis data
Notification system — already bridged via Genesis observability

Implementation pattern:

Add a Genesis button/icon to the sidebar or as a floating action button (minimal core touch — one line in sidebar HTML, or injected via extension)
Button opens Genesis dashboard modal via window.openModal()
Dashboard loads Genesis-specific Alpine.js components
Backend: new API handlers (genesis_status, genesis_config_get, genesis_config_set) following AZ's ApiHandler pattern
Config stored in Genesis DB tables, NOT in AZ's settings.json
MCP tools expose the same config surface for chat-based changes

Technical requirements:

All API calls need CSRF token + Origin header (http://localhost:5000)
Session cookie is scoped per runtime ID (session_{runtime_id})
Use API.callJsonApi("endpoint_name", params) from frontend JS
Follow AZ's ApiHandler base class with @requires_auth() decorator

Dashboard sections (refined):

System Status — health probes (DB, Qdrant, Ollama, scheduler)
Awareness Loop — current depth, last tick, signal readings
Routing — calls today, budget remaining, circuit breaker states
Surplus — queue depth, LM Studio availability, brainstorm schedule
Memory — Qdrant collection stats, entry counts
Pending Approvals — proposals awaiting user action
Quick toggles — enable/disable awareness, surplus, outreach

Outreach Timing: Why Day 1

The 1/day surplus outreach starts the moment V3's autonomous behavior goes live — not after a burn-in period. Rationale:

V4 calibration needs engagement data. No outreach = no data = can't calibrate.
V5 anticipatory intelligence needs engagement history. No early data = flying blind.
The governance gate + engagement tracking ensure even early outreach is tracked and learned from.
User can always disable if too noisy. Starting conservative but present > starting silent.
1/day is conservative enough that even mediocre early outreach won't overwhelm.

Bayesian Calibration Infrastructure (AGI Gap Items 1-3)

Start prediction logging early so calibration data accumulates before Phase 9 needs it. This is pure instrumentation — zero LLM cost.

Prediction logging table (predictions):
- Schema: {id, action_id, timestamp, prediction, confidence, confidence_bucket, domain, reasoning, outcome (nullable), correct (nullable), matched_at (nullable)}
- Every non-trivial decision logs a structured prediction with confidence
- confidence_bucket is the binned value (e.g., "0.7-0.8") for calibration curves
- domain enables per-domain calibration (outreach, triage, procedure, routing)
Prediction-outcome reconciliation — batch job (surplus compute) that:
- Finds predictions with no matched outcome
- Matches against execution trace results, user engagement signals, procedure success/failure
- Marks predictions as correct/incorrect/uncertain
- Runs daily alongside morning report infrastructure
Calibration curve computation — pure programmatic job:
- Groups predictions by confidence_bucket × domain
- Computes actual_success_rate / predicted_confidence per bucket
- Produces calibration correction function per domain
- Stores curves in calibration_curves table for Phase 9 context injection
Justification: Google research on "Bayesian Teaching" (2026) confirmed LLMs hit a "one-and-done plateau" — they fail to update beliefs from sequential evidence. This calibration system is the architectural compensation: a symbolic system that tracks belief accuracy, paired with LLM context injection (Phase 9). See research doc §26.

Verification

V3 Phase 9: Basic Autonomy ✅ COMPLETE

Risk: HIGH — Permission errors = trust damage. Dependencies: Phase 6, Phase 8.

What to Build

Autonomy hierarchy L1-L4 with fixed defaults:
- L1: Simple tool use → fully autonomous
- L2: Known-pattern tasks → mostly autonomous
- L3: Novel tasks → propose + execute with checkpoint
- L4: Proactive outreach → threshold-gated + governance check
- L5-L7 deferred to V5 (need operational evidence to grant)
Context-dependent trust ceiling:
- Direct session: earned level (no cap)
- Background cognitive: L3 max
- Sub-agent: L2 for irreversible, earned for reversible
- Outreach: L2 until engagement data proves calibration
Autonomy regression (rule-based, works immediately):
- 2 consecutive corrections → drop one level, re-earn
- 1 user-reported harmful action → drop to default, full re-earn
- Regression is announced, not silent
CLAUDE.md per-task isolation: sub-agents do NOT share a CLAUDE.md write path
Hard verification gate (Symphony Proof of Work pattern):
- Task completion MUST include: all tests pass + lint clean (ruff check .) + diff review + structured explanation of changes + before/after state comparison.
- This is architectural enforcement, not a skill suggestion. The CCInvoker post-session hook validates these artifacts before marking a task complete.
- If verification fails, the task is NOT marked complete — it stays in-progress with the failure reason attached.
- See 2026-03-08-research-insights-and-followups.md Section 6 for full pattern.
Bugbot self-review (auto-review on push to own repo):
- Genesis monitors pushes to its own repository (via webhook or git hook).
- On each push: runs lint + tests + semantic review of the diff.
- Flags regressions, style violations, and potential issues as observations.
- This is Genesis dogfooding its own autonomous quality control.
- See 2026-03-08-research-insights-and-followups.md Section 10.
Autonomous obstacle resolution escalation:
- Phase 6 builds static fallback chains and tool capability discovery.
- Phase 9 adds the autonomy judgment: how many alternatives to try before escalating to the user, when to spend compute on creative workarounds vs accept a limitation, and how to surface the obstacle in outreach.
- Uses autonomy level to gate escalation behavior: L1-L2 escalate quickly, L3-L4 exhaust more alternatives before involving the user.
No autonomy progression in V3 — levels are fixed by user. V4 adds evidence-based progression.

Calibration-Informed Autonomy (AGI Gap Items 4-6)

Phase 8 accumulates prediction data. Phase 9 uses it to inform autonomy decisions.

Calibration feedback injection — context assembler includes calibration history in prompts: "When you report 80% confidence on outreach decisions, you're historically right ~60% of the time. Adjust accordingly." Per-domain curves from Phase 8's calibration_curves table. This is the "symbolic + neural" pairing validated by Google's Bayesian Teaching research.
Hard disagreement gates — when cross-vendor review (call sites #17/#20) disagrees with primary model, the action BLOCKS until resolved:
- Third model tiebreaker, OR
- Escalation to user with both assessments
- Promotes "disagreement-as-signal" from deferred pattern (design doc line 2640) to core infrastructure
- Disagreement rates tracked — if >30% disagreement, the system has calibration problems beyond what gates can fix
Decision trace verification — for structured decisions (routing, triage, procedure selection), mechanically verify stated reasons match actual data:
- "I chose depth 2 because token count > 500" → verify token count was > 500
- "I routed to Sonnet because task is complex" → verify complexity score
- No LLM needed — pure programmatic validation
- Failed verifications logged as calibration.trace_mismatch observations

Approval Timeout & Auto-Reject Policy

When Genesis proposes an action requiring user approval (L4 outreach, scope changes, irreversible operations), the proposal cannot hang indefinitely:

Timeout window — configurable per-action-type (default: 60 minutes for outreach, 24 hours for task proposals, no timeout for irreversible operations)
Auto-reject on timeout — expired proposals are rejected, not silently executed. Genesis logs the timeout and may re-propose later if conditions still warrant it.
No auto-approve — timeout never means "go ahead." Silence is absence of consent, not consent.
Kill switch — user can cancel any pending or in-progress autonomous action via dashboard or chat command. Cancellation is immediate and logged.

This prevents the failure mode where Genesis queues up proposals during user absence and executes a backlog when the user returns, or worse, assumes silence means approval.

PAI-Informed Additions (from Miessler deep dive, 2026-03-12)

Mandatory task structure floor — ANY task entering the task execution pipeline (background sessions, surplus tasks, dispatched CC work) MUST include:
1. Planning: Define what you're doing and how.
2. Verification: Confirm output matches plan.
3. Learning: Extract what worked and what didn't. The LLM decides HOW MUCH of each, not WHETHER they happen. Foreground conversation is exempt (handled by Self-Learning Loop retrospective). Rationale: PAI's The Algorithm mandates 7 phases on every task. Our "proportionate effort" philosophy is correct for conversation but was letting the LLM cut corners on serious work. This is the minimum structural floor.
CC lifecycle hooks (3 of 5) — Implement CC hooks for Phase 9: SessionStart (identity + steering + memory load), Stop (learning trigger + state persistence), SessionEnd (retrospective + cleanup). These are the guaranteed lifecycle injection points, not fallback. The remaining 2 hooks (PreToolUse for security invariants, UserPromptSubmit for work detection) are deferred to V4 — they require a security model and work detection pipeline not yet defined.
CC session continuity (Context Mode pattern) — Adopt the session continuity hook architecture from Context Mode MCP server for Genesis's dispatched CC sessions. Key pattern: PostToolUse captures structured events (file/git/task/error) into SQLite, PreCompact builds a priority-tiered snapshot (≤2KB: active files, tasks, rules, errors, git ops), SessionStart restores state after compaction via FTS5-indexed event history + Session Guide. This extends CC session lifetime for long-running autonomous work (reflection, surplus, task execution). Complements the 3 lifecycle hooks above — those handle Genesis identity/learning injection; this handles working state preservation across compaction boundaries. Source: Context Mode MCP server (github.com/mksglu/context-mode), research 2026-03-14.
Work item detector — V4 scope (moved from Phase 9 — this is an advanced autonomy feature). During or immediately after conversation, identify actionable items not being addressed now. Stage them in surplus queue with conversation context, priority, and originating exchange. Connects to: surplus staging → background CC execution → outreach delivery. Inspired by PAI's AutoWorkCreation hook.
Fast-path steering rules + STEERING.md — When user gives unambiguous negative feedback ("never do X"), immediately create an observation tagged steering_rule that gets injected into future context. Don't wait for calibration cycles on clear corrections. STEERING.md is a first-class identity file loaded alongside SOUL.md — contains cross-cutting behavioral rules auto-populated from strong negative feedback.
Task state files — Simple JSON/MD per task, written by Stop/SessionEnd hooks, read by SessionStart hook. Survives context compaction. Contains: task description, current phase, key decisions, blockers, outputs. Inspired by PAI's PRD (Persistent Requirements Document) pattern.
Time budget awareness — Prompt-level injection in mandatory task structure: "You have N minutes. Allocate ~20% planning, ~60% execution, ~20% verification." LLM decides allocation; the structure enforces awareness.
/btw skill — "By the way" command: store a quick note to memory without derailing current task. Simple skill wrapping memory store + continue signal.

Verification

CC Update Compatibility (Cross-Cutting)

Claude Code is as critical a dependency as Agent Zero. CC updates (features, breaking changes, deprecations) directly affect CCInvoker, reflection_bridge, session_manager, and all CC background sessions.

Tracking document: docs/reference/cc-compatibility.md (to be created)

Maps CC features → Genesis components that depend on them
Maps CC version requirements → Genesis minimum compatible version
Tracks CC capabilities Genesis is NOT yet using → evaluation queue
Tracks CC deprecations → migration plans

Process: When CC updates, consult compatibility doc. Evaluate:

Does this affect our CCInvoker/reflection_bridge/session_manager?
Does it unlock something we're working around?
Does it obsolete something we built?
CC 2.1 features relevant to Genesis: hooks in frontmatter (Phase 7 session_config), forked skill context (Phase 6 skill wiring), wildcard permissions (session setup).

V3 Summary

V3 delivers a complete working copilot that:

Monitors signals and classifies urgency (Awareness Loop, fixed weights)
Routes compute intelligently with local/cloud fallback and cost observability
Leverages free compute from day 1 (surplus infrastructure early)
Reflects at 3 depths: Micro, Light, Deep (static prompts)
Maintains a cognitive state summary — regenerated after Deep reflections, loaded into every fresh context
Stores and retrieves memory with activation scoring
Classifies outcomes and builds procedural memory (null hypothesis defaults)
Runs 2+ daily brainstorming sessions on free compute ("upgrade user" + "upgrade self")
Sends a daily morning report (adaptive, not a checklist)
Sends 1 proactive surplus outreach/day + alerts/blockers
Runs a mandatory weekly self-assessment ("am I getting better?")
Has fixed L1-L4 autonomy with automatic regression
Quarantines speculative claims from context

What V3 does NOT do (by design, not by omission):

No meta-prompting (uses static prompts — V4)
No Strategic reflection (no MANAGER/DIRECTOR reviews — V4)
No calibration loops (fixed weights, fixed thresholds — V4)
No channel learning (config-driven — V4)
No confidence decay on procedures (V4)
No ISC (Ideal State Criteria) for task verification (V4 — needs Phase 9 data)
No explicit SIGNALS rating capture (V4 — user rates interactions, feeds calibration)
No inbox tag-based routing (V4 — requires vault-wide scanning + user model)
No proactive inbox research (V4 — Genesis-initiated research based on user's notes)
No graph-connected outputs (V4 — wiki links to user's existing vault notes)
No natural language sentiment extraction (V4 — needs SIGNALS consumer + calibration data)
No proactive context compaction (V4 — needs operational experience from Phase 9 tasks)
No work item detection from conversation (V4 — UserPromptSubmit hook + detection pipeline)
No autonomy progression (fixed levels — V5)
No identity evolution (static identity — V5)
No anticipatory intelligence (V5)
No meta-learning (V5)
No L5-L7 autonomy (V5)

Retroactive Fix: Cost-as-Observability (Phases 0-3)

The "Quality over cost" principle (see CLAUDE.md) was established during Phase 4 design. Several Phase 0-3 components have automatic cost-based behavior that violates this principle. These need updating before or during Phase 4:

Router budget gating (routing/router.py): Auto-skips paid providers when BudgetStatus.EXCEEDED. Change to: emit event, let Genesis + user decide. The Router should still report budget status; it should not act on it.
never_pays constraint (routing/router.py): Hard-filters call site chains to free-only. Change to: a routing preference in config, overridable by Genesis. Surplus tasks prefer free compute but aren't forbidden from paid.
ComputeTier.NEVER rejection (surplus/queue.py): Hard rejects tasks above a cost tier. Remove the NEVER tier concept — tasks go where compute is available.
DegradationTracker auto-skip (routing/degradation.py): Hardcoded skip lists for call sites at L2+ degradation. Change to: emit degradation event with affected call sites, let Genesis decide what to deprioritize. Note: degradation level computation (observability) is fine. Automatic action on it is not.
Design doc language (build-phases.md, autonomous-behavior-design.md): Audit for cost-frequency rules, "above cost threshold = NEVER" language, and budget-as-control framing. Rewrite as observability + user choice.

Availability-driven filtering is fine — if a model is down, you can't call it. That's infrastructure reality, not cost control. CircuitBreaker health checks stay.

Capability Modules Framework (Post-Phase 8, parallel with Phase 9)

External domain capabilities that plug into Genesis without modifying core. Design spec: docs/plans/capability-modules-design.md

Knowledge Pipeline (src/genesis/pipeline/):

Tiered research infrastructure: collect → triage → analyze → judge
Research profiles (YAML) define per-domain signal collection
Pluggable collector registry (web search built-in, extensible)
Integrates with SurplusScheduler for free-tier compute

Module Framework (src/genesis/modules/):

CapabilityModule protocol — pluggable register/deregister lifecycle
ModuleRegistry — manages module loading and runtime wiring
GeneralizationFilter — LLM quality gate between module-local learning and Genesis core. Only process/methodology/calibration lessons cross.

First modules:

prediction_markets/ — MarketScanner, CalibrationEngine (superforecasting), PositionSizer (fractional Kelly), OutcomeTracker (Brier scores)
crypto_ops/ — NarrativeDetector, PositionMonitor (exit signals), CryptoOutcomeTracker (P&L, narrative accuracy)

Design principle: "Hands, not brain" — modules use Genesis cognitive services but don't modify core identity, reflection, or learning. Module outcomes are isolated; only generalizable lessons are promoted through the quality gate.

Tests: 170 tests covering pipeline, framework, and both modules.

V3 Closing Infrastructure (Post-Phase 9)

Before transitioning to V4, spin up:

Automated changelog system — Every code change (by any agent or human) gets a professional changelog entry. Format TBD during implementation, but must be: machine-parseable, human-readable, categorized (feat/fix/refactor/ docs), and attributable (who/what made the change). This becomes V4's audit trail for understanding what changed and why. Design during Phase 9, activate before V4 begins.
Free compute fallback for surplus — When CC rate limits are hit, the surplus system should automatically route lower-priority work through free API alternatives (Gemini, Mistral, Groq free tiers — already cataloged in docs/reference/models.md Free Tier Terms). Not "free CC" — rather, alternative compute the surplus system can use when CC is unavailable. Priority-based: high-priority surplus waits for CC; medium-priority routes to free APIs; low-priority defers to next window. The existing router + free tier infrastructure already supports this — implementation is wiring the surplus system's rate-limit detection to the router's free tier pool. Source: competitive landscape research 2026-03-14.
cognitive_state schema migration — The cognitive_state table has a CHECK constraint limiting section to three values: active_context, pending_actions, state_flags. This was defensive scaffolding from Phase 0. The cognitive loop hierarchy (Phase 2 wiring) stores focus_next in state_flags and focus_next_week in pending_actions — workable but fragile because replace_section() is destructive (deletes all rows for a section). V4's executor needs arbitrary section keys for the living working document. Fix: remove the CHECK constraint (or expand it), and allow freeform section names. The CRUD code (cognitive_state.py) already handles arbitrary section names — only the schema enforces the limit. Migration: ALTER TABLE cognitive_state DROP CONSTRAINT ... or recreate table without CHECK. Low risk, zero code changes beyond schema.
Free model escalation validation (batch 2.5) — Test whether groq, mistral, and openrouter free models reliably produce escalate_to_deep JSON in light reflection output. Run 10-20 light reflections across free providers with signal data that should trigger escalation. If <80% reliability, upgrade light reflection's primary model path to Haiku (still cheap, much more reliable at structured output). If ≥80%, free models are adequate. Document results in docs/reference/free-model-escalation-validation.md. Best done after the system is running live with Phase 2-3 wiring merged.

Codebase Audit Hardening (2026-03-20)

Items identified by the codebase audit that are real issues but were deferred from the audit cleanup branch due to scope, risk, or V4 dependency.

Deferred: `_bootstrapped` partial init guard

File: src/genesis/runtime.py:399
_bootstrapped is set True unconditionally even if subsystems failed. Components checking is_bootstrapped get True when outreach/autonomy/etc may be None.
Why deferred: 10% breakage risk — FindingsBridge, health MCP, and shutdown all check this flag. Needs caller audit before changing.
Correct fix: Set True only if no subsystem has "failed" in manifest. Audit all is_bootstrapped callers for graceful degradation.

Deferred to V4: Hardcoded `/home/ubuntu` paths

Files: .mcp.json, .claude/settings.json, config/genesis-*.service, and 10+ modules using Path(__file__).parent chains.
All config paths, MCP commands, systemd services, and hook scripts assume /home/ubuntu. Migration to another machine = total system failure.
Why V4: Genesis runs on exactly one machine in V3. Proper fix (env vars, GENESIS_ROOT discovery, templated configs) is migration prep work.

Deferred to V4: Injection detection enforcement

File: src/genesis/security/patterns.py
Web content matching injection patterns is log-only — never blocks delivery to the LLM. Content safety hook emits advisory but never exit code 2.
Why V4: False positive risk is real. Blocking requires a review/appeal workflow and confidence-based filtering not yet designed.

Resolved: Items fixed in audit cleanup (2026-03-20)

For reference, the following audit findings were fixed in the fix/audit-cleanup merge:

str(Exception) class bug in job health tracking (3 locations + 3 new methods)
rate_limits.py config path depth (3→4 parents)
OutcomeClassifier SUCCESS→UNKNOWN on LLM failure
DisagreementGate/StateMachine emit signature mismatch (sync→async bridge)
Approval poller task handle lifecycle (store + cancel in shutdown)
ALTER TABLE suppress blocks narrowed to "duplicate column" only (15 blocks)
Path traversal in ProtectedPaths _normalize() (added normpath)
Stuck surplus task recovery (recover_stuck() added)
Content safety hook silent parse failure (added stderr warning)
Telegram error reply suppression → logging (6 critical blocks)
Event bus shutdown suppress → logging
Event DB writer done callback (compensating for circular import)

V4: The Self-Tuning Copilot

Prerequisite: ~1-2 months of V3 operational data.

V4 Phase 0: Survivable Infrastructure (Guardian)

Status: Implemented (2026-03-26) | Branch: feature/guardian

The Guardian is V4's first priority — foundational safety that all other V4 features benefit from. A host VM health monitor running outside the container's blast radius. When Genesis dies, the Guardian detects it, diagnoses root cause via CC, recovers (with user approval), and alerts via Telegram.

Form factor: systemd timer (fires every 30s). Script runs <5s when healthy. On failure detection, extends for the full confirm->diagnose->recover flow.

Key components:

5 health probes + 6 suspicious checks (host-side, stdlib-only)
10-state confirmation state machine with persistence
Self-heal-first dialogue protocol (Genesis fixes itself when possible)
CC-powered diagnosis with prime directive: "First, do no harm"
Without CC: alert only, zero recovery actions (any signal can lie)
Channel-agnostic alerting (Telegram first) with user approval links
Snapshot manager + recovery engine (6 escalation levels)

Deployment: scripts/install_guardian.sh on host VM. See docs/architecture/genesis-v3-survivable-architecture.md for full design.

V4 Feature-Flag Activation

V4 is NOT sequential phases — it's a feature-flag activation model. Features enable independently as V3 data accumulates, because different features need different data volumes.

V4 Activation Model

Each V4 feature has:

Data prerequisite check (programmatic: query tables, count rows, compare minimums)
Config flag (even with sufficient data, user must explicitly enable)
Optional shadow mode: feature runs but outputs go to staging/logs, not production. Allows quality comparison against V3 baseline before going live.

Knowledge autonomy reference: genesis-knowledge-autonomy.md — V4 is where proactive knowledge gap detection, interest pattern recognition, and channel-aware ingestion learning activate. These build on the Phase 8 ingestion infrastructure.

V4 Features

Feature	V3 Data Source	Min Data Volume	Shadow Mode?
Meta-prompting protocol	Simple deep reflection outputs	20+ deep reflections	Yes — compare against static prompt baseline
Strategic reflection (MANAGER/DIRECTOR)	Weeks of full-stack data	4+ weeks operation	Yes — log proposals without acting
Signal weight adaptation	Per-reflection utility tracking	100+ reflection events	Yes — compute adapted weights, compare
Drive weight adaptation	Engagement + outcome data	50+ outcomes, 30+ engagement	Yes — compute adapted weights, compare
Salience threshold self-adjustment	Outreach engagement per topic	20+ engagement per category	Yes — bounded ±20%
Finding/Insight/Opportunity outreach	User model + calibrated engagement	30+ user model evidence	No — live by nature
Outreach frequency growth ramp	Surplus engagement data	20+ outreach w/ >40% engagement	No — live by nature
Channel learning	Per-channel engagement data	10+ outreach per channel per type	No
Speculative hypothesis confirmation	Observation corpus	10+ quarantined hypotheses	No
Procedural memory confidence decay	Procedure corpus with age variance	50+ procedures w/ varying age	Yes — track what would decay
Fresh-eyes review (non-outreach)	High-stakes output corpus	10+ config/identity proposals	Yes — review without blocking
Prompt variation for reflection	Evidence of mode collapse	50+ same-depth reflections	No
ISC (Ideal State Criteria)	Phase 9 hard verification gate + prediction logs	20+ autonomous task completions	Yes — compare ISC-gated vs ungated task quality
Explicit SIGNALS rating capture	Outreach engagement tracking	30+ outreach interactions	No — live by nature. User rates interactions 1-10, low ratings auto-generate steering rules. Feeds calibration system.
Natural language sentiment extraction	Triage pass data + user engagement	50+ triage passes	Yes — classify implicit 1-5 sentiment, compare to explicit SIGNALS ratings. Deferred from Phase 9: no consumer without SIGNALS.
PreToolUse hook (security invariants)	Phase 9 hook foundation + security model	Security model defined	No — requires security model not yet designed
UserPromptSubmit hook (work detection)	Phase 9 hook foundation + conversation data	20+ conversations	No — work item detection pipeline is V4 scope
Proactive context compaction	Phase 9 task execution logs	50+ long-running tasks	Yes — trigger compaction at phase boundaries when >60% context consumed
Theater check (capability verification)	Tool invocation logs	30+ autonomous tasks	Yes — verify capabilities claimed in planning are actually invoked
ISC-annotated task state files	Phase 9 task state files	20+ completed tasks with state files	Yes — add ISC binary criteria to task state, compare gated vs ungated
Context recovery protocol	Compaction + state file maturity	Stable compaction + state files	No — needs both compaction and state file maturity
OPINIONS.md	Reflection + learning corpus	50+ deep reflections	No — graduated opinions from observation patterns
Procedure graduation system	Procedure corpus	50+ procedures with varying confidence	Yes — track confidence decay, compare graduated vs ungradated
Expanded hook coverage	Phase 9 3-hook foundation	Stable hook infrastructure	No — adds PreToolUse, UserPromptSubmit to existing 3 hooks
`/fork` and `/rewind` context management	Session management logs	Stable session infrastructure	No — branch/restore conversation contexts

V4 Research-Driven Features (2026-03-14)

From competitive landscape research. Full specs in docs/plans/v4-research-driven-features-spec.md.

Feature	Inspiration	Dependency
Hot-reload tool discovery	AWS Strands SDK directory-based tools	MCP server modification
AI Functions / runtime capability expansion	AWS Strands Labs AI Functions	Hot-reload tools + Phase 6 maturity
Agent-to-Agent protocol (A2A)	AWS Strands, Google ADK	A2A standard stabilization
API-to-MCP gateway	AWS AgentCore Gateway	Hot-reload tools + OpenAPI parser
Tool Search API (deferred loading)	Anthropic Tool Search Tool	Hot-reload tools + API-routed sessions
Context-efficient CC sessions (sandbox)	Context Mode MCP server	Phase 9 CC hooks + session continuity

V4 Daily Brainstorming Upgrade

V3's daily brainstorming sessions ("upgrade user" + "upgrade self") upgrade from static Light prompts to meta-prompted sessions:

Cheap model (Gemini free) generates brainstorming questions based on recent data, user model, system performance
Capable model (Sonnet) explores the best questions with depth
Synthesis: actionable proposals → staging area

V4 Morning Report Upgrade

V3's static-prompt morning report upgrades to meta-prompted adaptive content selection:

Cheap model asks: "What does the user most need to hear this morning?" based on recent journal, user model, engagement patterns on previous morning reports
Capable model generates the report with adaptive section selection
Morning report engagement data feeds back into content selection model

V4 Self-Assessment Integration

V3's standalone weekly self-assessment becomes an input to Strategic reflection (MANAGER role):

MANAGER cross-references the self-assessment against system metrics
Can propose parameter adjustments (drive weights, salience thresholds) based on assessment findings
Self-assessment trends (improving/declining/stable) inform MANAGER's strategic recommendations

With sufficient free compute, these can expand into ongoing background dialogues — multiple brainstorm rounds per day, each building on previous outputs. The cost-frequency rule still applies: free=always, cheap=often, above threshold=never.

V4 Summary

V4 makes V3 measurably better using operational evidence:

Reflections improve through meta-prompting and prompt variation
Parameters self-tune through calibration loops (bounded, auditable)
Outreach categories expand (findings, insights, opportunities)
Procedures decay without reinforcement (prevents stale knowledge)
Brainstorming sessions become genuinely creative (meta-prompted, multi-round)

V5: The Autonomous Copilot

Prerequisite: ~3-6 months of V4 operational data.

V5 features are qualitatively different from V4's parameter tuning. V4 makes the system better at what it already does. V5 makes the system capable of things it couldn't do before: proposing its own identity changes, anticipating needs, earning genuine autonomy, learning how to learn.

Knowledge autonomy reference: genesis-knowledge-autonomy.md — V5's anticipatory intelligence includes anticipatory knowledge acquisition: Genesis proactively deepens its knowledge in areas the user is trending toward, without being asked.

V5 Features

Feature	V4 Data Required	Min Data Volume	Why V5
L5-L7 autonomy levels	Per-category success evidence	10-20+ successes without correction	Trust requires time
Evidence-based autonomy progression	Success/correction tracking	Statistical significance	Cannot shortcut
Identity evolution (SOUL.md proposals)	Operational maturity	Months of stable operation	Identity is the user's call
Anticipatory intelligence	Rich user model + calibrated drives	3+ months user model data	Design doc: Month 3+
Social simulation ("imagine user reaction")	Deep user model + preferences	100+ user model evidence	Shallow model = bad simulation
Meta-learning ("learn how to learn")	Reflection utility data	200+ observations w/ utility	Need patterns in learning quality
Self-detected error → self-regression	Error pattern tracking	Enough to distinguish systematic	Premature self-regression harms
"Silence ≠ approval" check-ins	Active autonomy tracking	Running progression system	Needs progression first
Relationship rhythm learning	Temporal engagement patterns	3+ months engagement data	Per design doc

V5 Autonomy Levels (New)

L5: System configuration → propose only, user approves
L6: Learning system modification → propose only, always user review
L7: Identity evolution → draft only, user decides

Evidence-based progression: N successful executions without correction per category, explicit user acknowledgment required (silence ≠ approval). Periodic check-in: "I've been handling X autonomously with Y% success rate. Continue?"

V5 Outreach: Anticipatory

The hardest capability. Requires rich user model + calibrated drives + procedural memory. Cross- references new information against user model to identify things the user doesn't know they need. Includes social simulation ("will the user find this useful?") backed by months of engagement data.

V5 Summary

V5 is the "cognitive partner" level — the system that proposes changes to its own identity, anticipates needs before the user knows them, earns and loses autonomy based on evidence, and learns how to learn. This is where the vision document's progression from "aide" to "trusted advisor" to "cognitive extension" becomes real.

Knowledge Base: Parallel Track

Architecturally independent of the cognitive layer. Does NOT depend on V4/V5 data.

KB Phase	Can Start After	Dependency
KB-1: Storage + manual ingestion	V3 Phase 5 (memory-mcp operational)	Multi-collection Qdrant, source_type
KB-2: Distillation pipeline	KB-1	Needs storage working
KB-3: Transcription (audio/video)	KB-1	Needs storage working
KB-4: Acquisition Agent (browser)	KB-2	Needs distillation pipeline

Full design: post-v3-knowledge-pipeline.md in project docs directory.

KB becomes more valuable once V4's meta-prompting can cross-reference KB content during reflections, but KB-1/KB-2 are independently useful for task sub-agents.

Cross-Cutting Concerns (All Versions)

Testing Strategy

Per-phase verification: Each phase has verification criteria (listed above)
Integration tests: Does Phase N correctly use Phase N-1's output?
Regression tests: Does adding Phase N break any Phase N-k behavior?
V4 shadow mode: Compare feature output quality against V3 baseline before activation

Observability

Every phase produces structured logs: what happened, why (triggering signal/score), which model, outcome (success/failure/deferred).

Rollback Plan

Each phase and each V4 feature is independently disableable via config. If Deep reflection produces garbage, disable it and fall back to micro/light. If a V4 feature degrades quality, disable its flag and revert to V3 behavior.

Daily Brainstorm Sessions (Cross-Version)

Version	Session Type	Prompt Style	Model	Frequency
V3	"Upgrade user" + "Upgrade self"	Static Light template	20-30B / Gemini free	2/day minimum
V4	Same + expanded topics	Meta-prompted (3-step)	Multi-model	2/day minimum, more if free compute available
V5	Same + anticipatory	Meta-prompted + user model cross-ref	Multi-model	Autonomous frequency

These sessions are the LAST surplus tasks to skip when compute is constrained. They represent the system's commitment to continuous improvement — even 1 simple brainstorm/day compounds.

Design Doc Cross-Reference

Build Phase	Master Design Doc Section
V3 Phase 0	§4 MCP Servers, §Execution Trace Schema, §Procedural Memory Design
V3 Phase 1	§Layer 1: Awareness Loop, §Signal-Weighted Trigger System, §Three Categories of Scheduled Work
V3 Phase 2	§LLM Weakness Compensation → Pattern 1: Compute Hierarchy
V3 Phase 3	§Cognitive Surplus
V3 Phase 4	§Layer 2: Reflection Engine → Depth Levels (Micro, Light), §Pre-Execution Assessment, §Context Allocation by Model Tier, §Cognitive State Summary
V3 Phase 5	§Memory Separation, §What We Learned (A-MEM, ACT-R gaps), §Open Questions and Persistent Curiosity
V3 Phase 6	§Layer 3: Self-Learning Loop, §Procedural Memory, §LLM Weakness → Pattern 6, §Signal Weight Tiers
V3 Phase 7	§Reflection Engine (Deep), §Cognitive State Summary (regeneration), §Weekly Self-Assessment, §Quality Calibration Cycle, current Dream Cycle jobs
V3 Phase 8	§Proactive Outreach, §Daily Morning Report, §Bootstrap / Cold Start Strategy
V3 Phase 9	§Self-Evolving Learning: The Autonomy Hierarchy (L1-L4 only)
V4	§LLM Weakness → Patterns 2-5, §Loop Taxonomy → Tier 3
V5	§Autonomy Hierarchy (L5-L7), §Loop Taxonomy → Tier 4
KB	`post-v3-knowledge-pipeline.md`

Pre-V4 Follow-Up Items (V3 complete, not yet wired)

These components were built as Phase 9 infrastructure but intentionally deferred from V3's "Basic Autonomy" scope. They need wiring before V4.

Task Autonomy System (TAS)

ApprovalManager — request_approval() has zero call sites
TaskVerifier — initialized with code validator but never called
AutonomyManager — initialized but never accessed externally
Wire into: surplus executor, outreach pipeline, task completion paths

True E2E Cognitive Loop Test

Current tests/integration/test_cognitive_loop_e2e.py covers segment only
Need full: Awareness → Perception → Memory → Learning → Reflection → Outreach

GROUNDWORK Tag Registry (as of 2026-03-22)

Tag	File	Target Version
`v4-executor`	`cc/reflection_bridge/_bridge.py`	V4
`cross-vendor-review`	`autonomy/disagreement.py`	V4
`outreach-voice`	`outreach/pipeline.py`	V4
`observation-feedback-loop`	`db/crud/observations.py`	V4
`multi-person`	`db/schema/_tables.py` (8 tables)	V4+
`v4-parallel-dispatch`	`surplus/scheduler.py`	V4
`v4-surplus-tasks`	`surplus/types.py`	V4
`v4-rate-tracking`	`surplus/compute_availability.py`	V4
`skill-autonomy-graduation`	`learning/skills/applicator.py`	V4
`user-model-synthesis`	`perception/writer.py`	V4
`mcp-config`	`cc/session_config.py`	V4
`hook-inheritance`	`cc/session_config.py`	V4
`category-2-rhythms`	`learning/`	V4
`category-3-crons`	`learning/`	V4
`outreach-alerts`	`recon/cc_update_analyzer.py`	V4
`outreach-pipeline`	`outreach/pipeline.py`	V4
`provider-migration`	`learning/tool_discovery.py`	V4
`unified-bridge`	`channels/`	V4
`V4`	various	V4
`pre-execution-gate`	`autonomy/`	V4

FilesExpand file tree

genesis-v3-build-phases.md

Latest commit

History

genesis-v3-build-phases.md

File metadata and controls

Genesis: Versioned Build Plan — Safety-Ordered, Waterfall

Version Overview

V3: The Complete Copilot

V3 Phase Dependency Map

V3 Phase 0: Data Foundation ✅ COMPLETE

What to Build

Workspace Files

MCP Server Interfaces

Verification

V3 Phase 1: The Metronome (Awareness Loop) ✅ COMPLETE

What to Build

Verification

V3 Phase 2: Compute Routing ✅ COMPLETE

What AZ Already Provides

What to Build

Compute Hierarchy

3B Model Constraints

Verification

V3 Phase 3: Surplus Infrastructure ✅ COMPLETE

What to Build

Why This Phase Is Early (Phase 3, Not Phase 6)

Verification

V3 Phase 4: Perception (Micro/Light Reflection)

Known Gap: Signal Collectors (as of 2026-03-17)

What to Build

Deferred to Phase 4b (Surplus Wiring)

Deferred: Observation Sweep (#28)

Verification

Testing Strategy for LLM-Dependent Components

V3 Phase 5: Memory Operations

What to Build

AZ Memory System Internals (Discovered 2026-03-04)

Verification

V3 Phase 6: Learning Fundamentals

What to Build

Adaptive Obstacle Resolution (Procedural)

Tool Capability Discovery

Sub-Agent Memory Harvesting

Inbox Monitor Service

Skill Wiring for Genesis Sessions

Perception Template Externalization

Phase 4 Follow-Up: Reflection Template Expansion

Critical Risk

Verification

V3 Phase 7: Simple Deep Reflection ✅ COMPLETE

What to Build

Why This Exists (Not Just Micro/Light)

Verification

V3 Phase 8: Basic Outreach ✅ COMPLETE

What to Build

Approval Dashboard UI

Identity & Prompt File Audit Page

CC Chat Widget

AZ UI Architecture Findings (Discovered 2026-03-04)

Outreach Timing: Why Day 1

Bayesian Calibration Infrastructure (AGI Gap Items 1-3)

Verification

V3 Phase 9: Basic Autonomy ✅ COMPLETE

What to Build

Calibration-Informed Autonomy (AGI Gap Items 4-6)

Approval Timeout & Auto-Reject Policy

PAI-Informed Additions (from Miessler deep dive, 2026-03-12)

Verification

CC Update Compatibility (Cross-Cutting)

V3 Summary

Retroactive Fix: Cost-as-Observability (Phases 0-3)

Capability Modules Framework (Post-Phase 8, parallel with Phase 9)

V3 Closing Infrastructure (Post-Phase 9)

Codebase Audit Hardening (2026-03-20)

Deferred: _bootstrapped partial init guard

Deferred to V4: Hardcoded /home/ubuntu paths

Deferred to V4: Injection detection enforcement

Resolved: Items fixed in audit cleanup (2026-03-20)

V4: The Self-Tuning Copilot

Deferred: `_bootstrapped` partial init guard

Deferred to V4: Hardcoded `/home/ubuntu` paths