MACROS.md

Macro System

Expands on the Macro System section of TECHNICAL_CONCEPT.md.

What Macros Are

Macros are compiled execution subgraphs that capture whatever repeated — tool call sequences, reasoning hints, plan artifacts, or any combination thereof — as deterministic conditional logic.

A macro is not a reasoning engine. It does not yield, pause, or resume into general reasoning. It executes to completion as a deterministic piece of the execution graph.

The macro does not mandate what is captured. The discovery pipeline detects the repeating subgraph in execution traces, and the macro compiles it as-is.

Macros have two discovery sources:

• Pattern-derived — discovered from general execution traces through sliding window mining and subgraph matching.
• Skill-derived — discovered from skill execution traces, carrying provenance metadata linking them to their source skill and version. Skills are optional; pattern-derived macros are discovered regardless of skill presence.

Both sources feed the same compilation, validation, and execution pipeline. The only difference is that skill-derived macros carry provenance links that affect routing and demotion behavior.

Storage. Macro definitions are stored in the artifact version store as versioned entities. Macro execution traces are recorded in the world-state event store. Cross-store references link execution events to the macro version that produced them.

Captured Content

A macro captures whatever the discovery pipeline detects as the repeating subgraph. The captured content exists on a spectrum:

Level	Captured Content	When It Occurs
Minimal	Tool calls only	The repetition is purely sequential tool calls with no branching and no plan generation
Common	Tool calls + reasoning hints	Branch points where reasoning occurred are part of the repeating pattern
Full	Tool calls + reasoning hints + plan artifact	The entire planning-first execution chain repeated, including plan generation

The macro compiles the detected pattern as-is. It does not add or remove content based on a predetermined structure.

Plan Artifact Capture

When the repeating subgraph includes a planning-first execution chain, the plan artifact generated by the RPU is captured as part of the macro:

{
	"capturedPlan": {
		"objective": "Morning briefing for user",
		"steps": [
			{ "id": 1, "action": "fetch_weather", "description": "Get current forecast" },
			{ "id": 2, "action": "summarize_calendar", "description": "Today's events" },
			{ "id": 3, "action": "fetch_news", "description": "Topics of interest" }
		]
	}
}

The captured plan serves three purposes when present:

1. Observability — the user sees the macro's intent at the plan level, not just the tool-call level. Even though the macro executes deterministically, the plan provides a human-readable summary of what is happening.
2. Escalation context — when the macro needs RPU reasoning (novel situation, child macro failure, error escalation), the captured plan provides rich intent context for the RPU to reason about. The RPU receives the plan alongside recent hints and tool results.
3. Intent trace — preserves the original reasoning intent even in deterministic execution. The plan answers "what was this macro trying to achieve" without requiring expansion back to the original events.

Plan artifact capture is contingent on the detected pattern. If the repetition is at the tool-call level, the macro captures tool calls (and hints if present) without a plan artifact. If the repetition includes the full planning-first chain, the plan artifact is captured alongside everything else. This is not a deficiency — the macro faithfully compiles what repeated.

Reasoning Hints vs. captured plan artifacts. These are complementary: Reasoning Hints capture branch-level reasoning outcomes ("why this decision"), while the captured plan artifact captures the chain-level intent structure ("what we're doing"). Both are preserved in the macro when the repeating subgraph includes them.

Reasoning Hints

A Reasoning Hint is a distilled textual annotation written by the model during macro compilation. It captures the outcome of a reasoning step at a branch point, not the full chain-of-thought that produced it.

Example from a compiled macro:

If budget >= item.cost:
  Hint: "Budget allows for this item, proceeding with purchase"
  Tool: purchase(item)

If budget < item.cost:
  Hint: "Budget insufficient for this item, skipping"
  Tool: skip(item)

Hints serve three purposes:

1. Provenance — when auditing a macro execution, the hint chain explains why each branch was taken without reconstructing the original reasoning.
2. Context injection — hints and tool results from recent macro execution form a sliding window passed to the RPU when the kernel invokes it for new reasoning.
3. Future learning — the hint chain gives future macro discovery passes semantic context to work with, not just tool-call sequences.

Properties

Distilled — a hint captures the outcome of reasoning, not the reasoning process itself. It is a concise explanation of why a decision was made.

Parameterized — concrete values from the original execution become variables, allowing the same macro to generalize across different situations.

If distance <= {max_distance}:
  Hint: "Target within {max_distance} km, using local search"
  Tool: search_local(target)

Immutable — once a macro is validated, its hints become part of the macro's commit range. They are not regenerated during execution.

Expandable — a macro always preserves the link to its originating events. No behavior is hidden behind abstraction.

Contingent — reasoning hints are captured only when the repeating subgraph includes branch points where reasoning occurred. If the repetition is purely sequential tool calls with no branching, the macro has no hints. This is not a deficiency — the macro faithfully compiles what repeated.

Discovery Mechanisms

Sliding Window Mining

The system scans execution traces through a sliding time window (configurable, default 24 hours). Within each window, it identifies repeated sequences of tool calls, RPU invocations, and scheduler decisions. The window slides forward in increments (default 1 hour) to catch patterns that span window boundaries.

Normalization is applied before comparison: variable identifiers are replaced with type placeholders, timestamps are converted to relative offsets, and confidence scores are bucketed into ranges. This prevents superficial differences from masking structural similarity.

Subgraph Matching

Execution traces are represented as directed graphs where nodes are actions (tool calls, RPU invocations, state transitions) and edges are causal dependencies. The mining algorithm searches for frequently occurring subgraphs using a variant of the gSpan algorithm adapted for execution traces.

Key adaptations:

• Nodes carry typed payloads (tool name, RPU function, state update type) rather than simple labels
• Edges carry temporal offsets and priority context
• Subgraph frequency is weighted by recency — patterns that occurred recently are more valuable than patterns from months ago
• Semantic equivalence is checked at the payload level, not just the graph structure

Pattern Normalization and Hint Extraction

Before a pattern can be promoted, it must be normalized into a reusable form:

1. Concrete values are replaced with parameters
2. Fixed tool references are generalized to service abstractions
3. Hardcoded thresholds become configurable parameters
4. Branch conditions are extracted as decision points
5. The model distills the reasoning at each branch point into a Reasoning Hint

This normalization is assisted by LLM analysis of the trace patterns, but the resulting macro is validated deterministically.

Skill-Derived Macros

When skills are installed and executed, their execution traces enter the event graph like any other behavior. The discovery pipeline analyzes these traces for repeated patterns. When a pattern is detected, a skill-derived macro is proposed with provenance metadata. Skills are optional — pattern-derived macros are discovered from general execution regardless of skill presence.

Provenance Metadata

Every skill-derived macro carries a provenance block:

{
	"provenance": {
		"source_type": "skill",
		"source_id": "skill_weather_check",
		"source_version": "1.2.0",
		"discovered_at": "2024-03-15T08:00:00Z",
		"validated_at": "2024-03-20T03:00:00Z",
		"execution_count": 47
	}
}

The provenance block is immutable once the macro is validated. It links the macro to its authoritative source, enabling automatic invalidation when the source changes. Skill version references in provenance metadata enable sensible replay of memories and execution traces tied to skill execution. When a skill is updated, all dependent artifacts (macros, knowledge entries) are invalidated through the existing demotion mechanics — no special cross-reference resolution is needed.

Routing Decision Tree

When a request matches a skill, the kernel executes the following routing logic:

Request matches skill S (version V)
    ↓
Query: macro M where provenance.source_id == S.id AND provenance.source_version == V
    ↓
If M exists AND M.status == "promoted" AND M.confidence > threshold:
    → Execute M (fast path, deterministic)
    → Record execution trace (feeds discovery pipeline)
Else:
    → Execute S via RPU (normal reasoning path)
    → Record execution trace (feeds discovery pipeline)
    → Discovery pipeline continues mining for patterns

The routing decision is deterministic — no RPU invocation is needed to choose between macro and skill. The kernel performs a hash-table lookup on the macro graph.

Skill Version Updates

When a skill is updated (new version published), all derived macros are affected:

1. Detection. The kernel scans the macro graph for all macros with provenance.source_id matching the updated skill and provenance.source_version not matching the new version.
2. Flagging. Matching macros are flagged with status source_updated. They are excluded from the scheduler's macro lookup.
3. Re-validation. The offline optimization loop re-validates flagged macros against execution traces from the new skill version. If the new skill produces the same behavioral patterns, a new macro version is proposed with updated provenance.
4. Fallback. While the macro is source_updated, the skill executes via RPU. This is the same behavior as Day 1 — the system falls back to full reasoning until the new macro is validated.

This process uses the existing demotion mechanics — "source skill version changed" is a first-class demotion trigger alongside "usage decay," "primitive change," and "failure rate increase."

Knowledge Derived from Skills

Knowledge entries extracted from skill execution traces also carry provenance metadata. When a skill is updated:

1. Knowledge entries with provenance to the old version enter accelerated confidence decay — they are not immediately deleted, but they stop receiving corroboration from new executions.
2. If the new skill produces the same factual patterns, new knowledge entries are created with provenance to the new version.
3. The old knowledge entries decay naturally; the new ones accumulate confidence. There is no discontinuity — just a smooth transition.

This is the same mechanism as knowledge's existing temporal validity and confidence decay. Skill provenance adds another decay trigger: "fact's source changed."

Hierarchical Macros

Macros can reference other macros as child subgraph components. Since the execution model is event-driven (not call-driven) and the world-state graph is a DAG by construction, hierarchical macros inherit the acyclic property — circular references are impossible.

Composition Model

A hierarchical macro is a logical grouping of subgraph references, not a runtime call stack. At execution time, child macros are inlined into the parent's execution sequence. The hierarchy is a design-time abstraction for discovery partitioning and maintenance, not a runtime structure.

Macro: morning_routine
├── Macro: weather_check (child subgraph reference)
│   └── Tool: fetch_weather
├── Macro: calendar_briefing (child subgraph reference)
│   └── Tool: fetch_calendar
└── Macro: news_summary (child subgraph reference)
    └── Tool: fetch_news

At execution time, this is equivalent to a flat sequence of tool calls with conditional logic — the "macro calls" are just subgraph inlining. The hierarchy exists for:

• Discovery partitioning — leaf macros are discovered first (higher frequency), then composition patterns are detected.
• Reusability — the same weather_check macro can be a child of morning_routine and evening_routine.
• Maintenance — when weather_check is updated, all parent macros automatically use the new version.
• Auditability — the provenance chain traces through the hierarchy: parent → child → tool call.

Discovery Paths

Bottom-up composition. Leaf macros (tool-call patterns) are discovered first because they appear more frequently in execution traces. As leaf macros accumulate, the scheduler begins invoking them in sequence repeatedly. The discovery pipeline detects these invocation sequences as higher-level patterns and proposes parent macros that compose the existing leaf macros.

Top-down decomposition. A flat macro is discovered first (the entire morning routine as one subgraph). During normalization, the system identifies sub-patterns within the macro that are reusable across other contexts and extracts them as child macros. The parent macro is rewritten to reference the children. This is analogous to "extract function" refactoring in software engineering.

Both paths are supported. Bottom-up is the natural progression of the discovery algorithm. Top-down is useful when a flat macro is discovered first but later benefits from decomposition.

Parameter Passing

Parent macros pass parameters to child macros through a binding syntax:

Binding Type	Example	Resolves To
Literal	"location": "Boston"	The literal value
Parent parameter	"location": "{parent.location}"	The parent macro's parameter
User context	"location": "{user_home_location}"	A value from the knowledge artifacts
Computed	"radius": "{parent.max_distance / 2}"	A computed value (if arithmetic is supported)

Parameter binding is resolved at macro execution time, before the child subgraph is inlined.

Error Propagation

When a child macro fails, the parent's recovery strategy is captured in a Reasoning Hint at the decision point:

If child_macro.weather_check fails:
    Hint: "Weather service unavailable, skipping weather in morning briefing"
    Action: skip(weather_check)
    Continue: calendar_briefing, news_summary

Strategies:

• Abort — stop the entire parent macro. Used when the child's output is critical to the parent's intent.
• Skip — continue without the failed child's output. Used when the child is optional.
• Fallback — use a default value or alternative tool call. Used when a reasonable default exists.
• Escalate — invoke the RPU for reasoning about how to proceed. Used when the situation is novel and no deterministic strategy applies.

No Depth Limits

Macros are always expandable to their full event ranges. Like folders in a tree view of a file system, they can be recursively expanded to reveal the complete execution history. There is no architectural depth limit because:

• Macros replace one-to-one the reasoning + tool chain with hint + tool chain.
• They are seen as git ranges — any macro can be expanded to its full version with textual hints and tool outputs.
• Execution history for a specific event is a single monolithic block; macros are just a viewing layer on top of that block.
• Tracing events is a UI concern, not an architectural constraint. An "expand all" button reveals the full chain.

The provenance chain from parent macro through child macros to leaf tool calls is always preserved and always accessible.

Demotion Propagation

When a child macro is demoted, the parent macro evaluates the impact:

• If the child is critical to the parent's intent (no fallback exists), the parent is also demoted.
• If the child is optional (skip strategy applies), the parent continues with the skip behavior.
• If the child has a fallback (alternative tool or default value), the parent is rewritten to use the fallback.

The decision is encoded in the parent macro's error propagation hints.

Context Window Integration

When a macro completes and the kernel decides to invoke the RPU for new reasoning, the RPU receives a sliding window of recent execution context. This window contains the most recent hint/tool-result pairs from the macro's execution, and — when the macro captured a plan artifact — the captured plan:

interface RPURequest {
	function: string;
	objective: string;
	personality: PersonalityState;
	worldState: WorldState;
	context: ContextProjection;
	capturedPlan?: Plan; // Present only if the macro captured a plan artifact
	recentExecution: {
		hint: string;
		toolResult: unknown;
	}[];
	taskState?: TaskState;
	previousArtifacts?: Artifact[];
}

This gives the RPU local awareness of what just happened and why, alongside targeted memory artifacts and validated knowledge artifacts. When the captured plan is present, it provides rich intent context for the RPU to reason about. The window is bounded so it doesn't inflate context size unnecessarily — only recent decisions matter for immediate next-step reasoning.

Validation Test Design

A proposed macro is validated against historical execution data. The test process:

1. Replay — the macro is executed against the same input traces that originally produced the pattern
2. Equivalence check — the macro's output is compared to the original execution output at the semantic level (not byte-for-byte, but functionally equivalent)
3. Edge case testing — the macro is tested against variations of the original traces: missing inputs, different tool responses, priority conflicts
4. Performance measurement — the macro's resource consumption is measured against the original execution to confirm cost savings

A macro passes validation only when it produces semantically equivalent results across all test traces and demonstrates measurable efficiency gains.

Plan Artifact Validation (Contingent)

When the macro captured a plan artifact (because the repeating subgraph included the planning-first chain), an additional validation step applies:

5. Plan equivalence — the macro's captured plan must be consistent with the original execution's plan artifact. The macro may not deviate from the plan's stated objectives. This check ensures that the deterministic execution still aligns with the original intent structure.

If the macro did not capture a plan artifact (the repetition was at the tool-call level), this step is skipped.

Skill-Derived Macro Validation

For skill-derived macros, an additional validation step applies:

5. Provenance equivalence — the macro's behavior is compared against the source skill's intent, not just the observed traces. The skill's instruction file defines the expected behavior; the macro must preserve this intent across parameterized variations. This check ensures that the macro has not overfitted to a specific execution trace but genuinely captures the skill's general behavior.

Hierarchical Macro Validation

For hierarchical macros, validation operates at two levels:

• Child validation — each child macro is validated independently against its own historical traces. A child macro must pass validation before it can be referenced by a parent.
• Composition validation — the parent macro's composition logic (parameter binding, error propagation, sequencing) is validated against traces where the child macros were invoked together. The parent does not re-validate the children's internal behavior; it validates that the composition produces correct results given correct children.

Demotion Mechanics

A macro can be demoted for several reasons:

• Usage decay — the macro is not invoked for a configurable period (default 30 days)
• Primitive change — an underlying tool or service that the macro depends on has changed its contract
• Failure rate increase — the macro's success rate drops below a threshold due to environmental changes
• Better replacement — a more general macro subsumes the functionality of the existing macro
• Source skill version changed (skill-derived only) — the skill the macro was derived from has been updated; the macro is flagged for re-validation against the new version
• Child macro demoted (hierarchical only) — a child macro has been demoted; the parent evaluates whether to demote, skip, or fallback based on its error propagation hints

Demotion is reversible. A demoted macro remains in the macro graph with a demoted status. It can be re-promoted if conditions change. The full provenance chain — from original trace events through macro proposal, validation, compilation, and demotion — is preserved.

Demoted macros are excluded from the scheduler's macro lookup but remain accessible for audit and re-evaluation.

Skill Version Update Flow

When a skill is updated, the demotion flow for derived artifacts is:

1. Detection. The kernel scans the macro graph for all macros with provenance.source_id matching the updated skill and provenance.source_version not matching the new version.
2. Flagging. Matching macros are set to status source_updated. They are excluded from the scheduler's macro lookup immediately.
3. Fallback. Requests that previously matched the demoted macro now route to the skill (RPU execution path). This is the same behavior as Day 1 — full reasoning, no compression.
4. Re-validation. The offline optimization loop re-validates the flagged macro against execution traces from the new skill version. If the new skill produces the same behavioral patterns, a new macro version is proposed with updated provenance (source_version bumped to the new version).
5. Promotion or retirement. If re-validation succeeds, the new macro version is promoted. If the new skill's behavior differs significantly, the old macro is retired (status: superseded) and a new macro is discovered from scratch.

Knowledge entries derived from the old skill version follow a parallel path: they enter accelerated confidence decay (faster than normal decay, slower than immediate deletion). If the new skill produces the same factual patterns, new knowledge entries are created with provenance to the new version. The transition is smooth — no discontinuity, just a confidence curve.