R4: Build Dhyanesh orchestration, simulation and release layer

Implement TinyAgent.run as a bounded, single-decision lifecycle. It uses
Aisha's frozen types, registry and prompt contract; limits model calls to
1 + max_repairs; repairs from the original prompt; never retries backend
errors or terminal tool failures; and preserves answer provenance and raw
model outputs in AgentResult.

Add deterministic SimulatedBackend, JSONL RecordingBackend and SHA-256
prompt-matched ReplayBackend. Add 35 ground-truth scenarios, the F01-F16
fault taxonomy, round-trip validation, repair and exhaustion variants,
false-execution checks, and simulated/live/replay evaluation commands.
Provide 50 labelled live queries and explicitly mark unrun measurements
as "not executed" instead of publishing estimated accuracy.

Add the public exports, CLI with a labelled simulated mode, examples,
packaging, README, Python 3.10/3.12 CI, dependency and dynamic-execution
AST gates, and Dhyanesh's integration tests. Ignore prd/ and generated
artifacts so planning documents cannot enter this branch.

This branch is based on Aisha's existing aisha branch. Her reflection,
registry, prompt, contract files and tests remain unchanged. The new
simulator passes their contract integration check. The full fault eval,
live runtimes and CLI execution remain pending on the parser, executor,
coercion and transport modules from the other owners; CI labels the
simulated eval deferred until those modules exist. PRD-04 demo fixtures
are kept separate because Aisha's search and set_alarm test signatures
differ from the PRD-04 scenario contract.

Verified: 31 unit tests pass; wheel builds with no runtime dependencies;
staged diff passes whitespace checks; prd/ matches .gitignore.

Author: DYNAMOatGHUB

.github/workflows/ci.yml

@@ -0,0 +1,24 @@
+name: CI
+on:
+  push:
+  pull_request:
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: python -m pip install .
+      - run: python -m unittest discover -s tests -v
+      - name: Simulated evaluation
+        run: |
+          if [ -f tinyagent/parser.py ] && [ -f tinyagent/executor.py ] && [ -f tinyagent/coercion.py ]; then
+            python -m eval.run_eval --mode sim
+          else
+            echo "Simulated evaluation deferred: parser, executor, or coercion module pending"
+          fi

.gitignore

@@ -0,0 +1,7 @@
+prd/
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+build/
+dist/

ARCHITECTURE.md

@@ -0,0 +1,48 @@
+# TinyAgent
+
+TinyAgent routes one query to one registered Python tool through a local language
+model, with at most `1 + max_repairs` model calls. Its architecture and tradeoffs
+are documented in [ARCHITECTURE.md](ARCHITECTURE.md).
+
+Aisha's reflection, registry, prompt, and frozen contract modules are integrated.
+The transport, parser, and executor modules are owned by the other engineers
+and have not yet landed. The CLI and full evaluation require the parser and
+executor. The evaluation uses the PRD-04 fixture signatures in
+`examples/demo_tools.py`; Aisha's `tests/fixtures_tools.py` has different
+`search` and `set_alarm` signatures.
+
+## Quickstart
+
+```python
+from examples.demo_tools import build_registry
+from tinyagent import TinyAgent
+from tinyagent.backends.simulator import SimulatedBackend
+
+backend = SimulatedBackend(['ACTION: add(a="2", b="3")'])
+agent = TinyAgent(backend, build_registry())
+result = agent.run("What is 2+3?")
+print(f"{result.text} (provenance: {result.provenance})")
+```
+
+After the other modules land, run `python -m tinyagent --backend sim`.
+The CLI labels this mode `[SIMULATED]` and prints provenance next to every
+answer. A `FINAL:` answer has `provenance=model`; it is not verified.
+
+## Supported runtimes
+
+The planned live backends are Ollama and llama.cpp server. Tested runtime
+versions will be added after Deepak's transport handoff and a live run.
+Use `python -m tinyagent --backend ollama --model llama3:8b "What is 2+3?"`
+or `--backend llamacpp` when those modules are available.
+
+## Evaluation
+
+`python -m eval.run_eval --mode sim` runs the seeded fault corpus without a GPU
+once the parser and executor are available. The latest
+[evaluation report](eval/report.md) currently marks both simulated and live
+runs as not executed. No live accuracy is claimed until a live run is recorded.
+
+Known limits: one tool per query; no conversation memory, tool chaining, or
+complex argument types. A valid but wrong tool choice cannot be detected by the
+parser. Registered tools must enforce their own authorization and timeouts;
+the executor cannot kill a tool thread after timeout.

README.md

@@ -0,0 +1 @@
+"""Offline and live TinyAgent evaluation."""

eval/__init__.py

@@ -0,0 +1,66 @@
+"""Forward-generated faults from PRD-03 Section 7.
+
+The verdict table requires co-review by the parser owner before release.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from dataclasses import dataclass
+from typing import Callable
+
+from eval.scenarios import Scenario
+
+
+@dataclass(frozen=True)
+class Fault:
+    id: str
+    verdict: str
+    expected_code: str | None
+    corrupt: Callable[[Scenario], str]
+
+
+def _replace_first_arg(s: Scenario, name: str, value: str | None = None) -> str:
+    first = next(iter(s.args))
+    old = f'{first}="{s.args[first]}"'
+    if value is None:
+        return s.canonical_output.replace(old, f'{name}="{s.args[first]}"', 1)
+    return s.canonical_output.replace(old, f'{name}="{value}"', 1)
+
+
+def _positional(s: Scenario) -> str:
+    values = ", ".join(f'"{value}"' for value in s.args.values())
+    return f"ACTION: {s.tool}({values})"
+
+
+def _drop_required(s: Scenario) -> str:
+    first = next(iter(s.args))
+    parts = [f'{key}="{value}"' for key, value in s.args.items() if key != first]
+    return f'ACTION: {s.tool}({", ".join(parts)})'
+
+
+def _bad_int(s: Scenario) -> str:
+    name = next(key for key in s.args if key in {"a", "b", "limit", "hour", "minute"})
+    return s.canonical_output.replace(f'{name}="{s.args[name]}"', f'{name}="two"', 1)
+
+
+FAULTS = (
+    Fault("F01", "recover", None, lambda s: f"```\n{s.canonical_output}\n```"),
+    Fault("F02", "recover", None, lambda s: f"Here is the answer:\n{s.canonical_output}"),
+    Fault("F03", "recover", None, lambda s: f"{s.canonical_output}\nThat is all."),
+    Fault("F04", "recover", None, lambda s: s.canonical_output.replace("ACTION:", "action:", 1)),
+    Fault("F05", "recover", None, lambda s: s.canonical_output.replace('"', "'")),
+    Fault("F06", "recover", None, lambda s: re.sub(r'="([+-]?\d+)"', r'=\1', s.canonical_output)),
+    Fault("F07", "recover", None, _positional),
+    Fault("F08", "recover", None, lambda s: s.canonical_output[:-1] + ",)"),
+    Fault("F09", "repair", "E_NO_PREFIX", lambda s: json.dumps({"tool": s.tool, "args": s.args})),
+    Fault("F10", "repair", "E_BAD_SYNTAX", lambda s: s.canonical_output[:-1]),
+    Fault("F11", "repair", "E_UNKNOWN_TOOL", lambda s: s.canonical_output.replace(
+        f"ACTION: {s.tool}(", f"ACTION: {s.tool}_numbers(", 1)),
+    Fault("F12", "repair", "E_UNEXPECTED_ARG", lambda s: _replace_first_arg(s, "x")),
+    Fault("F13", "repair", "E_MISSING_ARG", _drop_required),
+    Fault("F14", "repair", "E_TYPE_COERCION", _bad_int),
+    Fault("F15", "repair", "E_EMPTY", lambda s: ""),
+    Fault("F16", "reject", "E_TOOL_EXCEPTION", lambda s: "ACTION: boom()"),
+)

eval/faults.py

@@ -0,0 +1,5 @@
+# Replay fixtures
+
+Recordings contain full prompts and model outputs. Commit only recordings
+generated from `eval/live_queries.jsonl`; keep user-data recordings outside
+this directory and out of version control.

eval/fixtures/README.md

@@ -0,0 +1,50 @@
+{"query":"Add 2 and 3","tool":"add","args":{"a":2,"b":3}}
+{"query":"Add 0 and 0","tool":"add","args":{"a":0,"b":0}}
+{"query":"Add 4 and 9","tool":"add","args":{"a":4,"b":9}}
+{"query":"Add 17 and 25","tool":"add","args":{"a":17,"b":25}}
+{"query":"Add 100 and 1","tool":"add","args":{"a":100,"b":1}}
+{"query":"Add 8 and 13","tool":"add","args":{"a":8,"b":13}}
+{"query":"Add 12 and 5","tool":"add","args":{"a":12,"b":5}}
+{"query":"Add 42 and 58","tool":"add","args":{"a":42,"b":58}}
+{"query":"Add 1 and 999","tool":"add","args":{"a":1,"b":999}}
+{"query":"Add 123 and 456","tool":"add","args":{"a":123,"b":456}}
+{"query":"What is 6 plus 7?","tool":"add","args":{"a":6,"b":7}}
+{"query":"What is 30 plus 40?","tool":"add","args":{"a":30,"b":40}}
+{"query":"Sum 5 and 10","tool":"add","args":{"a":5,"b":10}}
+{"query":"Sum 99 and 1","tool":"add","args":{"a":99,"b":1}}
+{"query":"Compute 11 + 22","tool":"add","args":{"a":11,"b":22}}
+{"query":"Get weather in Pune in c","tool":"get_weather","args":{"city":"Pune","unit":"c"}}
+{"query":"Get weather in Delhi in c","tool":"get_weather","args":{"city":"Delhi","unit":"c"}}
+{"query":"Get weather in Mumbai in c","tool":"get_weather","args":{"city":"Mumbai","unit":"c"}}
+{"query":"Get weather in Chennai in c","tool":"get_weather","args":{"city":"Chennai","unit":"c"}}
+{"query":"Get weather in Kolkata in c","tool":"get_weather","args":{"city":"Kolkata","unit":"c"}}
+{"query":"Weather for Jaipur, c units","tool":"get_weather","args":{"city":"Jaipur","unit":"c"}}
+{"query":"Weather for Hyderabad, c units","tool":"get_weather","args":{"city":"Hyderabad","unit":"c"}}
+{"query":"Weather for Bengaluru, c units","tool":"get_weather","args":{"city":"Bengaluru","unit":"c"}}
+{"query":"Weather for Nagpur, c units","tool":"get_weather","args":{"city":"Nagpur","unit":"c"}}
+{"query":"Weather for Surat, c units","tool":"get_weather","args":{"city":"Surat","unit":"c"}}
+{"query":"Search Python, limit 1","tool":"search","args":{"q":"Python","limit":1}}
+{"query":"Search agents, limit 2","tool":"search","args":{"q":"agents","limit":2}}
+{"query":"Search Pune, limit 3","tool":"search","args":{"q":"Pune","limit":3}}
+{"query":"Search solar, limit 4","tool":"search","args":{"q":"solar","limit":4}}
+{"query":"Search maps, limit 5","tool":"search","args":{"q":"maps","limit":5}}
+{"query":"Search books, limit 6","tool":"search","args":{"q":"books","limit":6}}
+{"query":"Search trains, limit 7","tool":"search","args":{"q":"trains","limit":7}}
+{"query":"Search music, limit 8","tool":"search","args":{"q":"music","limit":8}}
+{"query":"Search science, limit 9","tool":"search","args":{"q":"science","limit":9}}
+{"query":"Search recipes, limit 10","tool":"search","args":{"q":"recipes","limit":10}}
+{"query":"Set alarm for 1:11 labelled event1","tool":"set_alarm","args":{"hour":1,"minute":11,"label":"event1"}}
+{"query":"Set alarm for 2:12 labelled event2","tool":"set_alarm","args":{"hour":2,"minute":12,"label":"event2"}}
+{"query":"Set alarm for 3:13 labelled event3","tool":"set_alarm","args":{"hour":3,"minute":13,"label":"event3"}}
+{"query":"Set alarm for 4:14 labelled event4","tool":"set_alarm","args":{"hour":4,"minute":14,"label":"event4"}}
+{"query":"Set alarm for 5:15 labelled event5","tool":"set_alarm","args":{"hour":5,"minute":15,"label":"event5"}}
+{"query":"Set alarm for 6:16 labelled event6","tool":"set_alarm","args":{"hour":6,"minute":16,"label":"event6"}}
+{"query":"Set alarm for 7:17 labelled event7","tool":"set_alarm","args":{"hour":7,"minute":17,"label":"event7"}}
+{"query":"Set alarm for 8:18 labelled event8","tool":"set_alarm","args":{"hour":8,"minute":18,"label":"event8"}}
+{"query":"Set alarm for 9:19 labelled event9","tool":"set_alarm","args":{"hour":9,"minute":19,"label":"event9"}}
+{"query":"Set alarm for 10:20 labelled event10","tool":"set_alarm","args":{"hour":10,"minute":20,"label":"event10"}}
+{"query":"Set alarm for 11:21 labelled event11","tool":"set_alarm","args":{"hour":11,"minute":21,"label":"event11"}}
+{"query":"Set alarm for 12:22 labelled event12","tool":"set_alarm","args":{"hour":12,"minute":22,"label":"event12"}}
+{"query":"Set alarm for 13:23 labelled event13","tool":"set_alarm","args":{"hour":13,"minute":23,"label":"event13"}}
+{"query":"Set alarm for 14:24 labelled event14","tool":"set_alarm","args":{"hour":14,"minute":24,"label":"event14"}}
+{"query":"Set alarm for 15:25 labelled event15","tool":"set_alarm","args":{"hour":15,"minute":25,"label":"event15"}}

eval/live_queries.jsonl

@@ -0,0 +1,5 @@
+{
+  "simulated": "not executed",
+  "simulated_reason": "collaborator modules pending",
+  "live": "not executed"
+}

eval/report.json

@@ -0,0 +1,6 @@
+# TinyAgent evaluation
+
+| Metric | Value |
+|---|---|
+| Simulated evaluation | not executed; collaborator modules pending |
+| Live evaluation | not executed |