Skip to content

Commit 935d9c1

Browse files
DYNAMOatGHUBclaude
andcommitted
Docs: rewrite README for the completed release
Replaces the "modules pending" status with user-facing documentation covering everything PRD-04 R4-74 requires: - problem summary linking to ARCHITECTURE.md; install - quickstart with a real tool (checked against the live model) and CLI - result semantics with provenance shown next to every answer and FINAL answers marked as unverified (R4-21, R4-22) - error codes and which are retried - supported runtimes with the tested Ollama version, model and hardware - tool-writing rules and the security model, including the prompt injection and thread-timeout limitations (R3-63) - simulated, live and replay results, each linked to its committed report - development commands and known limitations (ARCH-000 Sections 14-15) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent f7beb28 commit 935d9c1

1 file changed

Lines changed: 129 additions & 30 deletions

File tree

‎README.md‎

Lines changed: 129 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,147 @@
11
# TinyAgent
22

3-
TinyAgent routes one query to one registered Python tool through a local language
4-
model, with at most `1 + max_repairs` model calls. Its architecture and tradeoffs
5-
are documented in [ARCHITECTURE.md](ARCHITECTURE.md).
3+
TinyAgent is a zero-dependency Python library (stdlib only, Python >= 3.10) that
4+
lets a 7B/8B local language model, served by Ollama or llama.cpp, reliably call
5+
your Python functions.
66

7-
Aisha's reflection, registry, prompt, and frozen contract modules are integrated.
8-
The transport, parser, and executor modules are owned by the other engineers
9-
and have not yet landed. The CLI and full evaluation require the parser and
10-
executor. The evaluation uses the PRD-04 fixture signatures in
11-
`examples/demo_tools.py`; Aisha's `tests/fixtures_tools.py` has different
12-
`search` and `set_alarm` signatures.
7+
Agent frameworks built for frontier cloud models assume the model can follow
8+
long system prompts, emit valid JSON and recover from its own errors. A local
9+
8B model does not, which leads to context exhaustion, JSON parse crashes and
10+
retry loops that never end. TinyAgent instead treats the model as a
11+
**deterministic router**: each query becomes one bounded decision, either call
12+
one registered tool with arguments, or return a final answer. A query uses at
13+
most `1 + max_repairs` model calls (`max_repairs` is 0, 1 or 2), so infinite
14+
loops are structurally impossible. The full design and tradeoffs are in
15+
[ARCHITECTURE.md](ARCHITECTURE.md).
16+
17+
## Install
18+
19+
```bash
20+
pip install .
21+
```
22+
23+
There are no runtime dependencies. For live use, run a local
24+
[Ollama](https://ollama.com) or llama.cpp server.
1325

1426
## Quickstart
1527

1628
```python
17-
from examples.demo_tools import build_registry
18-
from tinyagent import TinyAgent
19-
from tinyagent.backends.simulator import SimulatedBackend
29+
from tinyagent import TinyAgent, ToolRegistry, tiny_tool
30+
from tinyagent.backends import OllamaBackend
2031

21-
backend = SimulatedBackend(['ACTION: add(a="2", b="3")'])
22-
agent = TinyAgent(backend, build_registry())
32+
registry = ToolRegistry()
33+
34+
@tiny_tool(registry=registry)
35+
def add(a: int, b: int) -> int:
36+
"""Add two integers."""
37+
return a + b
38+
39+
agent = TinyAgent(OllamaBackend("mistral:latest"), registry)
2340
result = agent.run("What is 2+3?")
24-
print(f"{result.text} (provenance: {result.provenance})")
41+
print(f"{result.text} (provenance: {result.provenance})") # 5 (provenance: tool)
2542
```
2643

27-
After the other modules land, run `python -m tinyagent --backend sim`.
28-
The CLI labels this mode `[SIMULATED]` and prints provenance next to every
29-
answer. A `FINAL:` answer has `provenance=model`; it is not verified.
44+
Without a model, use the scripted backend:
45+
`python examples/quickstart.py --backend sim`.
46+
47+
### Command line
48+
49+
```bash
50+
tinyagent --backend sim # scripted, prints [SIMULATED]
51+
tinyagent --backend ollama --model mistral:latest "What's the weather in Pune?"
52+
tinyagent --backend llamacpp --host http://127.0.0.1:8080 "Add 4 and 9" --verbose
53+
```
54+
55+
`python -m tinyagent` is equivalent. The CLI routes to the demo tools in
56+
`tinyagent/demo_tools.py`; exit code is 1 when `status` is `error`.
57+
58+
## Reading results
59+
60+
`agent.run()` returns an `AgentResult`. Always show `provenance` next to the
61+
answer:
62+
63+
| `status` | `provenance` | `text` | Meaning |
64+
|---|---|---|---|
65+
| `tool` | `tool` | Tool output, unmodified | A registered tool ran with validated arguments |
66+
| `final` | `model` | The model's own answer | No tool was used. **This text is not verified.** Pass `require_tool=True` to disallow it |
67+
| `error` | `none` | `CODE: detail` | See `error_code`; `run()` does not raise for model, backend or tool failures |
68+
69+
`run()` raises only `ValueError`, for an empty query or invalid configuration.
70+
71+
Error codes: parse (`E_EMPTY`, `E_NO_PREFIX`, `E_BAD_SYNTAX`) and argument
72+
(`E_UNKNOWN_TOOL`, `E_UNEXPECTED_ARG`, `E_MISSING_ARG`, `E_TYPE_COERCION`)
73+
failures trigger a repair prompt; if repairs run out the result is
74+
`E_RETRY_EXHAUSTED`. Backend failures (`E_BACKEND_*`) and tool failures
75+
(`E_TOOL_EXCEPTION`, `E_TOOL_TIMEOUT`) are never retried, because tools may
76+
have side effects.
3077

3178
## Supported runtimes
3279

33-
The planned live backends are Ollama and llama.cpp server. Tested runtime
34-
versions will be added after Deepak's transport handoff and a live run.
35-
Use `python -m tinyagent --backend ollama --model llama3:8b "What is 2+3?"`
36-
or `--backend llamacpp` when those modules are available.
80+
| Runtime | Endpoint | Tested |
81+
|---|---|---|
82+
| Ollama | `POST /api/generate` | Ollama 0.34.0, `mistral:latest` (7.2B, Q4_K_M), Linux x86_64, NVIDIA RTX 5070 |
83+
| llama.cpp server | `POST /completion` | Contract-tested against a local stub only |
84+
85+
Both backends use greedy, seeded decoding (`temperature 0`, `top_k 1`). They
86+
warn when the host is not loopback, because prompts would leave the machine.
87+
88+
## Writing tools
89+
90+
- Parameters may be `str`, `int`, `float`, `bool` or `Optional[...]` of those.
91+
`*args`, `**kwargs`, positional-only and complex types are rejected at
92+
decoration time.
93+
- The first docstring line (max 80 chars) becomes the tool description the
94+
model sees. Clear descriptions are the main lever on tool-selection accuracy.
95+
- Arguments are coerced strictly: `"2.5"` is not an `int`, `"nan"` is not a
96+
`float`, and booleans accept only `true/false/yes/no/1/0/on/off`.
97+
- Tool output is returned as data. It is never parsed again or executed.
98+
99+
## Security model
100+
101+
- Model text never becomes code: no `eval`, `exec`, `compile`, `pickle` or
102+
`importlib` in the package (enforced by an AST gate in CI).
103+
- Only registered tools can run, by exact name. There is no fuzzy matching.
104+
- Ambiguous output (unterminated quotes, duplicate arguments, non-integral
105+
integers) is rejected, never guessed.
106+
- **Prompt injection:** a user query can still persuade the model to call any
107+
registered tool. Tools that change state must enforce their own authorization.
108+
- **Timeouts:** tools run in a daemon thread with a timeout (default 10 s).
109+
Python cannot kill threads, so a timed-out tool keeps running in the
110+
background. Tools that call external systems must enforce their own timeouts.
111+
- Error details are single-line and capped; prompts are redacted from backend
112+
errors and file paths from tool exceptions.
37113

38114
## Evaluation
39115

40-
`python -m eval.run_eval --mode sim` runs the seeded fault corpus without a GPU
41-
once the parser and executor are available. The latest
42-
[evaluation report](eval/report.md) currently marks both simulated and live
43-
runs as not executed. No live accuracy is claimed until a live run is recorded.
116+
| Run | Command | Result |
117+
|---|---|---|
118+
| Simulated (no model, CI) | `python -m eval.run_eval --mode sim` | 795 fault-injection cases: 280/280 recoverable faults recovered on the first call, **0 false executions**, max 2 model calls (ceiling 2), median framework overhead 0.05 ms — [report](eval/report.md) |
119+
| Live | `python -m eval.run_eval --mode live --backend ollama --model mistral:latest --record eval/fixtures/live-mistral.jsonl --out eval/report_live.json` | 50 queries: **100% tool selection, 100% argument accuracy**, 0 repairs, p50 130 ms, p95 184 ms — [report](eval/report_live.md) |
120+
| Replay | `python -m eval.run_eval --mode replay --fixture eval/fixtures/live-mistral.jsonl` | Re-runs the recorded live session exactly; also run in CI by `tests/test_replay.py` |
121+
122+
Live numbers are for one machine and model as listed above. They are
123+
measurements, not guarantees. Other models, quantizations or hardware can
124+
differ.
125+
126+
## Development
127+
128+
```bash
129+
python -m unittest discover -s tests -t . # unit, integration, gate and replay tests
130+
python -m eval.run_eval --mode sim # simulated evaluation (exit 1 on any NFR failure)
131+
python -m bench.bench_parser # parser p99 latency (NFR-8)
132+
```
133+
134+
CI runs all three on Python 3.10, 3.12 and 3.13, plus a clean-install smoke test.
135+
136+
## Known limitations
44137

45-
Known limits: one tool per query; no conversation memory, tool chaining, or
46-
complex argument types. A valid but wrong tool choice cannot be detected by the
47-
parser. Registered tools must enforce their own authorization and timeouts;
48-
the executor cannot kill a tool thread after timeout.
138+
- One tool per query; no conversation memory, tool chaining, async API or
139+
streaming.
140+
- No complex argument types (`list`, `dict`, nested objects).
141+
- A valid but wrong tool choice cannot be detected by the parser; it is
142+
returned with `provenance="tool"`. The live evaluation measures how often this
143+
happens.
144+
- Temperature 0 is not bit-identical across hardware and runtime versions; use
145+
record/replay for reproducible tests.
146+
- No OS-level sandboxing: TinyAgent controls *which* registered function runs
147+
and *with what* validated arguments, not what the function does.

0 commit comments

Comments
 (0)