You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: re-anchor README on live-state replay and recovery
Lead with replay-under-live-state and executed recovery (decision #5); move the Auditable Agents paper to Citation as evidence, not the banner. The quickstart now shows the executed rollback through the gate (verified). State the v0.1 boundary honestly and position the data layer (PyOD, v0.2) at roadmap altitude without exposing the private method.
[Quickstart](#60-second-quickstart) · [How It Works](#how-it-works) · [The Full Chain](#the-full-chain) · [Roadmap](#roadmap)
13
+
14
14
</div>
15
15
16
-
`auditable` is an open-source SDK that makes an AI agent's decisions reconstructable after the fact. For every consequential action your agent takes, it captures a signed record of what the agent read, which model decided, and what the agent did, then lets you **replay** that decision under the state that was actually live and route a fix: allow, block, hand off, or roll back.
16
+
`auditable` is an open-source SDK for the moment after an agent acts. For every consequential decision it captures a signed record of what the agent read, the dependency state the decision relied on, which model decided, and the action taken. Later it **replays** that decision against the state that is live now, and when the action no longer holds it **executes**a fix through a rail: allow, block, hand to a human, or roll back.
17
17
18
-
> By [Yue Zhao](https://github.com/yzhao062), creator of [PyOD](https://github.com/yzhao062/pyod) (42M+ downloads). Built on the Auditable Agents framework ([arXiv:2604.05485](https://arxiv.org/abs/2604.05485)).
18
+
> [!NOTE]
19
+
> By [Yue Zhao](https://github.com/yzhao062), creator of [PyOD](https://github.com/yzhao062/pyod) (42M+ downloads, ~12k citations). `auditable` holds each layer to PyOD's bar: a method with data and an experiment behind it before that layer is promoted.
19
20
20
21
## The Problem
21
22
22
-
Most agent tools log *what happened*. They do not record *what the agent relied on when it decided*, so when a payment, an approval, or a tool call later looks wrong, the budget, the policy, and the allow-list that were live at that moment are already gone. The action was reasoned; the dependency it trusted had drifted. By the time anyone asks, the decision can no longer be reconstructed.
23
+
Most agent tools log what happened. They do not record what the agent relied on when it decided, so when a payment, an approval, or a tool call later looks wrong, the budget, the policy, and the allow-list that were live at that moment are already gone. The action was reasoned; the dependency it trusted had drifted. Flagging that after the fact is observability. Re-deciding under the current state and reversing the action is recovery, and recovery is the gap `auditable` fills.
23
24
24
25
## Install
25
26
@@ -30,65 +31,85 @@ pip install auditable
30
31
## 60-Second Quickstart
31
32
32
33
```python
33
-
from auditable import Action, DependencySnapshot, audit, replay
See [`examples/payment_audit.py`](examples/payment_audit.py) for the full flagship demo.
60
+
See [`examples/payment_audit.py`](examples/payment_audit.py) for the full demo that binds all three layers, and [`examples/standalone_report.py`](examples/standalone_report.py) for scoring a single layer on its own.
52
61
53
62
## How It Works
54
63
55
-
One agent decision crosses three layers, and `auditable`records all three in a single signed, hash-chained record:
64
+
One agent decision crosses three layers, and `auditable`binds all three in a single signed, hash-chained record:
56
65
57
-
| Layer | What It Captures|
58
-
|---|---|
59
-
|**Data**| What the agent read: inputs, retrieved context, and the dependency snapshot (budget, policy, allow-list, config versions) that was live|
60
-
|**Model**| Which model produced the output, and the stated decision basis |
61
-
|**Harness**| The action the agent executed, and its cost |
66
+
| Layer | What the record binds | Signal in v0.1|
67
+
|---|---|---|
68
+
|**Data**| What the agent readand the dependency snapshot it relied on | Snapshot freshness|
69
+
|**Model**| Which model produced the output, and its stated basis | Decision-basis trust flag|
70
+
|**Harness**| The action executed and its cost| A static rule, plus the replay verdict|
62
71
63
-
`replay()` re-derives whether the action is still justified under the live dependency state versus the snapshot the agent actually used. A decision that passed on a stale snapshot but fails on live state is exactly the failure `auditable` exists to catch.
72
+
`replay()` re-derives whether the action still holds under the live dependency state versus the snapshot the agent used, and returns one of four verdicts: `ALLOW`, `ROLLBACK` (justified on the snapshot but not on live state, the stale-state case), `BLOCK` (justified on neither), or `HUMAN_REVIEW`. The `ActionGate` then executes that verdict through a rail, so a rollback reverses the action rather than printing a recommendation. Replay is pure: it never mutates the signed record.
64
73
65
-
## The Full Chain: AI Risk Audit and Control across Agents, Foundation Models, and Data
74
+
## The Full Chain
66
75
67
-
A single agent decision crosses the whole stack, so one `auditable`record already spans all three layers, and the library is built to connect a signal source at each:
76
+
The data, model, and harness signals live in one record, so a decision is judged as a unit rather than as three disconnected logs. Each layer's check is a standalone module (an `Auditor` that returns a signed `Report`), and the same modules compose into the decision record. Each layer deepens on its own cadence, and the method behind it carries its own benchmark before it is promoted.
68
77
69
-
| Layer |What the record captures | Signal source it connects|
78
+
| Layer |v0.1 (shipping) | Deepens to|
70
79
|---|---|---|
71
-
|**Agent (harness)**|The action executed, its cost, and the replayable verdict | Native (shipping now)|
72
-
|**Foundation model**|Which model produced the output, and the stated basis | TrustLLM trust and behavior signals (roadmap) |
73
-
|**Data**|What the agent read, and the dependency snapshot it relied on |[PyOD](https://github.com/yzhao062/pyod) anomaly scores on inputs and retrieved context (roadmap) |
80
+
|**Harness (agent)**|signed record, replay, executed gate over a rail | dynamic rules layered on static rules|
81
+
|**Data**|snapshot freshness | anomaly detection on the data a decision relied on, [PyOD](https://github.com/yzhao062/pyod) lineage (v0.2) |
82
+
|**Model**|decision-basis trust flag |[TrustLLM](https://github.com/HowieHwong/TrustLLM) trust signals (v0.3) |
74
83
75
-
The agent decision is the spine. `auditable` starts there, captures the full chain in one signed record, and brings the data-layer and model-layer signals in as plug-in inputs. That is the main line, delivered through one replayable decision record that threads all three layers.
84
+
> [!IMPORTANT]
85
+
> **v0.1 scope, stated honestly.** The full chain, replay under live state, and executed recovery through a rail-neutral gate ship today, with thin but real data and model signals bound into the record and two sinks (in-memory and append-only JSONL). The deep PyOD and TrustLLM methods, the calibrated cross-layer risk, and the data and model control faces are on the [roadmap](#roadmap). v0.1 does not yet claim a learned data-anomaly method or a model-trust score.
76
86
77
-
**v0 scope (honest):** the decision record spans all three layers, and the agent-decision capture and replay ship today. The PyOD and TrustLLM signal integrations that fill the data and model spans are on the [roadmap](#roadmap); `auditable` does not yet score data anomalies or model trust itself.
87
+
## Using a Single Layer
78
88
79
-
## Roadmap
89
+
Each layer's check runs on its own, with no agent and no chain, and returns a signed report:
80
90
81
-
The path to the full chain, one signal source at a time:
91
+
```python
92
+
import time
93
+
from auditable import DataAuditor, DependencySnapshot
-[ ]**Data layer**: anomaly-triggered surfacing via [PyOD](https://github.com/yzhao062/pyod), flag which decisions to replay from anomalies in what the agent read
85
-
-[ ]**Model layer**: TrustLLM trust and behavior signals attached to the model span
0 commit comments