Each scenario is intentionally small. No fictional company needs every feature.
| Scenario | Break | Minimum fix | Primary metric |
|---|---|---|---|
deep_research |
unsupported synthesis | immutable citations + verifier | unsupported claims |
repo_change |
duplicate or unsafe tool call | typed registry + effect ledger | invalid/duplicate actions |
approval |
parameters change after approval | exact action digest + recheck | mutated executions (must be 0) |
purchase |
provider receives charge, response lost | UNKNOWN reconciliation | blind retries (must be 0) |
long_wait |
worker dies or deploy changes | checkpoints + version rules | recovered committed step |
malicious_source |
retrieved prompt injection | untrusted screen + policy/egress | unsafe block rate |
multi_tenant |
tenant B requests A's data | storage PEP + PostgreSQL RLS | cross-tenant hits (must be 0) |
provider_failure |
timeouts/brownout | retry budget + breaker + fallback | amplification and degraded rate |
memory_poisoning |
stored content modified | trust promotion + signature read gate | poisoned retrievals (must be 0) |
model_migration |
V1 run resumes after V2 | component PIN/MIGRATE/LATEST | successful compatible resume |
Use experiments/topology.py for T0 single, T1 producer/verifier, and T2
specialists/verifier. Record task success, unsupported claims, verifier catch
rate, duplicate work, tokens, cost, wall clock, and failure containment. A
single agent winning under the constraint is a successful experiment result.
Use patterns/lab.py only after T2 has earned its cost. Peer/debate patterns
duplicate work by design; dynamic fan-out is bounded by explicit task breadth.