Awesome Auditable AI

Note

Auditability is the goal: being able to establish what an agent did, what it relied on, why it acted, and whether the action was right. Reliability engineering is how a system gets there. An agent that behaves inconsistently, loses its trace under stress, or cannot attribute a failure to a step is one that cannot be audited afterwards, whatever records it keeps. This list therefore covers both: monitoring, robustness, fault tolerance, and failure attribution alongside decision records, accountability, and post-hoc review.

Maintained by the author of auditable and GRADE, two components of The Auditable Agents Ecosystem. Every component is also listed in the topical section it belongs to, under the same inclusion bar as every other entry.

Agent capability is rising much faster than agent reliability. Towards a Science of AI Agent Reliability (ICML 2026) decomposes reliability into twelve metrics over fifteen models and finds that recent capability gains produced only small reliability gains. The cost of that gap shows up after a run rather than during it: in the original Who&When study (ICML 2025), the strongest of the three attribution methods released with the benchmark identified the responsible agent in 53.5% of 184 annotated failure tasks, drawn from logs of 127 LLM multi-agent systems, and the decisive error step in 14.2%. Later systems in this list, including AgenTracer (ICLR 2026), report gains on that benchmark. What narrows the gap is the quality of the record. TraceElephant (ACL 2026) reports that full execution traces raised step-level attribution accuracy from 17% to 30% in its static-agentic setting, a 76% relative gain over output-only traces.

This list maps the work on that problem: 191 entries across nine sections, covering 132 unique arXiv papers, 90 GitHub repositories, 16 standards, and one framework. Five papers are deliberately cross-listed, appearing once in a topical section and once as a dataset.

Two properties make the list checkable. Every paper row carries a venue field, so of the 134 table rows, 69 name a publication venue and 65 are marked Preprint; a reader can see the evidence level of a row without opening it. Naming a venue is not the same as peer review, and the field records what the venue is rather than grading it: a non-archival workshop and a main-conference track both appear as what they are. And the citation check goes past asking whether a link responds: each arXiv entry's title is compared with the title the arXiv page itself reports, and only an exact match counts. A near-identical title is not treated as agreement, because that is precisely what a link to the wrong paper in a series looks like; "Part I" against "Part II" scores 0.995 on similarity. The 2026-08-23 run compared 129 entry titles and 132 identifiers across 265 destinations and found no disagreements. It names every destination that refuses automated clients rather than quietly counting it as passing. That run is also what caught a dataset repository going 404 three days after it last resolved. The four destinations it does not audit are this repository's own badges and the pages behind them, and the report names those too.

Regenerate the report with tools/check_links.py, recount the list with tools/inventory.py, and read the result in LINK-AUDIT.md. The ways this audit could report a false pass are kept as tests in tools/test_check_links.py, which runs on every pull request along with the audit itself.

Contents

• The Reliability Map
• The Auditable Agents Ecosystem
• Surveys and Foundations
• Failure Attribution and Diagnosis
• Reliability and Robustness
• Runtime Monitoring and Guardrails
• Audit Trails and Decision Records
• Security Auditing and Scanners
• Datasets and Benchmarks
• Tools and Platforms
• Standards and Governance
• Contributors
• Maintained By
• Citation

---

The Reliability Map

A reader-first map of where AI agent reliability is won and lost. Reliability is the umbrella; auditing (decision records, accountability, post-hoc review) is one visible strand within it. The dimensions below describe outcomes a builder or reviewer should be able to demand from an agent system, not the internal machinery any one tool uses to deliver them.

Use these pointers to move from each reliability outcome to the sections that most directly measure or implement it.

Dimension	Start with
Consistency and Determinism	Reliability and Robustness; Datasets and Benchmarks
Robustness	Reliability and Robustness; Security Auditing and Scanners
Fault Tolerance and Recovery	Reliability and Robustness; Tools and Platforms
Runtime Monitoring	Runtime Monitoring and Guardrails; Tools and Platforms
Failure Attribution and Diagnosis	Failure Attribution and Diagnosis; Datasets and Benchmarks
Decision Records and Accountability	Audit Trails and Decision Records; Standards and Governance
Security Review	Security Auditing and Scanners; Runtime Monitoring and Guardrails
Evaluation and Benchmarks	Datasets and Benchmarks; Tools and Platforms

Coverage note: Consistency and Determinism is the least represented dimension here, with eight direct resources against 76 for Security Review. That gap reflects where the field has put its effort, not a deliberate editorial choice. Contributions on run-to-run reproducibility, deterministic replay, bounded stochasticity, and consistency controls are especially welcome.

Reliability dimensions: detailed definitions

Dimension	What a builder or reviewer should be able to demand
Consistency and Determinism	The same task under the same conditions produces the same decision, or the variation is bounded and explained; runs are reproducible enough to debug and to trust.
Robustness	Behavior holds under perturbed inputs, adversarial prompts, distribution shift, and unreliable tool or model responses, instead of degrading silently or unpredictably.
Fault Tolerance and Recovery	When a step, tool, or sub-agent fails, the system degrades safely, contains the blast radius, and returns to a known-good state rather than compounding the error downstream.
Runtime Monitoring	Live signals (traces, spans, tool calls, token and cost budgets, policy checks) surface risky or anomalous behavior while the agent is running, early enough to intervene.
Failure Attribution and Diagnosis	After a bad outcome, the responsible step, agent, and root cause can be located from the execution record, so the fix targets the cause rather than the symptom.
Decision Records and Accountability	Each consequential action leaves a durable, inspectable record of what was decided, on what inputs, and why, supporting later review, dispute resolution, and external audit.
Security Review	The agent is assessed for prompt injection, tool and data poisoning, privilege misuse, and unsafe action execution, with controls that hold across the tool and multi-agent surface.
Evaluation and Benchmarks	Standard tasks, datasets, and metrics measure reliability comparably across systems and over time, so claims of improvement are testable rather than anecdotal.

Common failure modes: spec and verification gaps, error propagation, outdated context, tool faults, unsafe actions, silent degradation

• Specification and task-verification gaps: the agent satisfies the literal instruction but misses the user's intent, or never verifies that its own output is correct before acting on it.
• Error propagation across steps: an early mistake (a wrong tool argument, a misread observation) is carried forward and amplified instead of being caught, so a small fault becomes a failed task.
• Acting on outdated or wrong context: the agent decides against information that no longer reflects reality, or that it misremembered, and the action is correct only for a world that has changed.
• Inter-agent miscoordination: in multi-agent setups, agents talk past each other, duplicate work, deadlock, or disagree on who owns a step, and no one detects the breakdown.
• Tool and environment faults: a tool times out, returns malformed data, or behaves differently than expected, and the agent neither retries sensibly nor surfaces the failure.
• Unsafe or out-of-policy actions: the agent executes a consequential side effect (a write, a payment, a deletion) that violates a constraint, sometimes induced by injected or poisoned input.
• Nondeterministic drift: the same prompt yields materially different trajectories across runs, making behavior hard to reproduce, test, or trust.
• Silent degradation: quality, latency, or cost worsens gradually with no alarm, because the system has no live signal tied to acceptable behavior.
• Loop and runaway behavior: the agent repeats steps, retries without progress, or consumes unbounded tokens, budget, or wall-clock time without converging.
• Opaque failure: something went wrong, but the execution record is too thin to say which step or agent caused it, so diagnosis stalls.

Evaluation axes: task success and variance, robustness under stress, safety, attribution accuracy, detection quality, cost, calibration

• Task success rate and quality on held-out tasks, reported with confidence intervals rather than a single headline number.
• Reproducibility and run-to-run variance: how stable the outcome and the trajectory are across repeated runs of the same task.
• Robustness under stress: success retained under perturbed inputs, adversarial prompts, injected instructions, and induced tool or model faults.
• Safety and policy adherence: rate of constraint violations and unsafe actions, including completion measured only when policy is respected.
• Failure attribution accuracy: how often the responsible step or agent is correctly identified from the execution record.
• Detection quality for monitoring: true-positive rate at a fixed false-positive rate, plus precision, recall, and ranking metrics on flagged behavior.
• Cost and efficiency: tokens, tool calls, latency, and dollars per successful task, since a reliable agent that is unaffordable does not ship.
• Calibration and uncertainty: whether the agent's confidence and its decision to defer or ask for help track its actual accuracy.
• Auditability: whether an independent reviewer can reconstruct what happened and why from the recorded evidence alone.
• Generalization: performance on tasks, domains, and tools held out of development, not only on the data used to tune the system.

Open gaps: no shared benchmark, weak failure-label ground truth, attribution at scale, silent-failure detection, calibrated abstention

• No broadly adopted, comprehensive reliability benchmark with community-agreed tasks and metrics. ReliabilityBench covers consistency, perturbation robustness, and injected tool faults, but the eight dimensions above are not yet unified in one shared suite.
• Failure-label ground truth is uneven rather than absent. Who&When, TRAIL, AgentRx, TraceElephant, and Aegis supply labels, and Who&When Pro reaches 12,326 golden-labeled trajectories across 26 source benchmarks by replaying a successful prefix and injecting a single failure, but manually annotated sets remain in the low hundreds of trajectories and scale now comes from construction rather than annotation.
• Attribution at scale remains unreliable. AgenTracer, AgentRx, FALAT, and StepFinder automate localization, but accuracy on long, branching, multi-agent traces stays below what practice needs.
• Silent, benign failures are detected but not localized. HINTBench audits risk that arises under benign conditions and reports that strong models detect trajectory-level risk while scoring below 35 Strict-F1 on locating the risky step, and that existing guard models transfer poorly to this setting.
• Calibrated uncertainty and reliable abstention remain open. AgentAbstain and HiL-Bench now measure whether agents know when to stop or ask, and the best models are far from reliable at it.
• Reproducibility under inherent model stochasticity remains open beyond measurement and replay. Runs are still difficult to compare causally without discarding useful nondeterminism.
• Monitoring overhead and signal quality remain open despite promising low-latency results, and monitors themselves degrade with context length and can be evaded by an agent that knows the protocol.
• Standard decision records remain open. OpenTelemetry standardizes agent telemetry, and Agent-BOM, ActiveGraph, and GRADE propose richer representations, but no widely adopted cross-vendor schema captures decisions, dependencies, rationale, integrity, and responsibility together.
• Recovery evaluation is emerging rather than absent. ReliabilityBench evaluates agents under injected faults and PALADIN reports explicit recovery metrics, but there is no agreed cross-system recovery protocol or metric suite.
• Cost-aware reliability is partly addressed. AI Agents That Matter argues for cost-controlled evaluation and motivates the goal of jointly optimizing accuracy and cost, and several tools report spend, but joint evaluation of safety, success, recovery, and cost is not standardized.
• Evaluation integrity is itself unsettled. BenchJack shows widely used agent benchmarks can be scored near-perfectly without solving a task, and AgentRewardBench shows rule-based scoring disagrees with expert review.

---

The Auditable Agents Ecosystem

Open-source components built by this list's maintainer and collaborators around one question: after an agent acts, can you establish what it relied on, whether it should have acted at all, and how to undo it? The first three rows are the mechanism classes the Auditable Agents framework defines, alongside five auditability dimensions. The last two rows supply the run-level representation and the benchmark evidence used to evaluate those mechanisms; neither is one of the mechanism classes. Ecosystem home: auditable-agents.github.io, which currently documents the first three.

Stage	Component	What it does
Detect	agent-audit	Static analysis of agent code and MCP configuration before deployment, with rules mapped to the OWASP Agentic Top 10
Enforce	Aegis	Policy check before a tool call executes, with a cryptographic audit trail and human approval gates
Recover	auditable	Records what a committed action relied on, re-evaluates it against current state, and reverses it through a pluggable rail
Represent	GRADE	One graph over a run with execution and dependency layers, used to predict failure and localize the faulting step
Evaluate	CatchBench	Seven boards over five scenarios test failure auditing from declared configuration before a run, growing prefixes during it, and the whole trace after it

The framework paper is Auditable Agents.

Every component above is also listed in the topical section it belongs to, under the same inclusion bar as every other entry.

---

Surveys and Foundations

Resource	Venue	Summary	Links
The Rise and Potential of Large Language Model Based Agents: A Survey	Sci. China Inf. Sci. 2025	Broad survey framing LLMs as the basis for autonomous agents, covering single-agent, multi-agent, and human-agent settings.	[Paper List]
A Survey on Large Language Model based Autonomous Agents	Front. Comput. Sci. 2024	Systematic review of LLM-based autonomous agents with a unified construction framework and a section on agent evaluation strategies.	[Paper List]
Large Language Model based Multi-Agents: A Survey of Progress and Challenges	IJCAI 2024	Survey of LLM multi-agent systems organized by agent profiling, communication, and capability growth, with open challenges.	[Paper List]
LLM Multi-Agent Systems: Challenges and Open Problems	Preprint 2024	Position-style survey of unresolved problems in multi-agent systems, including task allocation, reasoning debate, context, and memory.
Survey on Evaluation of LLM-based Agents	ACL Findings 2026	Survey of agent evaluation and benchmarks, highlighting gaps in cost, safety, and robustness.	[Paper List]
A Survey on Trustworthy LLM Agents: Threats and Countermeasures	KDD 2025	TrustAgent taxonomy separating intrinsic components (brain, memory, tool) from extrinsic ones (user, other agents, environment), mapping attacks, defenses, and evaluation onto each.
Visibility into AI Agents	FAccT 2024	Accountability-focused position paper proposing agent identifiers, real-time monitoring, and activity logging as measures to make agent activity answerable.
AgentOps: Enabling Observability of LLM Agents	Preprint 2024	Taxonomy of the artifacts and trace data to record across an agent's lifecycle, drawn from a mapping study of observability tools.

---

Failure Attribution and Diagnosis

Resource	Venue	Summary	Links
Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems	ICML 2025	Defines automated failure attribution for LLM multi-agent systems and releases the Who&When dataset of annotated failure logs labeling the responsible agent and the decisive error step.	[Code]
Why Do Multi-Agent LLM Systems Fail?	NeurIPS 2025 Datasets and Benchmarks Track	Derives MAST, a 14-mode failure taxonomy in three groups (system design, inter-agent misalignment, task verification), from 150 expert-examined traces, and releases MAST-Data, 1,642 annotated traces across seven multi-agent frameworks.	[Code]
TRAIL: Trace Reasoning and Agentic Issue Localization	Preprint 2025	Annotates 148 single- and multi-agent traces with 841 errors under a reasoning, planning, and execution taxonomy, and shows long-context models score near 11% at locating them.
AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?	ICLR 2026	Curates the TracerTraj attribution dataset and trains AgenTracer-8B to identify the responsible agent and step in long failed trajectories.	[Code]
Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution	NeurIPS 2025 Workshop (LLM Evaluation)	Proposes ECHO, an agent- and step-level attribution method combining hierarchical context, objective analysis, and consensus voting for subtle reasoning errors.
Aegis: Automated Error Generation and Attribution for Multi-Agent Systems	ICLR 2026	Generates controlled faults to build a 9,533-trajectory dataset with annotated faulty agents and error modes for training and evaluating attribution methods.	[Code]
AgentRx: Diagnosing AI Agent Failures from Execution Trajectories	Preprint 2026	Synthesizes constraints, checks them step by step, and records the results in an auditable validation log that localizes the critical failure step within a 10-category taxonomy across three domains.	[Code]
Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems	ACL 2026	Introduces TraceElephant, a failure-attribution benchmark with fully observable traces and reproducible environments, reporting attribution accuracy up to 76% higher with full execution traces than with output-only logs.	[Code]
Demystifying the Lifecycle of Failures in Platform-Orchestrated Agentic Workflows	Preprint 2025	Releases AgentFail, a collection of 307 failures from low-code agent platforms, and characterizes them by manifestation, root cause, and repair strategy at the workflow-node level.
FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search	Preprint 2026	Frames attribution as dependency-guided coarse-to-fine search that separates root-cause steps from inherited downstream errors, evaluated on the Who&When benchmark.
Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?	Preprint 2026	Builds Who&When Pro, 12,326 failed trajectories with golden labels across three modalities and 26 source benchmarks, constructed by replaying a successful prefix exactly and then injecting a single failure.	[Code], [Project]
Where LLM Agents Fail and How They can Learn From Failures	Preprint 2025	Introduces AgentErrorTaxonomy across memory, reflection, planning, action, and system operations, releases AgentErrorBench of annotated failure trajectories from ALFWorld, GAIA, and WebShop, and reports 24% higher all-correct accuracy for its AgentDebug root-cause debugger.	[Code]
StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems	KDD 2026	Uses a language model only to encode execution logs into temporal semantic sequences and scores steps with parameter-efficient temporal and attention modules, cutting inference time by 79% against the fastest language-model-based method on Who&When.
Towards Self-Improving Error Diagnosis in Multi-Agent Systems	ACL Findings 2026	Prunes trajectory context by symptom-driven backward tracing and gates memory updates on executable evidence, so ErrorProbe needs no annotation and transfers without retraining.
Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluation	Preprint 2026	Releases MP-Bench and argues that reported language-model weakness at attribution largely reflects single-root-cause benchmark design rather than model capability.
Tracing Agentic Failure from the Flow of Success	Preprint 2026	Trains one-class neural controlled differential equations on 100 successful trajectories alone and scores deviation, gaining 20% in-domain F1 at 200 to 5000 times the speed of prompting baselines.
Conformal Agent Error Attribution	Preprint 2026	Applies filtration-based conformal prediction over trajectories, returning contiguous step sets with finite-sample distribution-free coverage guarantees.
SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents	Preprint 2026	Releases SearchAuditBench, 1,243 failed deep-search trajectories averaging 73.1 messages and 65.1K tokens, on which the strongest baseline reaches a 26.6% end-to-end pass rate.
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories	Preprint 2026	Annotates 2,790 deep-research trajectories and filters them into TELBench, a 1,000-instance benchmark for erroneous-span and first-error localization, on which DRIFT gains up to 30 percentage points.
SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation	ICML 2026 Workshop (AIWILD)	Runs a tool-augmented diagnostic loop over short-term memory instead of loading a whole trajectory, gaining 20% on Who&When at a 1M token budget and 19% on the TRAIL GAIA subset at a 25K token budget.
Failure as a Process: An Anatomy of CLI Coding Agent Trajectories	Preprint 2026	Annotates 1,794 CLI coding trajectories over 63,000 steps from 7 models and 3 scaffolds on Terminal-Bench, finding that failures are mostly epistemic, start early, and stay hidden until recovery is impossible.
TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems	Preprint 2026	Localizes error steps in agentic coding trajectories and releases RootSE, a benchmark of 102 failure trajectories with human-annotated error steps spanning localization, patch generation, verification, and environment failures.	[Code]
Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes	Preprint 2026	Mines 13,602 issues and pull requests from 40 agentic repositories, samples 385 faults, derives 34 fault types, and validates the taxonomy with 145 practitioners.
CUADebug: Diagnosing and Repairing Computer-Use Agent Failures	Preprint 2026	Releases CUAErrorBench, 204 human-annotated OSWorld failures, and raises joint subtype-and-step diagnosis from 11.2% to 19.6% on the Claude-agent split.
GRADE: Graph Representation of LLM Agent Dependency and Execution	Preprint 2026	Models a run as one graph with two edge layers, execution for what ran in what order and dependency for what each step relied on; across six corpora the dependency layer predicts failure where run size carries little signal, and the execution layer localizes the faulting step.	[Code]

---

Reliability and Robustness

Resource	Venue	Summary	Links
ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions	Preprint 2026	Measures tool-using agent reliability along three axes: consistency under repeated runs (pass^k), robustness to semantically equivalent task perturbations, and fault tolerance under injected tool and API failures.
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains	ICLR 2025	Benchmarks agents in simulated tool-agent-user dialogues and introduces the pass^k metric, which scores whether an agent solves the same task on all of k independent trials.
Non-Determinism of "Deterministic" LLM System Settings in Hosted Environments	Eval4NLP 2025 Workshop	Documents variation in output text and task accuracy across repeated runs under nominally deterministic settings (temperature 0 and fixed decoding), identifying one source of run-to-run inconsistency in agent pipelines.	[Code]
When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents	ICML 2026 Workshop (Statistical Frameworks for Uncertainty in Agentic Systems)	Runs the same agent on the same task many times, finding several distinct ReAct-style action paths per ten runs, with inconsistent-path tasks scoring lower than consistent-path ones.
AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition	ICML 2026	Quantifies how tool-using agents degrade under user-side instruction noise and tool-execution noise, reporting accuracy drops across models under realistic perturbations.
PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Cases	AAAI 2026 Workshop (LaMAS)	Trains agents to detect and recover from tool malfunctions (timeouts, API exceptions, inconsistent outputs) that otherwise trigger cascading reasoning errors and task abandonment.	[Code], [Data]
Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows	Preprint 2026	Wraps tool use in progress-aware transactions that commit only after per-resource frontiers rule out earlier conflicting work, at microsecond-scale wrapper overhead.
ACRFence: Preventing Semantic Rollback Attacks in Agent Checkpoint-Restore	ASPLOS 2026 Workshop (CoDAIM)	Names semantic rollback attacks, in which an agent re-synthesizes a slightly different request after restore and the server treats it as new, enabling duplicate payments and credential reuse.
ProtocolBench: Which LLM MultiAgent Protocol to Choose?	ICML 2026	Compares A2A, ACP, ANP, and Agora on success, latency, message overhead, and failure robustness, finding that completion time in the Streaming Queue scenario varies by up to 36.5% across protocols and mean end-to-end latency differs by 3.48 seconds.	[Code]
Towards a Science of AI Agent Reliability	ICML 2026	Argues that a single success rate hides operational flaws and proposes twelve metrics decomposing agent reliability into consistency, robustness, predictability, and safety.
Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents	Preprint 2026	Proposes reliability metrics for long-horizon tasks (reliability decay curve, variance amplification, graceful degradation, meltdown onset) and evaluates 10 models over 23,392 episodes.

---

Runtime Monitoring and Guardrails

Resource	Venue	Summary	Links
AI Control: Improving Safety Despite Intentional Subversion	ICML 2024	Evaluates monitoring protocols intended to improve deployment safety when a model tries to subvert oversight, using a trusted weaker model and limited trusted labor.	[Code]
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations	Preprint 2023	Screens user prompts and model responses against a safety risk taxonomy as an input-output filter for agent applications.	[Model]
GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning	ICML 2025	Reads safety requirements, generates a check plan, and compiles the plan into guardrail code that checks a target agent's runtime actions without retraining.	[Code]
AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems	Preprint 2024	Captures per-agent inputs and outputs to predict task outcomes ahead of time and apply real-time corrections when a malicious agent threatens the multi-agent run.	[Code]
G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems	ACL 2025	Runs a graph neural network over the multi-agent utterance graph to flag compromised agents and then applies topological intervention to remediate the attack mid-run.	[Code]
AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection	ACL 2025	Generates and optimizes safety checks during execution, adapting them to task-specific and systemic risks over an agent's lifetime rather than relying on a fixed rule set.	[Code]
GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling	NeurIPS 2025	Models a multi-agent collaboration as a temporal attributed graph and uses an unsupervised encoder-decoder to detect anomalous nodes and edges as errors propagate.	[Code]
SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems	Preprint 2025	Watches the live execution graph, scores nodes, edges, and paths against security policies, and intervenes on single-point faults or collusion.
ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction	ASE 2026	Estimates the likelihood of reaching an unsafe future state with a discrete-time Markov chain so the monitor can warn and intervene before a violation occurs.	[Code]
AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents	CCS 2025	Intercepts sensitive operations in a computer-use agent and halts execution until a security audit clears, correlating task context with system traces in real time.	[Code]
Trajectory Guard: A Lightweight, Sequence-Aware Model for Real-Time Anomaly Detection in Agentic AI	AAAI 2026 Workshop (TrustAgent)	Uses a Siamese recurrent autoencoder to classify a whole agent trajectory as either a wrong plan for the task or a malformed plan structure, reporting 32 ms per sample and a 17 to 27 times speedup over language-model judge baselines.
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols	ICLR 2026	Shows that an untrusted model which knows the control protocol and the monitor can embed prompt injections in its own outputs to evade diverse language-model monitors on two AI control benchmarks, and that the Defer-to-Resample protocol amplifies the injection into a best-of-n attack.
Reliable Weak-to-Strong Monitoring of LLM Agents	ICLR 2026	Stress-tests monitors watching a stronger agent for covert misbehavior across SHADE-Arena and CUA-SHADE-Arena, showing that monitor scaffolding matters more than monitor awareness and that escalating only pre-flagged cases to human reviewers raises the true-positive rate by about 15 percent at a false-positive rate of 0.01.
Classifier Context Rot: Monitor Performance Degrades with Context Length	Preprint 2026	Measures how language-model monitors of coding agents degrade as transcripts grow, reporting that Opus 4.6, GPT 5.4, and Gemini 3.1 miss dangerous actions 2x to 30x more often after 800K tokens of benign activity than when the same actions appear alone.
ActPlane: Programmable OS-Level Policy Enforcement for Agent Harnesses	Preprint 2026	Enforces agent-declared policy in the kernel with eBPF and an information-flow language, covering indirect execution paths that tool-call interception cannot observe, at 1.9% to 8.4% overhead.
SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors	Preprint 2026	Assembles 40 covert attacks across 11 evasion categories, on which an Opus 4.6 monitor with extended thinking catches 32% overall and never catches 20 of the 40 at a 1% false-positive rate over ten trials.
AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents	ICSE 2026	Defines a trigger, predicate, and enforcement-action rule language that prevents over 90% of unsafe code-agent executions and all hazardous embodied actions at millisecond overhead.
PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors	Preprint 2026	Induces typed-step adapters offline and then trains prefix-risk scorers for online warnings, reaching 0.900 AUPRC on WebArena and gaining 0.137 on average over raw-text controls across four benchmarks.
AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security	Preprint 2026	Trains four open-weight guard models from 0.8B to 8B parameters on about 1,000 taxonomy-guided samples and deploys them as a training-free online guardrail.
AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection	Preprint 2025	Reconstructs traces as control-flow, data-flow, and program-dependence graphs and type-checks policy over them, cutting AgentDojo attack success to 3 percent at 1 percent utility cost.
AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations	USENIX Security 2026	Replays each proposed tool call under control-attenuated counterfactual views and blocks calls that vanish without the untrusted content, reaching 0% attack success across four AgentDojo suites at about 3% utility cost and holding adaptive attacks to single-digit success rates.
AgentAbstain: Do LLM Agents Know When Not to Act?	Preprint 2026	Pairs 263 act and abstain tasks across 42 executable environments, on which the best of 17 frontier models reaches 59.5% paired accuracy and some agents act irreversibly before recognizing a reason to stop.
Enforcing Temporal Constraints for LLM Agents	Preprint 2025	Compiles temporal safety properties to first-order logic and enforces them at generation time with an SMT solver, raising conformance to 100% for two frontier models while also raising utility.
Online Monitoring and Corrective Steering of Programming Agents	Preprint 2026	Combines a deterministic rule-based trajectory monitor with an advisor model consulted only on drift, gaining up to 15.2% resolution rate on SWE-bench Verified and Pro at $0.08 per instance.
HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?	Preprint 2026	Hides human-validated blockers in software-engineering and text-to-SQL tasks and scores selective escalation with an Ask-F1 metric.

Guardrail Tools

[Tool] NeMo Guardrails (NVIDIA-NeMo/Guardrails): an open-source toolkit that adds programmable rails to LLM applications and agents, intercepting requests to enforce Colang-defined policies and validate tool inputs and outputs before and after a call. Apache-2.0, 2023-present. [Paper] (EMNLP 2023 Demo)

[Tool] Guardrails AI (guardrails-ai/guardrails): an open-source Python framework that runs input and output guards in an application, composing validators from Guardrails Hub to detect and mitigate risks such as toxicity, PII exposure, and hallucination. Apache-2.0, 2023-present.

[Tool] LlamaFirewall (meta-llama/PurpleLlama): a layered guardrail framework combining PromptGuard 2 for jailbreak and injection detection, AlignmentCheck as an experimental chain-of-thought auditor that inspects agent reasoning for goal hijacking, and CodeShield static analysis across 8 languages for insecure generated code. Llama 3.2 Community License, which restricts redistribution and is not an OSI-approved open-source license, 2023-present. [Paper] (Preprint 2025)

---

Audit Trails and Decision Records

Resource	Venue	Summary	Links
Decision Provenance: Harnessing Data Flow for Accountable Systems	IEEE Access 2019	Uses data-flow lineage to expose decision pipelines, including inputs, decisions, and downstream effects, as a basis for accountability and audit.
Audit Trails for Accountability in Large Language Models	Preprint 2026	Proposes lifecycle audit trails that record technical lineage and governance decisions (approvals, waivers, attestations) in tamper-evident logs for LLMs in high-stakes use.	[Code]
Responsible Agentic AI Requires Explicit Provenance	Preprint 2026	Argues that responsibility becomes assignable only when agents emit traceable, attributable records across the lifecycle, and formalizes a causal attribution function and responsibility tensor.
Auditable Agents	ACM AI Leadership Summit 2026	Defines five auditability dimensions for tool-using agents (action recoverability, lifecycle coverage, policy checkability, responsibility attribution, evidence integrity) and an Auditability Card.
Towards Security-Auditable LLM Agents: A Unified Graph Representation	Preprint 2026	Proposes Agent-BOM, a unified hierarchical attributed graph over agent execution that captures capability bindings, cognitive-state evolution, memory contamination, and cross-agent risk propagation as one structured record for post-hoc security auditing.
The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems	Preprint 2026	Treats an append-only event log as the canonical record and derives the working graph from that log, supporting replay, forks from prior events, and end-to-end lineage for each run.	[Code]
TraceAegis: Securing LLM-Based Agents via Hierarchical and Behavioral Anomaly Detection	Preprint 2025	Builds lineage records from agent execution traces and abstracts them into behavioral rules; ships TraceAegis-Bench with labeled benign and abnormal traces.
Agent-Sentry: Bounding LLM Agents via Execution Provenance	Preprint 2026	Records execution lineage from prior legitimate runs and uses it to bound and account for an agent's later actions at runtime.
From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents	Preprint 2026	Surveys evidence tracing and execution lineage for process-level accountability, organizing trace sources, evidence units, and lineage relations for auditable agent systems.	[Paper List]
MemLineage: Lineage-Guided Enforcement for LLM Agent Memory	Preprint 2026	Signs each memory entry per principal over an RFC 6962 Merkle log and refuses sensitive actions whose active justification descends from external content, driving attack success to zero on three memory-poisoning workloads and six AgentDojo banking pairs at sub-millisecond overhead.
TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectories	Preprint 2026	Leaves action choice distortion-free on a selection channel and keys a tally channel on log structure alone, so a reseller who can rewrite the log cannot erase attribution; detection survives deletion of 70% of steps.

Decision-Record Tools

[TypeScript] MakerChecker (makerchecker/MakerChecker): self-hosted governance for AI agents with role-based execution, human approval gates, and a hash-chained, Ed25519-signed audit log of every action. AGPL-3.0, 2026-present.

[TypeScript] aegis (Justin0504/Aegis): runtime policy enforcement for AI agents with a cryptographic audit trail, human-in-the-loop approvals, and a kill switch, implemented as a wrapper around agent code. MIT-licensed, 2026-present.

[Python] halo-record (bkuan001/halo-record): dependency-free recorder that seals tool calls, model calls, and approvals into a SHA-256 hash-chained JSONL log verifiable without trusting the operator, with optional RFC 3161 timestamping and OpenTelemetry GenAI span ingestion. Apache-2.0, 2026-present.

[Python, TypeScript] Agent Governance Toolkit (microsoft/agent-governance-toolkit): policy enforcement, agent identity, and execution sandboxing with a Merkle-chained audit trail and a Decision BOM, published with control mappings to the OWASP Agentic Top 10, NIST AI RMF 1.0, the EU AI Act, and SOC 2. MIT-licensed, 2026-present; public preview with breaking changes expected.

[Python] TRACE (agentrust-io/trace-spec): open specification and Python SDK for hardware-attested agent trust records that bind model, policy, data class, tool calls, and TEE attestation into one signed artifact a third party can verify offline, with a conformance test suite. CC BY 4.0 spec, Apache-2.0 tooling, 2026-present.

[TypeScript, Python] AgentLens (agentkitai/agentlens): MCP-native observability and audit-trail platform that records LLM calls, tool invocations, and decisions in an append-only, SHA-256 hash-chained, verifiable event log. MIT-licensed, 2026-present.

[Python] auditable (yzhao062/auditable): records which inputs a consequential agent action relied on, re-evaluates that action against current state, and reverses the committed action through a pluggable rail when its supporting conditions no longer hold. Apache-2.0, 2026-present.

[Python] Proofline (ceodaradigu/proofline-agent): verification-first agent that turns task contracts into deterministic SHA-256 proof packets, rejects missing, stale, or contradictory evidence, and gates external actions behind human approval. MIT-licensed, 2026-present.

---

Security Auditing and Scanners

Resource	Venue	Summary	Links
Agent Audit: A Security Analysis System for LLM Agent Applications	CAIS 2026	Static security analysis for LLM agent code and configuration, with tool-boundary taint tracking and MCP config auditing; the tool itself is listed under Scanners.
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents	ACL Findings 2024	A benchmark of 1,054 cases that measures how often tool-integrated LLM agents follow malicious instructions embedded in tool outputs.	[Code]
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents	NeurIPS 2024 Datasets and Benchmarks Track	An evaluation environment with 97 tasks and 629 security tests for prompt-injection attacks and defenses on agents that run tools over untrusted data.	[Code]
SoK: The Attack Surface of Agentic AI: Tools, and Autonomy	Preprint 2026	A systematization of security risks and attack vectors in agentic systems that combine language models with tools, retrieval, and autonomous decision loops.
Defeating Prompt Injections by Design	SaTML 2026	Extracts the control and data flow from the trusted query so untrusted retrieved data cannot affect the program flow, and enforces capability-based policies when tools are called, solving 77% of AgentDojo tasks with provable security against 84% for an undefended system.	[Code]
Design Patterns for Securing LLM Agents against Prompt Injections	Preprint 2025	Presents six design patterns that constrain a tool-using agent once it ingests untrusted input, with a utility and security trade-off analysis for each.
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents	ICLR 2025	Formalizes 10 prompt-injection attacks, memory poisoning, a Plan-of-Thought backdoor, 4 mixed attacks, and 11 defenses across 13 backbones, 10 scenarios, and over 400 tools, reaching a peak average attack success rate of 84.30 percent.
Memory Injection Attacks on LLM Agents via Query-Only Interaction	NeurIPS 2025	Shows that MINJA injects malicious records through ordinary queries and observations alone, with injection success above 90% in most configurations across three agents.
MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair	Preprint 2026	Follows 310 memory-poisoning cases from 48 contexts through write, execute, and forget stages across 24 harness, backend, and model combinations, reporting 84.2% persistence and 50.3% full-chain success, with selective repair at 56.1% among successfully poisoned cases.
StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection	Preprint 2026	Decomposes an adversarial goal into innocuous sub-steps along the navigation path, raising attack success on 3 of 6 computer-use agents by up to 31.2 points across 480 examples.
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents	Preprint 2026	Introduces StakeBench, which scores prompt-injection harm by the affected stakeholder (user, seller, platform) over 264 executable cases from 22 templates on a live e-commerce environment, where indirect injection succeeds 41.67% to 68.16% of the time and the same agent shows a distinct failure profile per stakeholder.	[Code]

MCP and Skill Supply Chain

Risks that arrive through the tool and skill supply chain rather than through the agent's own reasoning.

Resource	Venue	Summary	Links
Model Context Protocol Threat Modeling and Analyzing Vulnerabilities to Prompt Injection with Tool Poisoning	J. Cybersecur. Priv. 2026	Threat model for Model Context Protocol clients showing how unvalidated tool metadata enables prompt injection and tool poisoning, with static-analysis and behavioral defenses.	[Code]
Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystem	Preprint 2025	A survey of security and safety issues across the Model Context Protocol ecosystem, including tool poisoning, supply-chain risk, and proposed mitigations.
Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability	Preprint 2026	Builds MCPZoo from 64,611 unique MCP servers, over 37,288 of which support dynamic analysis, and finds existing scanners flag 96.89% of servers as risky at 45.53% average alert precision.
Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications	Preprint 2026	Finds with structure-aware static analysis that 9.93% of 19,200 description-code pairs from 2,214 real servers misstate what the tool code does.
A First Measurement Study on Authentication Security in Real-World Remote MCP Servers	Preprint 2026	Measures 7,973 live remote servers, of which 40.55% expose tools with no authentication, and reports 325 flaws and 9 CVE identifiers from 119 OAuth-enabled servers.
SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse	Preprint 2026	Audits skill-package reuse across expression, implementation, and operational traces, reaching AUROC 0.938 and F1 0.898 on 820 transformed positives against 751 negative controls, then audits 36,446 marketplace skills.
Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware	Preprint 2026	Shows self-extracting skill packing bypasses all 8 tested scanners at over 90% across 1,613 in-the-wild malicious skills, while its sandbox auditor detects 97% of benchmark attacks at a 2% false-positive rate and 87% of real-world malicious skills.
OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills	Preprint 2026	Collects 263 marketplace skills across 7 threat categories, on which the safest of 3 frameworks and 13 models still executes unsafe actions about 17% of the time.

Scanners

[Python] agent-audit (HeadyZhang/agent-audit): static security scanner for LLM agents that flags prompt-injection sinks, audits MCP configuration, and runs tool-boundary taint analysis. Its rules map to the OWASP Agentic Top 10 (2026), and it supports LangChain, CrewAI, and AutoGen. MIT-licensed, 2026-present.

[Python] garak (NVIDIA/garak): an LLM vulnerability scanner that probes a model or any REST-accessible endpoint for prompt injection, jailbreaks, data leakage, and toxic generation, with detectors that score each probe. Apache-2.0, 2023-present.

[Python] Agentic Radar (splx-ai/agentic-radar): open-source scanner that maps agentic workflows through static analysis, detects MCP servers and tools, and reports OWASP-aligned vulnerabilities. It supports LangGraph, CrewAI, OpenAI Agents, AutoGen, and n8n. Apache-2.0, 2025 (last commit 2025-11-27).

[Python] Snyk Agent Scan (snyk/agent-scan): security scanner that auto-discovers the MCP servers, agent tools, and skills installed on a machine across 13 agent harnesses including Claude Code, Cursor, VS Code, Windsurf, Gemini CLI, and Codex. It flags more than 15 risk classes including prompt injection, tool poisoning, tool shadowing, toxic flows, malware payloads, and hardcoded secrets. Apache-2.0, 2025-present; it requires a Snyk API token, so it is an open-source client to a commercial service.

---

Datasets and Benchmarks

[Python] CatchBench (yzhao062/catchbench): benchmark for auditing agent failures from declared configurations before a run, growing trace prefixes during a run, and complete traces after a run, with seven boards over five scenarios for over-privilege, fault localization, failure detection, cause attribution, injected-fault localization, streaming early warning, and online stale-state detection. MIT-licensed code, 2026-present.

Resource	Venue	Summary	Links
TRAIL: Trace Reasoning and Agentic Issue Localization	Preprint 2025	A corpus of 148 human-annotated agent execution traces with 841 labeled errors across reasoning, execution, and planning categories, drawn from GAIA and SWE-bench tasks to support error localization in agent runs.
Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems	ICML 2025	Who&When failure logs from 127 multi-agent systems, annotated with the responsible agent and decisive error step for research on automated failure attribution.
Aegis: Automated Error Generation and Attribution for Multi-Agent Systems	ICLR 2026	A dataset of 9,533 trajectories with annotated faulty agents and error modes, generated by injecting context-aware errors into successful runs across several multi-agent architectures.
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains	ICLR 2025	Retail and airline benchmark (tau-bench) that compares final database states with goal states and reports pass^k to measure consistency across repeated trials.
ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions	Preprint 2026	A benchmark that measures agent reliability along three axes: consistency under repeated execution (pass^k), tolerance to semantically equivalent task perturbations, and fault tolerance under injected tool and API failures.
GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems	Preprint 2026	An evaluation framework that generates synthetic multi-agent interaction datasets and standardizes the comparison of graph-based anomaly detectors over agent communication graphs.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?	ICLR 2024	2,294 software-engineering tasks drawn from real GitHub issues and their fix pull requests across 12 Python repositories, with unit-test verification of each generated patch.	[Code]
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering	NeurIPS 2024	An open-source coding-agent framework with released SWE-bench trajectories that contain step-by-step tool calls and edits for trace-level study.	[Code]
AgentBench: Evaluating LLMs as Agents	ICLR 2024	A suite of eight interactive environments for measuring multi-turn decision-making of LLM agents, with logged interaction records across many models.	[Code]
GAIA: a benchmark for General AI Assistants	ICLR 2024	466 real-world questions that require multi-step reasoning, web browsing, and tool use, with a single correct answer per item for unambiguous scoring of assistant behavior.	[Data]
WebArena: A Realistic Web Environment for Building Autonomous Agents	ICLR 2024	A reproducible environment of self-hosted websites with long-horizon tasks scored by functional correctness of the end state, supporting repeatable agent evaluation.	[Code]
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments	NeurIPS 2024 Datasets and Benchmarks Track	369 executable computer-use tasks on Ubuntu, each with a setup configuration and an execution-based checker, in an environment that also supports Windows and macOS, on which humans complete over 72.36% of tasks while the best evaluated model reaches 12.24%.	[Code]
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces	ICLR 2026	Terminal-Bench 2.0 curates 89 command-line tasks, each with its own container environment, a human-written solution, and verification tests, on which frontier models and agents score below 65%.	[Code]
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents	ACL 2024	750 interactive coding tasks over 9 simulated apps exposing 457 APIs, on which GPT-4o solves about 49% of normal tasks and 30% of challenge tasks.
Identifying the Risks of LM Agents with an LM-Emulated Sandbox	ICLR 2024	ToolEmu emulates tool execution so agents can be stress-tested on 144 cases without real side effects; the safest agent still fails 23.9% of the time and 68.8% of flagged failures are judged valid.
τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment	ICML 2026	Extends τ-bench to dual control where the user also acts on the shared environment, modeling Telecom as a Dec-POMDP and separating agent reasoning errors from coordination errors.
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents	NeurIPS 2024 Datasets and Benchmarks Track	Nine partially observable environments scored by a fine-grained progress rate that credits incremental advancement rather than final success alone.
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation	Preprint 2026	60 bilingual multimodal tasks inside real command-line harnesses in Docker, on which the best of 19 models reaches 62.2% and swapping the harness alone moves one model by up to 18 points.
ClawBench: Can AI Agents Complete Everyday Online Tasks?	Preprint 2026	Evaluates browser agents on 153 write-heavy tasks across 144 live websites, with final-request interception and synchronized video, screenshot, network, action, and message traces for auditable scoring and failure analysis.	[Code], [Project]
Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions	ICLR 2026	MemoryAgentBench recasts long-context data into incremental multi-turn interactions testing accurate retrieval, test-time learning, long-range understanding, and selective forgetting; no agent class masters all four.
HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark	Preprint 2026	629 trajectories (523 risky, 106 safe) averaging 33 steps that audit risk arising under benign conditions; strong models detect trajectory-level risk but fall below 35 Strict-F1 on locating the risky step.
OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents	NeurIPS 2025 Datasets and Benchmarks Track (Spotlight)	150 OSWorld-based tasks spanning deliberate misuse, prompt injection, and model misbehavior, with a judge agreeing with human annotation at 0.76 and 0.79 F1.
R-Judge: Benchmarking Safety Risk Awareness for LLM Agents	EMNLP Findings 2024	569 multi-turn records across 27 risk scenarios and 10 risk types, on which GPT-4o reaches 74.42% while no other evaluated model significantly beats random.
TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis?	Preprint 2026	Compares metadata, OpenTelemetry-compatible, and OpenInference-compatible views, which retain 99.5% to 100% detection F1 while holding fault-origin step accuracy at or below 0.5%.

Several entries above cross-reference a paper listed in a topical section elsewhere; they are repeated here so the reader can find the datasets in one place.

Evaluation Integrity (Judges and Benchmark Gaming)

Work on whether the measurement instrument itself can be trusted: benchmarks that can be gamed without solving the task, and automatic judges that disagree with human review.

Resource	Venue	Summary	Links
Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack	Preprint 2026	Red-teams 10 agent benchmarks for reward-hacking exploits, surfacing 219 distinct flaws across eight recurring classes and scoring near-perfect on most benchmarks without solving a single task, then patches four of them below a 10% hackable-task ratio and fully repairs WebArena and OSWorld in three iterations.	[Code]
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories	COLM 2025	1,302 web agent trajectories drawn from 5 benchmarks and 4 language models, each reviewed by an expert for success, side effects, and repetitiveness, used to score 12 language-model judges and to show that the rule-based evaluation of common benchmarks underreports agent success.	[Code]
AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents	NeurIPS 2025	Training-free memory-augmented framework that lifts language-model safety evaluators toward human-expert accuracy, released with ASSEBench: 2,293 annotated records, 15 risk types, and 29 scenarios.
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents	Preprint 2026	Separates visible validation tests from held-out compositional tests across 30 systems-level tasks, finding the pass-rate gap grows 28 percentage points per tenfold increase in code size.
AI Agents That Matter	TMLR 2025	Critique of agent benchmarking practice, arguing for cost-aware evaluation, adequate holdout sets, and reproducibility alongside accuracy.	[Code]

---

Tools and Platforms

Tools and platforms for agent observability, evaluation, and reliability. An inspectable public source repository is the inclusion bar; the one exception is marked [Managed] and its closed-source status is stated in the entry. A license and activity note follows each entry.

[Python, TypeScript] Langfuse (langfuse/langfuse): self-hostable platform for tracing LLM and agent calls, running evaluations, managing prompts, and tracking cost and latency, with OpenTelemetry, LangChain, and OpenAI SDK integrations. MIT-licensed core, 2023-present.

[Python] Arize Phoenix (Arize-ai/phoenix): open-source observability and evaluation tool built on OpenTelemetry for tracing, evaluating, and debugging LLM and agent applications, with auto-instrumentation for common frameworks. Elastic License 2.0, 2022-present.

[Python] OpenInference (Arize-ai/openinference): a set of OpenTelemetry-compatible conventions and instrumentation packages for capturing traces from LLM and agent frameworks; exports traces to OpenTelemetry-compatible backends. Apache-2.0, 2023-present.

[Python] Opik (comet-ml/opik): open-source platform for tracing, evaluating, and monitoring LLM applications and agentic workflows, with LLM-as-a-judge scoring, experiment tracking, and production dashboards. Apache-2.0, 2024-present.

[Python] OpenLLMetry (traceloop/openllmetry): OpenTelemetry-based instrumentation for LLM applications that emits standard OpenTelemetry traces routable to existing observability backends; SDKs across Python, TypeScript, Go, and Ruby. Apache-2.0, 2023-present.

[Python] Helicone (Helicone/helicone): open-source observability platform and AI gateway that logs LLM and agent traces, costs, and latency through a proxy and a unified API across providers. Apache-2.0, 2023-present.

[Python] DeepEval (confident-ai/deepeval): open-source evaluation framework that runs LLM and agent tests in a pytest-style workflow, with metrics for hallucination, relevancy, and task correctness that can be included in CI checks. Apache-2.0, 2023-present.

[Python] Evidently (evidentlyai/evidently): open-source framework to evaluate, test, and monitor ML and LLM systems, with 100+ built-in metrics spanning tabular data quality, data drift, and LLM judges. Apache-2.0, 2020-present.

[Python] AgentOps (AgentOps-AI/agentops): open-source SDK for monitoring AI agents, with session replays, cost tracking, and failure detection; integrates with CrewAI, OpenAI Agents SDK, LangChain, and AG2 (formerly AutoGen). MIT-licensed, 2023-present.

[Managed] LangSmith (langchain.com/langsmith-platform): framework-agnostic platform from the LangChain team for tracing, offline and online evaluation, and monitoring of LLM and agent applications. Commercial product, not open source; self-hosted and hybrid deployment are available on the Enterprise plan, 2023-present.

[Python] Ragas (vibrantlabsai/ragas): evaluation library for LLM applications, with RAG metrics such as faithfulness, answer relevancy, and context precision, agent metrics such as tool-call accuracy and topic adherence, and test set generation. Apache-2.0, 2023-2026 (last commit 2026-02-24, latest release v0.4.3 on 2026-01-13). [Paper] (EACL 2024 Demo)

[Python] Inspect (UKGovernmentBEIS/inspect_ai): evaluation framework from the UK AI Security Institute, with a built-in ReAct agent, per-run token, message, and time limits, approval policies that route a tool call to a human before execution, and sandboxed tool execution on Docker, Kubernetes, Modal, or EC2; a companion collection ships over 200 pre-built evaluations. MIT-licensed, 2023-present.

[Python] AgentDebugX (AgentDebugX/AgentDebugX): closes the debugging loop as detect, attribute, recover, and rerun, reaching 28.8% exact agent-and-step accuracy on Who&When with a qwen3.5-9b backbone against 21.7% for the strongest single-pass baseline and repairing 13 of 73 failed GAIA tasks in one rerun. MIT-licensed, 2026-present. [Paper] (Preprint 2026)

[Python] Docent (TransluceAI/docent): summarizes, searches, and clusters completed agent transcripts to surface broken tasks; adding packages that Docent revealed as missing raised the GPT-4o InterCode solve rate from 68.6% to 78% across 79 tasks. Apache-2.0, 2025-present.

[Python, TypeScript] Laminar (lmnr-ai/lmnr): OpenTelemetry-native agent observability with Signals that track a behavior described in plain English, such as an agent stuck in a loop, and alert on it across runs, plus SQL queries over span data. Apache-2.0, 2024-present.

[Python] TruLens (truera/trulens): OpenTelemetry-native evaluation recording latency, tokens, and cost per step, with seven agent-specific evaluators covering plan adherence, plan quality, tool selection, tool calling, execution efficiency, logical consistency, and external tool reliability. MIT-licensed, 2020-present.

[Python, TypeScript] E2B (e2b-dev/E2B): runs AI-generated code in isolated cloud sandboxes so an unsafe action is contained rather than executed on the host, with Terraform-based self-hosting. Apache-2.0, 2023-present.

[Rust, Python] Microsandbox (superradcompany/microsandbox): self-hosted microVM runtime giving agent code hardware-level isolation at an average boot under 100 milliseconds, spawned in-process with no daemon, with SDKs in five languages. Apache-2.0, 2024-present.

[Python] Giskard (Giskard-AI/giskard-oss): generates adversarial test suites from a plain-language description of the agent across OWASP LLM Top-10 threat categories, with multi-turn agent testing in v3. Apache-2.0, 2022-present.

[Python] AgentRunProof (FU-max-boop/agentrunproof): deterministic runtime-conformance harness for the OpenAI Agents SDK that drives the real Runner through scripted responses on the public Model interface, and writes content-addressed evidence that can be rechecked without rerunning the SDK; built-in scenarios make no model API calls. MIT-licensed, 2026-present.

[Python] A2E (datamllab/A2E): end-to-end agent auditing engine that uses the Agent Task Protocol to integrate tasks with different harnesses, automatically captures standardized execution traces, and assesses execution efficiency, tool use, task planning, and error recovery with multidimensional metrics. MIT-licensed, 2026-present. [Paper] (Preprint 2026)

[JavaScript] Awesome Agentic Engineering (lindixu6-hash/awesome-agentic-engineering): GitHub Action and CLI that scores an agent specification against risk-tiered profiles, fails a release on unresolved blockers, and runs eight prompt-injection fixtures through LangGraph and OpenAI Agents adapters that retain node-linked traces and machine-readable results. MIT-licensed, 2026-present.

[Python] AgentCheck (WaseemGhanem98/AgentCheck): open-source behavioral testing tool for tool-using AI agents that simulates declared tool execution while evaluating tool calls, failures, retries, confirmations, destructive actions, action sequencing, and fabricated success, with integrations for OpenAI Agents SDK, PydanticAI, and custom Python agents. Apache-2.0, 2026-present.

---

Standards and Governance

Standards and governance instruments for agent reliability and accountability. Papers appear first, then the standards and frameworks.

Resource	Venue	Summary	Links
Model Cards for Model Reporting	FAT* 2019	Proposes short standardized documents that report a model's intended use, evaluation conditions, and performance across groups as accountability artifacts for deployed models.
Datasheets for Datasets	CACM 2021	Introduces a standard datasheet for documenting a dataset's motivation, composition, collection process, and recommended uses to improve transparency and accountability.
Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing	FAT* 2020	Defines SMACTR, an internal audit framework that produces a documented decision trail across the development lifecycle so teams can assess systems before deployment.
Black-Box Access is Insufficient for Rigorous AI Audits	FAccT 2024	Argues that meaningful third-party AI audits need more than query access, comparing black-box, white-box, and outside-the-box methods and their accountability implications.
Position: AI Agents Need Authenticated Delegation	ICML 2025 Position Paper Track	Extends OAuth 2.0 and OpenID Connect with agent-specific credentials to scope permissions delegated to autonomous agents and link those permissions to a responsible human.	[Paper]
AI Identity: Standards, Gaps, and Research Directions for AI Agents	Preprint 2026	Surveys current identity, delegation, and accountability standards for agents acting across organizational boundaries and maps the open gaps that current frameworks leave unaddressed.

Standards and Frameworks

[Standard] Model Context Protocol (MCP) Specification (modelcontextprotocol.io, spec repo): JSON-RPC 2.0 protocol that standardizes how applications expose tools, resources, and prompts to language models, with an optional OAuth 2.1 authorization profile requiring Protected Resource Metadata discovery (RFC 9728), resource-indicator audience binding (RFC 8707), and issuer validation (RFC 9207) on HTTP transports. Agentic AI Foundation (a directed fund under the Linux Foundation), revision 2026-07-28.

[Standard] Agent2Agent (A2A) Protocol (a2a-protocol.org): task delegation between independent agents, where each server publishes an Agent Card declaring identity, capabilities, skills, endpoint, and authentication across API key, HTTP, OAuth 2.0, OpenID Connect, and mutual TLS. Linux Foundation, contributed by Google, v1.0.1 (28 May 2026).

[Standard] Agent Payments Protocol (AP2) (ap2-protocol.org): users sign Checkout and Payment Mandates as verifiable credentials, with an open stage carrying spending constraints and a closed stage carrying a specific authorization, so each agent-initiated transaction leaves a non-repudiable cryptographic record. FIDO Alliance, donated by Google, v0.2.

[Standard] OpenTelemetry Semantic Conventions for Generative AI (gen-ai) (semantic-conventions-genai): vendor-neutral telemetry schema for GenAI and agent spans, metrics, and events, covering model calls, tool calls, token usage, and agent and framework operations to support comparison of traces across stacks. OpenTelemetry / CNCF; all gen_ai.* attributes were deprecated in the core semantic-conventions repository at v1.42.0 (12 June 2026) and moved to this dedicated repository.

[Standard] NIST AI 600-1, Generative AI Profile (nvlpubs.nist.gov): cross-sector companion to the NIST AI RMF that names twelve GenAI risk areas and over 200 suggested actions across the Govern, Map, Measure, and Manage functions. NIST, July 2024.

[Standard] NIST AI 100-1, AI Risk Management Framework (AI RMF 1.0) (nvlpubs.nist.gov): the base voluntary framework that organizes AI trustworthiness and risk work into the Govern, Map, Measure, and Manage functions; the parent document the GenAI profile extends. NIST, January 2023.

[Standard] EU AI Act (Regulation 2024/1689), Article 12: Record-Keeping (artificialintelligenceact.eu): legal duty requiring high-risk AI systems to record events throughout their lifetime for risk identification, post-market monitoring, and operational oversight. Article 26(6) requires deployers to retain the logs for at least six months. EU, 2024; the Digital Omnibus on AI (Regulation (EU) 2026/1744, in force 27 July 2026) moved high-risk application to 2 December 2027 for Annex III areas and 2 August 2028 for AI embedded in regulated products.

[Standard] ISO/IEC 42001:2023, Artificial Intelligence Management System (iso.org): first certifiable AI management system standard, setting requirements to establish, operate, and improve governance controls across the AI lifecycle, including risk assessment, impact assessment, and performance measurement. ISO/IEC JTC 1/SC 42, 2023.

[Standard] MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) (atlas.mitre.org, atlas-data v2026.07): living knowledge base of adversary tactics and techniques against AI systems, modeled on MITRE ATT&CK and built from real-world attacks and red-team demonstrations. MITRE, v2026.07 (7 August 2026): 16 tactics, 101 techniques, 77 sub-techniques, 37 mitigations, and 68 case studies, adding three AI Agent Tool Poisoning sub-techniques (AML.T0110.000/.001/.002) and AML.T0115.

[Standard] OWASP Top 10 for LLM Applications and OWASP Top 10 for Agentic Applications (genai.owasp.org, agentic 2026, securing agentic applications 1.0): community-maintained risk lists for language-model and agentic applications, covering prompt injection, excessive agency, tool misuse, memory and context poisoning, and goal hijacking, with a builder-side companion guide of technical recommendations. OWASP, 2025 (LLM) / 2026 (Agentic).

[Standard] CWE-1427: Improper Neutralization of Input Used for LLM Prompting (cwe.mitre.org): base-level weakness entry cataloging prompt injection as a mappable software weakness, with a LangChain tool-agent demonstrative example. MITRE CWE 4.20, mapping status ALLOWED.

[Framework] Agentic AI Threat Modeling Framework: MAESTRO (cloudsecurityalliance.org): seven-layer threat-modeling method a builder can run against an agent architecture. Cloud Security Alliance, 2025; a single-author blog publication rather than a ratified standard, so it sits at a lower evidentiary register than the NIST, ISO, MITRE, and OWASP entries above.

[Standard] C2PA Technical Specification / Content Credentials (spec.c2pa.org, c2pa-rs): cryptographically signed, tamper-evident metadata standard that records the origin and edit history of media, including a manifest for AI-generated and AI-edited content. C2PA (Adobe, Microsoft, BBC, Intel, Truepic, Sony, and others), v2.2 (2025).

The five entries below are the primary sources for the verifiable-log and attestation patterns that several tools in this list build on.

[Standard] Certificate Transparency (RFC 6962, RFC 9162) (RFC 6962, RFC 9162): append-only Merkle tree log with inclusion proofs, showing an entry is in the log, and consistency proofs, showing the log was only appended to; the origin of the verifiable-log pattern that tamper-evident agent trails reuse. IETF, 2013 (v1) / 2021 (v2).

[Standard] in-toto Attestation Framework (in-toto/attestation, in-toto.io): signed, machine-readable statements binding a subject artifact to a predicate describing what was done to it, giving a common envelope for provenance claims across steps and tools. in-toto / CNCF, 2018-2026 (living spec).

[Standard] SLSA (Supply-chain Levels for Software Artifacts) (slsa.dev): graded framework defining what provenance a build must produce and how tamper-resistant it must be, with each level naming concrete requirements rather than intent. OpenSSF, v1.0 (2023).

[Standard] DSSE (Dead Simple Signing Envelope) (secure-systems-lab/dsse): signing envelope that authenticates a payload together with its type, removing the canonicalization ambiguity that lets a signature be replayed against a different interpretation of the same bytes; used by in-toto and Sigstore. Secure Systems Lab, 2021-2026.

[Tool] Rekor (sigstore/rekor): transparency log service for signed software artifacts and attestations, built on an append-only Merkle log and serving inclusion and consistency proofs over a public API. Sigstore / OpenSSF, Apache-2.0, 2020-present.

---

Contributing

More items will be added to this repository. Please suggest other resources by opening an issue, submitting a pull request, or dropping me an email at yzhao010@usc.edu. New entries should be real, with a working link to the paper, repository, or specification.

Contributors

People outside the lab have added entries to this list. Each of those went through a pull request, met the inclusion bar in CONTRIBUTING.md, and is checked by the same link and citation audit as everything else here.

The strip reads from the repository itself, so it stays current without anyone maintaining a list by hand.

---

Maintained By

Maintained by Yue Zhao, University of Southern California, USC-FORTIS lab. He created PyOD and co-authored ADBench. His publication list is on Google Scholar.

---

Citation

If this list is useful in your work, cite the Auditable Agents framework paper:

@misc{nian2026auditable,
  title         = {Auditable Agents},
  author        = {Nian, Yi and Yuan, Aojie and Zhang, Haiyue and Li, Jiate and Zhao, Yue},
  year          = {2026},
  eprint        = {2604.05485},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  note          = {Accepted to the ACM AI Leadership Summit 2026},
  url           = {https://arxiv.org/abs/2604.05485}
}

To cite the list itself as a resource: