agent-harness-landscape.md

title	Agent Harness Landscape
description	An evidence-backed map of coding-agent runtimes, repository harnesses, orchestrators, frameworks, sandboxes, memory, evaluation, and observability projects.
tags	agents harness comparison runtimes orchestrators agent-tools deepseek-harness codex claude-code

Agent Harness Landscape

Use this map to separate four questions that product lists often merge: which model generates, which runtime owns the tool loop, which repository configuration controls local behavior, and which orchestrator coordinates multiple runs. A project can be valuable without being a runtime harness.

The broad directory below normalizes the 160 projects and 12 categories in Best of Agent Harnesses, then adds 32 guide supplements discovered through direct project research. It uses the upstream snapshot at commit ece3146, dated 2026-08-23. The strict map applies an additional test: does the product own the cycle that plans, acts through tools, observes results, and decides what happens next?

An agent harness is the runtime around a model that assembles context, exposes tools, applies permissions, executes the action loop, records state, and handles failure. Simon Willison's concise definition, "models using tools in a loop", identifies the behavioral boundary. The 2026 Agent System and Harness Design survey expands that runtime into six responsibilities: observation, context, control, action, state, and verification. The SWE-agent paper names the coding-specific interface between the model and computer the agent-computer interface.

Comparison requires a second boundary: every outcome belongs to a model-harness pair under a disclosed task set and budget. In a controlled 300-run study across two models and three harnesses, The Scaffold Effect found up to a 40-times difference in tokens per solved task, while pass-rate differences stayed within 0 to 8 percentage points and were mostly not statistically significant. A harness can strongly affect cost and failure behavior without being the main accuracy bottleneck.

This page answers which layer and which project. Agent Harness Engineering documents the components inside a runtime harness. Loop & Graph Engineering covers feedback, topology, state transitions, stopping rules, and judgment boundaries. Agent Tools: Beyond Claude Code provides deeper product profiles. The glossary separates runtime harnesses from repository harnesses and evaluation harnesses.

Choose the right entry point

Question	Canonical page
How does the loop, context, tool, hook, permission, and recovery machinery work?	Agent Harness Engineering
How should a loop or executable workflow graph be designed and bounded?	Loop & Graph Engineering
Which runtime or adjacent project fits a particular job?	This Agent Harness Landscape page
What does a named coding-agent product support in practice?	Agent Tools: Beyond Claude Code
How should a repository prepare instructions, setup, state, and verification for any runtime?	Repository Harness Engineering
How should a shortlist be evaluated and instrumented?	Agent Evaluation and Observability
Which systems optimize a harness rather than run tasks directly?	Harness Optimizers and Meta-Harnesses

These pages are linked but deliberately not merged. The engineering reference explains stable mechanisms. This map is a dated evidence snapshot whose projects, licences, features, and GitHub signals need a separate refresh cycle.

160 Projects Does Not Mean 160 Runtime Harnesses

The upstream catalog mixes ready-to-run coding agents with SDKs, frameworks, memory systems, sandboxes, evaluation tools, observability products, prompt libraries, and multi-agent control planes. Treating all 160 as interchangeable runtimes produces invalid comparisons. The strict map retains projects with evidence that they own an agent loop; adjacent products remain in the directory under their actual job.

The 160-project and 12-category figures come from Best of Agent Harnesses, snapshot 2026-08-23. Catalog size is not runtime count.

The Twelve-Category Map

The category names come from the pinned upstream snapshot. Usually, Sometimes, and No describe the category's typical relation to a loop. They do not assign one answer to every project inside it.

Category	Projects	What it contributes	Owns an agent loop?	Guide layer
Coding agent products (IDEs, CLIs, full suites)	22	Turnkey coding agents	Usually	Runtime
Coding harness configs and SDKs	17	Configuration packs and agent SDKs	Sometimes	Repository / construction
Evaluation and benchmarking harnesses	18	Benchmarks and evaluation harnesses	No	Evaluation
Frameworks	25	Libraries for building an agent loop	Sometimes	Construction
Libraries and SDKs	15	Reusable agent building blocks	No	Construction
Memory and state	5	Persistent state and retrieval	No	Memory
Multi-agent and orchestration	12	Coordination across agents or runtimes	Sometimes	Orchestrator
Observability and eval-ops	4	Tracing, quality, and operations	No	Observability
Personal agent runtimes	10	Ready-to-run personal agents	Usually	Runtime
Plugins, MCPs, CLI tools	19	Tools connected to an existing runtime	No	Extension
Progressive disclosure harnesses	8	Context and prompt-loading strategies	Sometimes	Repository / context
Research and task-specific harnesses	5	Domain-specific agents and research systems	Sometimes	Runtime / task-specific

Core Coding Harnesses

The strict map contains 42 runtimes. Every name links to its official product page or canonical repository. Open-source rows include GitHub stars when a canonical repository was available. Stars are a dated popularity signal, not a quality score. Entries with detailed coverage in this guide keep that internal profile in the role cell.

Snapshot: 2026-08-28. GitHub stars are captured on the date shown in each project cell.

Legend: ? = not established from the pinned sources; N/A = does not apply.

Harness	Interface	Provider strategy	Loop evidence	Licence	Role
AgentForge ★ 60 · 2026-08-28	?	?	Confirmed	?	The README documents a ReAct-style loop that repeats model requests, typed tool calls, and observations until completion.
agentic-harness ★ 84 · 2026-08-28	?	?	Confirmed	?	The README documents a coding-agent loop that iterates through model tool calls against a workspace.
aider ★ 48,420 · 2026-08-23	?	?	Claimed	Open source	Git-aware CLI pair programmer; edits in-repo, supports multiple models and MCP so agents see version control and tools. Guide profile
Amp	?	?	Claimed	Proprietary	Amp presents its product as an agentic coding tool.
Augment Code	?	?	Claimed	Proprietary	Augment presents its product as an AI coding platform with agent capabilities.
Autonomous Coding Harness ★ 7 · 2026-08-28	?	?	Claimed	?	The README claims a milestone-based autonomous coding loop with implementation, verification, and human checkpoints.
Claude Code	?	?	Claimed	Proprietary	Anthropic documents Claude Code as an agentic coding tool that reads codebases, edits files, and runs commands.
claw-code-agent ★ 543 · 2026-08-23	?	?	Claimed	?	Python reimplementation of the Claude Code agent architecture with zero external dependencies; interactive chat, streaming, plugin runtime, nested agent delegation, cost...
Cline ★ 66,707 · 2026-08-23	?	?	Claimed	Open source	VS Code extension whose harness is a plan-then-act loop with per-step human approval and cost transparency; the VS Code integration is the UI shell.
Codex ★ 114,837 · 2026-08-23	?	?	Claimed	Open source	OpenAI's terminal coding agent. Guide profile
crush ★ 27,601 · 2026-08-23	?	?	Claimed	Restricted (fsl-1.1-mit)	Charm's terminal coding agent (Charm's fork of the original OpenCode). Guide profile
Cursor Agent	?	?	Claimed	Proprietary	Cursor presents Agent as its coding-agent product across editor and remote surfaces.
DeepSeek Harness ★ 201,064 · 2026-08-28	?	?	Claimed	Mit	The official repository describes a developer-preview agent harness built on the Cordis plugin framework. Guide profile
DeepSeek-Reasonix ★ 35,065 · 2026-08-23	?	?	Claimed	?	DeepSeek-native terminal coding agent.
Devin	?	?	Claimed	Proprietary	Cognition presents Devin as an AI software engineer.
Factory Droid	?	?	Claimed	Proprietary	Factory presents Droids as software-development agents.
Gemini CLI ★ 106,626 · 2026-08-23	?	?	Claimed	Open source	Google's first-party terminal agent for Gemini. Guide profile
GitHub Copilot CLI	?	?	Claimed	Proprietary	GitHub presents Copilot CLI as a coding agent for the terminal.
goose ★ 53,295 · 2026-08-23	?	?	Claimed	Open source	Block-originated Rust agent, now stewarded by the Linux Foundation's Agentic AI Foundation (aaif-goose/goose). Guide profile
Hermes ★ 234,688 · 2026-08-23	?	?	Confirmed	Open source	Nous Research's self-improving agent: a learning loop turns experience into reusable skills, builds a persistent user model across sessions, and checkpoints state to disk with...
jcode ★ 18,308 · 2026-08-23	?	?	Confirmed	?	Rust terminal coding agent pitched as the most RAM-efficient harness in its class; MCP support, multi-provider (Claude/OpenAI).
Jules	?	?	Claimed	Proprietary	Google presents Jules as an asynchronous coding agent.
Junie	?	?	Claimed	Proprietary	JetBrains presents Junie as its coding agent.
Kilo Code ★ 26,978 · 2026-08-23	?	?	Claimed	?	VS Code extension and CLI in the Cline/Roo-Code lineage : a natural pick now that Roo-Code is archived upstream.
Kimi Code CLI ★ 7,126 · 2026-08-28	?	?	Claimed	Mit	The official repository describes Kimi Code CLI as an agentic coding tool for terminals and IDEs.
Kiro	?	?	Claimed	Proprietary	Kiro presents its product as an agentic development environment.
oh-my-pi ★ 26,658 · 2026-08-23	?	?	Claimed	Open source	Terminal coding agent (fork of Pi) that wires the IDE into the harness: hash-anchored edits, a 32-tool loop tuned per-model, LSP rename/references/diagnostics on every write, a...
OpenHarness (HKUDS) ★ 15,492 · 2026-08-23	?	?	Confirmed	Open source	Open agent harness with a built-in personal agent ("Ohmo") that runs across Feishu, Slack, Telegram, and Discord; core tool-use, skills, memory, multi-agent coordination with...
Open Interpreter ★ 68,121 · 2026-08-23	?	?	Claimed	Open source	Lightweight terminal coding agent oriented to open models (DeepSeek, Kimi, Qwen).
Open SWE ★ 10,624 · 2026-08-28	?	?	Claimed	Mit	The official repository describes Open SWE as an asynchronous coding agent for repository tasks.
opencode ★ 200,557 · 2026-08-23	?	?	Claimed	Open source	Open-source terminal coding agent (formerly sst/opencode; transferred to anomalyco). Guide profile
OpenCode Harness ★ 148 · 2026-08-28	?	?	Confirmed	?	The README documents a coding-agent loop with tools, permissions, traces, evaluation, and repair feedback.
OpenHands ★ 84,844 · 2026-08-23	?	?	Claimed	Restricted ((multi-license))	Dockerized software-engineering agent. Guide profile
OpenHarness ★ 12 · 2026-08-28	?	?	Confirmed	?	The README documents a provider-to-tools agent loop and runtime injection points.
pi ★ 95,747 · 2026-08-23	?	?	Claimed	?	The upstream AI agent toolkit behind this list's oh-my-pi fork: a unified multi-provider LLM API, agent loop, and TUI shell providing the harness that oh-my-pi's Rust rewrite...
qwen-code ★ 27,310 · 2026-08-23	?	?	Claimed	?	Alibaba's official terminal coding agent, forked from Gemini CLI's agent loop and retuned for Qwen models.
Replit Agent	?	?	Claimed	Proprietary	Replit presents Agent as a product that builds applications from user goals.
Roo Code ★ 24,326 · 2026-08-23	?	?	Claimed	Open source	VS Code/Cursor extension in the Cline lineage.
Spettro ★ 33 · 2026-08-28	?	?	Confirmed	?	The README documents autonomous goal runs, native tool calls, subagent workflows, and verification loops.
SWE-agent ★ 20,112 · 2026-08-23	?	?	Claimed	Open source	LM-driven harness built for SWE-bench: edit state, command execution, and issue-focused loop: the reference agent stack next to the benchmark itself. Guide profile
Warp Agent Mode	?	?	Claimed	Proprietary	Warp presents its AI surface as an agentic development environment in the terminal.
Windsurf Cascade	?	?	Claimed	Proprietary	Windsurf presents Cascade as its agentic coding assistant.

DeepSeek Harness needs a maturity caveat. The official repository describes dsh as a developer preview and warns that compatibility may change. Its plugin architecture, permissions, sandbox choices, and local-first processing do not make untrusted repositories safe by default. See the DeepSeek Harness profile for setup, architecture, and security boundaries.

Creator interviews as supplementary evidence

Official documentation and canonical repositories remain the source of truth for current product behavior. Creator interviews add dated design rationale that a feature table cannot capture:

• Boris Cherny's Building Claude Code interview traces the product's move from a chatbot to a tool-using agent, while the permission discussion explains the layered controls behind command execution.
• Dax Raad's Building OpenCode interview documents the open-source, provider-neutral runtime positioning. The later control-plane discussion separates organization-wide provider, permission, budget, and rate-limit controls from the runtime itself.

These interviews do not upgrade a generated evidence state on their own. They are dated testimony, so current availability, licence, and feature behavior still require a direct official source. The practitioner video evidence ledger records the short verbatim, timestamp, and boundary for each source.

Orchestrators: Products Above the Runtime

The adjacent map contains 15 control planes. An orchestrator coordinates queues, isolated workspaces, or multiple agent sessions. Use one after a single runtime is no longer the bottleneck. Parallel runs multiply throughput, context drift, and review load at the same time.

The runtime/control-plane distinction also appears in Dax Raad's May 2026 OpenCode interview. He describes organization-wide provider setup, permissions, budgets, and rate limits as a separate control plane. That supports the architectural boundary used here, but it is not evidence that the enterprise product described in the interview is publicly available today.

Control plane	Role	Loop ownership	Licence
AgentBox ★ 374 · 2026-08-23	Runs multiple coding agents in parallel, each in its own sandboxed VM, locally or in the cloud, from one command.	No; evidence claimed	Open source
AgentsMesh ★ 2,330 · 2026-08-28	The README documents a control-plane and data-plane system for operating external coding agents.	No; evidence confirmed	?
Autonomous Workstream ★ 32 · 2026-08-28	The README documents a plugin and CLI that sequences bounded tasks through an external coding-agent backend.	No; evidence confirmed	?
Harness ★ 10 · 2026-08-28	The README documents hardened container environments for agents rather than an agent decision loop.	No; evidence confirmed	?
cc-haha ★ 14,189 · 2026-08-23	Local-first desktop workspace harness for Claude Code and other agents: multi-agent sessions, Git worktrees, code diffs, a skill marketplace, and chat-app access (WeChat...	No; evidence claimed	?
eigent ★ 15,083 · 2026-08-23	Open-source desktop harness positioned as a local, free alternative to Claude Cowork and Codex: multi-agent workspace orchestration in a self-hosted app rather than a hosted...	No; evidence claimed	?
Agent Harness ★ 7 · 2026-08-28	The README explicitly describes an adapter-driven control plane that delegates scheduling to the Codex runtime.	No; evidence confirmed	?
Harness CLI ★ 15 · 2026-08-28	The README describes an orchestration and repair layer over Claude Agent SDK or Codex App Server runtimes.	No; evidence confirmed	?
Liza ★ 363 · 2026-08-28	The source implements a control plane over external coding-agent CLIs, with a persistent YAML blackboard, isolated git worktrees, doer/reviewer roles, leases, recovery...	No; evidence confirmed	Apache-2.0
OpenAgents ★ 4,005 · 2026-08-28	The official repository describes OpenAgents as a network for persistent agents and shared workspaces.	No; evidence claimed	Apache-2.0
Proliferate ★ 310 · 2026-08-23	Open-source AI IDE for Claude Code, Codex, OpenCode, and more.	No; evidence claimed	Open source
Symphony ★ 26,812 · 2026-08-23	OpenAI's harness for fanning a task out into many isolated, autonomous coding-agent implementation runs and surfacing the ones that pass, so a team manages outcomes instead of...	No; evidence claimed	?
harness ★ 20 · 2026-08-28	The README explicitly describes a unified subprocess wrapper for existing coding-agent CLIs.	No; evidence confirmed	?
vibe-kanban ★ 27,893 · 2026-08-23	Kanban-style fleet manager for running Claude Code, Codex, or any coding agent across many tasks at once.	No; evidence claimed	?
Vigilante ★ 37 · 2026-08-28	The README explicitly calls Vigilante a control plane over supported headless coding-agent CLIs.	No; evidence confirmed	?

Liza: a repository harness and control plane combined

Liza coordinates external coding-agent CLIs rather than replacing their inner model-and-tool loop. Its pinned provider catalog defines nine subprocess adapters: Claude Code, Codex, Cursor, OpenCode, Gemini, Mistral, Kimi, Qwen, and Devin. Gemini and Mistral are disabled in that snapshot. This is why Liza appears in the adjacent map and does not increase the 42-runtime count.

The repository contains more than a prompt-level role description. The supervision model at commit a22c123 defines a persistent YAML blackboard, file-locked state changes, isolated git worktrees, lease and generation fencing, doer/reviewer task transitions, recovery operations, and supervised merge authority. The cloned snapshot contained 296 Go test files. Its exact commit passed the upstream Ubuntu and macOS jobs on 2026-08-27. Local tests were not executed for this review because the review host did not have Go installed.

The pinned INVARIANTS.md names system properties and their declared enforcement locations. The CONTRACT_FAILURE_MODE_MAP.md maps documented failure modes to contract clauses, and the snapshot tracks 28 modular skills. These artifacts make Liza's enforcement design inspectable. They do not measure whether the countermeasures improve agent behavior. The competitive survey's "Code Quality Evidence" is a project-published Claude Code assessment, so its grade is internal self-assessment rather than independent validation.

The satellite utilities are also optional, not runtime prerequisites. Liza's toolchain profiles can install Stacklit, scip-search, and functional-cluster tooling, while its agent tool contract falls back to rg, ast-grep, and direct reads when those indexes or tools are unavailable. This supports graceful degradation; it does not establish task quality.

The source confirms code-enforced orchestration, not production safety. A git worktree prevents concurrent agents from editing the same checkout; it does not isolate processes, credentials, or network access. Five of the nine provider adapters define an opt-in ACP path that passes --approve-all. OpenCode and Devin also use broad permission modes on their default CLI paths. For Codex, Cursor, and Qwen, the broad flag appears only on the ACP path; Claude's default path uses --permission-mode auto. Review the selected provider path, run Liza inside an OS-level sandbox with scoped credentials, and pilot recovery plus merge gates on real repository tickets.

Liza's own architectural issues ledger records remaining single-gate, cross-pair review, specification-feedback, and context-pressure risks. The separate Spec Evolution Protocol does provide procedural long-term synchronization through task blocking, human edits, a spec changelog, a spec_updated activity entry, and impact assessment. This protocol is human-driven rather than code-enforced, and the issue ledger still records missing automated lifecycle and staleness controls.

Loop and graph lens. Liza combines a frozen organization graph with a runtime work graph. The pinned pipeline.yaml declares roles, doer/reviewer pairs, state vocabularies, quorums, and cross-pair transitions. The supervisor creates and advances task dependencies at runtime. This is a domain-specific executable graph, not a general graph framework: the pinned TransitionDef exposes source, destination, trigger, and cardinality, but not a general edge schema, join policy, or arbitrary node program.

Judgment boundary. Mechanical checks can reject illegal state transitions, stale leases, missing dependencies, or an unmet review quorum. They cannot establish that the decomposition, implementation, or reviewer verdict is semantically correct. The same issue ledger says cross-pair decomposition judgment is a consequential single gate, provider diversity is preferred rather than guaranteed at verdict time, and reviewer accuracy is unmeasured. Manual checkpoints make human availability load-bearing only when auto_resume is disabled. With auto_resume, agents advance CHECKPOINT and COMPLETED states automatically. Pairing yolo pre-approves doer-side human prompts but retains reviewer approvals, validation, stop conditions, merge-conflict handling, and user stop instructions. Human authority remains load-bearing for governance and irreversible effects, even when live human availability is not an execution prerequisite. Compare Liza on two axes: workflow correctness and task correctness. Passing the first does not imply the second.

The evidence is no longer limited to the maintainer. An Ippon practitioner report describes a deliberately small catalog project with roughly 30 tasks, 5 automated sprints, 35 review verdicts, 3 corrected rejections, and 3 to 4 hours of human time. It also reports massive token consumption and required human planning checkpoints. This is a useful bounded REX, not a production or comparative benchmark.

Selection boundary. Liza is a candidate when a repository needs persistent multi-role state, code-enforced transitions, recovery, and supervised integration across many tasks. A bounded implementation-review loop has less coordination surface and should remain the default when those control-plane properties are unnecessary.

This responsibility-boundary reading was prompted by private comparison notes shared by Liza maintainer Tangi Vass. The published claims above are independently tied to the pinned repository rather than to those notes.

Harness Optimizers and Meta-Harnesses

Harness optimizers are adjacent to the strict runtime map. They do not primarily execute user tasks or coordinate a fleet. They change a target harness, evaluate candidates, and decide which version should govern later runs. Most entries below are research systems, not production products.

System	Optimizes	Evidence	Maturity and main limit
ADAS	Code-defined prompts, tools, and workflows	ICLR 2025 experiments across coding, science, and math	Peer-reviewed, but headline comparisons are system-level rather than pure fixed-model ablations
AFlow	Workflow topology represented as code	5.7% average improvement across six benchmarks	ICLR 2025; heterogeneous baselines
ACE	Context and memory playbooks	+10.6% on agents and +8.6% on finance tasks	ICLR 2026; narrower than whole-harness optimization
GEPA	Prompts using reflected trajectories	6% average gain over GRPO across six tasks; up to 35 times fewer rollouts	ICLR 2026 Oral; prompt-level optimizer
Meta-Harness	End-to-end harness code	Classification, math, and TerminalBench-2 improvements	2026 preprint; coding search and final evaluation reuse the same 89 tasks
Agentic Harness Engineering	Prompt, tools, middleware, skills, subagents, and memory	Terminal-Bench 2 pass@1 from 69.7% to 77.0% over ten iterations	2026 preprint focused on coding benchmarks
HarnessOpt-Bench	Evaluates the optimizer, not one target harness	Five optimizer models, four downstream tasks, 111 scored runs	2026 benchmark preprint; early protocol awaiting broader reproduction

Do not fold these systems into the 42-runtime count. A runtime owns the task loop. An optimizer owns a search loop over candidate harnesses. Agent Harness Engineering §11 documents the evidence and the minimum evaluation protocol.

Four Layers, Four Responsibilities

Layer	Owns	Typical artifacts	Selection question
Model	Generation and reasoning	Weights, API, context window	Which model meets the task, latency, privacy, and cost constraints?
Repository harness	Project-specific instructions and controls	AGENTS.md, CLAUDE.md, skills, hooks, policies	What behavior must remain portable with the repository?
Runtime harness	Tool loop, permissions, state, recovery	CLI, IDE agent, desktop or cloud runtime	Who owns plan, act, observe, repeat?
Orchestrator	Queues, workspaces, budgets, multiple runs	Scheduler, fleet manager, task board	What must coordinate more than one runtime or agent?

Frameworks, SDKs, sandboxes, memory systems, evaluation tools, observability platforms, and protocols sit beside or below these layers. LangGraph can help build a runtime; E2B can isolate its execution; Mem0 can persist memory; Langfuse can observe it; MCP can connect tools. None of those roles alone proves ownership of the coding loop.

The term meta-harness has two incompatible uses. Optimizer research uses it for a system that changes one or more harness layers under evaluation. Products such as Omnigent use it for a common interface that dispatches tasks to existing harnesses. Keep the roles separate: this guide classifies a dispatcher as an orchestrator or control plane, while a harness optimizer changes the system that will perform future runs. The generated directory below preserves each pinned source's wording, so Omnigent's row retains its upstream meta-harness label even though the guide layer is orchestration. See the Databricks cost-management resource evaluation for the terminology boundary.

Complete Project Directory

The directory preserves every upstream project and lists the 32 guide supplements separately. ? means the pinned source did not support a conclusion. N/A means the field does not apply to that category. Archived projects remain visible and marked, because removal would hide the history behind current comparisons.

Coding agent products (IDEs, CLIs, full suites) (22)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
AgentBox ★ 374 · 2026-08-23	Runs multiple coding agents in parallel, each in its own sandboxed VM, locally or in the cloud, from one command.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	Open source
cc-haha ★ 14,189 · 2026-08-23	Local-first desktop workspace harness for Claude Code and other agents: multi-agent sessions, Git worktrees, code diffs, a skill marketplace, and chat-app access (WeChat...	Memory, Multi-agent, Typescript	Complex	N/A / N/A	?
claw-code-agent ★ 543 · 2026-08-23	Python reimplementation of the Claude Code agent architecture with zero external dependencies; interactive chat, streaming, plugin runtime, nested agent delegation, cost...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Checkpoint gated / None	?
Cline ★ 66,707 · 2026-08-23	VS Code extension whose harness is a plan-then-act loop with per-step human approval and cost transparency; the VS Code integration is the UI shell.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Step gated / Resumable	Open source
Codex ★ 114,837 · 2026-08-23	OpenAI's terminal coding agent.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	Open source
crush ★ 27,601 · 2026-08-23	Charm's terminal coding agent (Charm's fork of the original OpenCode).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	Restricted (fsl-1.1-mit)
DeepSeek-Reasonix ★ 35,065 · 2026-08-23	DeepSeek-native terminal coding agent.	Cli, Memory, Tui, Typescript	Slightly complex	N/A / N/A	?
eigent ★ 15,083 · 2026-08-23	Open-source desktop harness positioned as a local, free alternative to Claude Cowork and Codex: multi-agent workspace orchestration in a self-hosted app rather than a hosted...	Local, Multi-agent	Complex	N/A / N/A	?
Gemini CLI ★ 106,626 · 2026-08-23	Google's first-party terminal agent for Gemini.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	Open source
goose ★ 53,295 · 2026-08-23	Block-originated Rust agent, now stewarded by the Linux Foundation's Agentic AI Foundation (aaif-goose/goose).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / Resumable	Open source
jcode ★ 18,308 · 2026-08-23	Rust terminal coding agent pitched as the most RAM-efficient harness in its class; MCP support, multi-provider (Claude/OpenAI).	Cli, Mcp, Memory, Provider-agnostic	Slightly complex	N/A / N/A	?
Kilo Code ★ 26,978 · 2026-08-23	VS Code extension and CLI in the Cline/Roo-Code lineage : a natural pick now that Roo-Code is archived upstream.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Step gated / Resumable	?
oh-my-pi ★ 26,658 · 2026-08-23	Terminal coding agent (fork of Pi) that wires the IDE into the harness: hash-anchored edits, a 32-tool loop tuned per-model, LSP rename/references/diagnostics on every write, a...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	Open source
Open Interpreter ★ 68,121 · 2026-08-23	Lightweight terminal coding agent oriented to open models (DeepSeek, Kimi, Qwen).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / Resumable	Open source
opencode ★ 200,557 · 2026-08-23	Open-source terminal coding agent (formerly sst/opencode; transferred to anomalyco).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / Resumable	Open source
OpenHands ★ 84,844 · 2026-08-23	Dockerized software-engineering agent.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Restricted ((multi-license))
pi ★ 95,747 · 2026-08-23	The upstream AI agent toolkit behind this list's oh-my-pi fork: a unified multi-provider LLM API, agent loop, and TUI shell providing the harness that oh-my-pi's Rust rewrite...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	?
Proliferate ★ 310 · 2026-08-23	Open-source AI IDE for Claude Code, Codex, OpenCode, and more.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	Open source
qwen-code ★ 27,310 · 2026-08-23	Alibaba's official terminal coding agent, forked from Gemini CLI's agent loop and retuned for Qwen models.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	?
Roo Code ★ 24,326 · 2026-08-23	VS Code/Cursor extension in the Cline lineage.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Step gated / Resumable	Open source
Symphony ★ 26,812 · 2026-08-23	OpenAI's harness for fanning a task out into many isolated, autonomous coding-agent implementation runs and surfacing the ones that pass, so a team manages outcomes instead of...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	?
vibe-kanban ★ 27,893 · 2026-08-23	Kanban-style fleet manager for running Claude Code, Codex, or any coding agent across many tasks at once.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	?

Coding harness configs and SDKs (17)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
addyosmani/agent-skills ★ 89,221 · 2026-08-23	Addy Osmani's production-grade skill pack: 24 engineering skills and 4 specialist agent personas that encode senior-dev workflows (spec through deploy) across 70+ coding agents...	Ide, Workflow	Mostly simple	N/A / N/A	Open source
agents-cli ★ 5,708 · 2026-08-23	Google's official CLI and skill pack that layers agent-creation, evaluation, and deployment skills on top of whatever coding assistant you already run, rather than shipping its...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	N/A / N/A	?
Anthropic Skills ★ 171,127 · 2026-08-23	Anthropic's official Agent Skills repository: SKILL.md-based folders (instructions, scripts, resources) Claude dynamically loads on Claude Code, Claude.ai, and the API.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	N/A / N/A	Restricted (anthropic terms)
AutoHarness ★ 368 · 2026-08-23	Lightweight governance harness: wraps any LLM client in ~2 lines for automated harness engineering: 6-14 step pipeline, YAML constitution, risk-pattern matching, session...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	Bounded / None	Open source
awesome-claude-code ★ 52,857 · 2026-08-23	Large community-curated index of Claude Code skills, slash commands, status lines, and plugins: resources for extending the harness, not a harness itself, but the most-followed...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	N/A / N/A	?
Claude Agent SDK ★ 7,957 · 2026-08-23	Official Anthropic SDK (Python + TypeScript, demos, quickstarts): built-in tools, MCP, long-running coding agents with session bridging.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
get-shit-done ★ 8,617 · 2026-08-23	Goal-backward planning and wave-based execution over fresh context windows; avoids context rot by design.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / Resumable	Open source
GStack ★ 129,281 · 2026-08-23	Garry Tan's Claude Code skill stack: 23 slash-command modes (CEO/eng/design review, QA, ship, browse, retro, …) that structure one assistant as a virtual engineering team.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	Open source
LoopTroop ★ 123 · 2026-08-23	Config layer that chains LLM councils for planning, Ralph loops for iterative refinement, and OpenCode worktrees for shipping.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / Retry	Open source
Meta-Harness ★ 1,448 · 2026-08-23	Reference implementation from the Meta-Harness paper: an academic testbed for harness-engineering research, not a product: useful as a citation-grade baseline rather than...	?	Slightly complex	N/A / N/A	?
planning-with-files ★ 26,305 · 2026-08-23	Skill for persistent, file-based planning across long-running coding-agent sessions: crash-proof markdown plans, session recovery after /clear/compaction, and a deterministic...	Memory	Mostly simple	N/A / N/A	?
pmstack ★ 8 · 2026-08-23	Claude Code config for AI product managers: CLAUDE.md plus skills for competitive analysis, PRD-from-signal, metric frameworks, stakeholder briefs, and agent eval design.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	N/A / N/A	Open source
RepoMaster ★ 543 · 2026-08-23	Repo-scoped research harness: builds function-call and module-dependency graphs to explore only what's needed; large relative gains on MLE-bench and GitTaskBench with lower...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / None	?
skillhub ★ 4,891 · 2026-08-23	iFlytek's self-hosted registry for publishing, versioning, and governing agent skill packages: the harness config layer treated as an enterprise artifact store rather than a...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	N/A / N/A	?
superpowers ★ 276,518 · 2026-08-23	Performance-oriented harness pack for Claude Code and 13 other harnesses (Codex, Cursor, OpenCode, Gemini CLI, more): skills, instincts, memory, security, research-first workflows.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	N/A / N/A	Open source
SWE-agent ★ 20,112 · 2026-08-23	LM-driven harness built for SWE-bench: edit state, command execution, and issue-focused loop: the reference agent stack next to the benchmark itself.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / Resumable	Open source
wshobson/agents ★ 39,042 · 2026-08-23	Cross-harness marketplace of drop-in subagents and skills for Claude Code, Codex CLI, Cursor, OpenCode, and Copilot; specialized, production-ready agent definitions you install...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	N/A / N/A	Open source

Evaluation and benchmarking harnesses (18)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
AgencyBench ★ 94 · 2026-08-23	Long-horizon agent benchmark: 32 scenarios, 138 tasks, ~1M tokens and ~90 tool calls; Docker sandbox and rubric-based + LLM judges.	Evals, Python, Sandbox	Complex	Headless / None	Open source
Agent Lightning ★ 17,609 · 2026-08-23	Microsoft's training-oriented harness: optimization loops for agent behavior: when you need to improve policies over rollouts, not only score a fixed prompt.	Evals, Python, Training	Complex	Headless / Resumable	Open source
agent-qa ★ 936 · 2026-08-23	Self-improving QA harness for web and mobile apps: natural-language tests, memory-backed self-healing, dashboard/CLI, MCP and skills support, plus sandboxed hooks for...	Cli, Mcp, Memory, Sandbox	Slightly complex	Headless / Retry	Restricted (fsl-1.1-alv2)
AgentBench ★ 3,682 · 2026-08-23	ICLR'24 benchmark: agents across AlfWorld, DB, knowledge graphs, OS, webshop; Docker Compose, function-calling interface.	Evals, Python, Rag, Sandbox	Complex	Headless / None	Open source
ARC-AGI-2 ★ 734 · 2026-08-23	ARC Prize task set: grid-based abstraction/reasoning; public and private splits for generalization.	?	Super simple	N/A / N/A	Open source
arc-agi-benchmarking ★ 363 · 2026-08-23	Runner for ARC-AGI: multi-provider (OpenAI, Anthropic, Gemini, etc.), rate limits, retries, and scoring.	Evals, Provider-agnostic, Python	Mostly simple	Headless / Retry	Open source
inspect_ai ★ 2,606 · 2026-08-23	Inspect AI core: composable eval tasks, sandboxes, scorers, and multi-model runs; the framework behind inspect_evals, not just the task bundle.	Evals, Python, Sandbox	Complex	Headless / Resumable	Open source
inspect_evals ★ 639 · 2026-08-23	UK AISI/Arcadia/Vector: GAIA and other evals in Inspect AI; level 1-3, sandboxed, tool-calling solvers.	Evals, Sandbox	Slightly complex	Headless / Resumable	Open source
letta-evals ★ 83 · 2026-08-23	Eval harness for stateful Letta agents; configurable suites and grading (LLM or rule-based) so you can measure what you ship.	Memory, Python	Mostly simple	Headless / None	Open source
SUPER ★ 58 · 2026-08-23	Agents that set up and run ML/NLP from GitHub repos; 45 expert problems, 152 masked tasks, 602 AutoGen tasks; Docker-based.	Python, Sandbox	Slightly complex	Headless / None	Open source
SWE-bench ★ 5,691 · 2026-08-23	LMs resolve real GitHub issues; Docker harness, instance IDs; standard for code-agent evals.	Evals, Python, Sandbox	Slightly complex	Headless / Resumable	Open source
SWE-Gym ★ 723 · 2026-08-23	Training and evaluation for SWE agents and verifiers (ICML 2025).	Evals, Python, Training	Slightly complex	Headless / None	Open source
swe-smith ★ 748 · 2026-08-23	Data generation for SWE agents; 50k+ instances across 128 repos; used for SWE-agent-LM training.	Python, Training	Slightly complex	Headless / None	Open source
Terminal-Bench ★ 533 · 2026-08-23	The terminal-task benchmark coding agents now cite next to SWE-bench: hard, containerized terminal tasks scored end to end.	Cli, Evals, Python	Slightly complex	Headless / None	Open source
TRAIL ★ 22 · 2026-08-23	Trace reasoning and agentic issue localization; 148 long-context traces, 841 errors, 20+ error types; Hugging Face dataset.	?	Mostly simple	N/A / N/A	Open source
VitaBench ★ 164 · 2026-08-23	ICLR'26: 66 tools, real-world apps (delivery, travel, retail); 100 cross-scenario + 300 single-scenario tasks; adopted by Qwen/Seed.	?	Complex	Headless / None	Open source
WebArena ★ 1,584 · 2026-08-23	Realistic web env (e.g.	Python	Complex	Headless / None	Open source
WebVoyager ★ 1,122 · 2026-08-23	End-to-end web agent with LMMs: screenshots + actions on real sites; benchmark on 15 sites, GPT-4V for automatic eval.	Evals, Vision	Slightly complex	Headless / None	Open source

Frameworks (25)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
agent-squad ★ 7,742 · 2026-08-23	AWS-originated orchestrator (now under 2FastLabs): intent classification, streaming, SupervisorAgent; "agent-as-tools" so one agent delegates to a squad.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	Open source
AgentSilex ★ 454 · 2026-08-23	~300 lines of readable agent code on top of LiteLLM; the "I want to see the whole loop" option for learning or minimal production.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	Bounded / None	Open source
AgentStack ★ 2,185 · 2026-08-23	Scaffolds full agent projects; plugs in CrewAI, LangGraph, OpenAI Swarm, LlamaStack and wires AgentOps observability from day one.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	Open source
AgentVerse ★ 5,113 · 2026-08-23	Task-solving and simulation envs for multi-LLM agents; deploy many agents in custom environments without building infra from scratch.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / None	Open source
agno ★ 41,848 · 2026-08-23	Python agents with memory, knowledge bases, tools, and structured outputs; continues the PhiData-era product line under the Agno name: production apps, evals, and pipelines.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	Open source
AutoGPT ★ 186,806 · 2026-08-23	The original autonomous loop: goal in, agent iterates with tools and memory; Forge is the dev framework, Benchmark the eval harness.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Restricted (polyform-su)
Bee Agent Framework ★ 3,383 · 2026-08-23	Python + TypeScript, LF AI-backed; MCP/ACP, workflows, Requirement Agent; the one that pushes "production multi-agent" without LangChain.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	Open source
botpress ★ 14,876 · 2026-08-23	Visual bot builder and runtime; multi-channel, open-source alternative to commercial bot platforms.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
browser-use ★ 110,219 · 2026-08-23	Python web-agent harness: natural-language goals become browser actions, driven directly over the Chrome DevTools Protocol (it dropped Playwright in August 2025).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Retry	Open source
Dify ★ 153,269 · 2026-08-23	One-stop LLM app platform: visual workflows, RAG pipeline, 50+ tools, model management; "ship from prototype to prod" in a single UI.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Retry	Restricted (fair-code)
Google ADK ★ 21,234 · 2026-08-23	Google's official Agent Development Kit: code-first Python toolkit for building, evaluating, and deploying agents.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
Haystack ★ 26,293 · 2026-08-23	Open-source orchestration framework for context-engineered LLM apps: modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and...	Memory, Python, Rag	Complex	N/A / N/A	Open source
langchain ★ 144,822 · 2026-08-23	Chains, tools, retrievers, and agents; the usual entry point for "add tools to an LLM" in Python/JS.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Retry	Open source
langflow ★ 153,582 · 2026-08-23	Low-code UI to build and deploy LangChain/LangGraph flows; visual DAG editor and one-click run.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Retry	Open source
langgraph ★ 40,276 · 2026-08-23	State-machine graphs over LLM steps; checkpointing, human-in-the-loop, and durable execution so workflows survive restarts.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / Durable	Open source
letta ★ 24,360 · 2026-08-23	Python agent runtime with tool use and control flow; lean API; stateful agents with long-horizon memory.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Headless / Durable	Open source
llama-index ★ 51,815 · 2026-08-23	Data-centric: indexing, RAG, and query engines; agent abstractions sit on top of your data pipelines.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Retry	Open source
mastra ★ 27,375 · 2026-08-23	TypeScript-first; agents, tools, and workflows with a single runtime and minimal boilerplate.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Durable	Restricted (elastic-2.0)
n8n ★ 202,069 · 2026-08-23	Fair-code workflow engine with 400+ nodes and native AI nodes; the self-hosted Zapier that actually does agents and LangChain.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Durable	Restricted (fair-code)
R2R ★ 7,973 · 2026-08-23	RAG-first: hybrid search, knowledge graphs, multimodal; the framework for "production RAG" when you care more about retrieval than chat UI.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Retry	Open source
rasa ★ 21,302 · 2026-08-23	Conversational AI stack (NLU, dialogue, actions); long-standing OSS choice for chat and voice bots.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
semantic-kernel ★ 28,481 · 2026-08-23	Microsoft's plugin and planner layer for LLMs; C#, Python, Java; strong on enterprise auth and orchestration.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Retry	Open source
Stagehand ★ 24,023 · 2026-08-23	Browserbase's SDK for browser agents: natural-language actions (act, extract, observe) and deterministic Playwright code mix in one script, so agent flexibility and repeatable...	Browser, Typescript	Slightly complex	Bounded / None	Open source
SuperAgentX ★ 203 · 2026-08-23	Lightweight multi-agent orchestrator with an AGI-angle; minimal surface, docs-first, for teams that want orchestration without the kitchen sink.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / None	Open source
youtu-agent ★ 4,601 · 2026-08-23	Tencent Cloud's agent framework: a minimal tool-calling harness designed to perform well with open-source models, positioned as a lighter alternative to heavier orchestration...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / Retry	?

Libraries and SDKs (15)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
Agent Sandbox ★ 3,591 · 2026-08-23	Kubernetes-native sandbox primitive for agent runtimes: a Sandbox resource plus warm pools and claims for fast-start, isolated, stateful workloads.	Local, Memory, Sandbox	Slightly complex	N/A / N/A	Open source
Cloudflare Agents ★ 5,478 · 2026-08-23	Persistent, stateful agents on Durable Objects: state, websockets, scheduling, and AI chat baked in.	Memory, Typescript	Slightly complex	Headless / Durable	Open source
Community-curated agent lists ★ 15 · 2026-08-23	Broader directories: e.g.	?	Super simple	N/A / N/A	?
Composio ★ 29,840 · 2026-08-23	1,000+ toolkits with auth, tool search, and a sandboxed workbench: drop-in tool layer so agents stop reinventing OAuth + integrations.	Python, Sandbox, Tool-discovery, Typescript	Complex	N/A / N/A	Open source
Daytona ★ 71,914 · 2026-08-23	Elastic dev environments for AI-generated code: workspaces, Git, previews: infra harness between "the model wrote a patch" and "it ran in a real machine." ⚠️ Public repo...	Sandbox	Slightly complex	N/A / N/A	Open source
deepagents ★ 28,172 · 2026-08-23	LangChain's Python+TypeScript agent harness on top of LangGraph: planning tool, virtual filesystem, shell sandbox, sub-agent spawning: the "Claude Code-style" harness as a...	Multi-agent, Python, Sandbox, Typescript	Slightly complex	Bounded / Durable	Open source
E2B ★ 13,523 · 2026-08-23	Firecracker sandboxes for executing agent-generated code; the hosted isolation layer many tool-calling demos use instead of running arbitrary LLM output on your laptop.	Python, Sandbox	Slightly complex	N/A / N/A	Open source
LiteLLM ★ 57,066 · 2026-08-23	One interface to 100+ LLMs; routing, caching, budgets.	Provider-agnostic, Python	Mostly simple	N/A / Retry	Open source
open-harness ★ 599 · 2026-08-23	TypeScript Agent class on Vercel AI SDK; streaming events, filesystem/bash tools, MCP, and subagent delegation.	Mcp, Multi-agent, Typescript	Slightly complex	Bounded / None	Open source
openai-agents-js ★ 3,685 · 2026-08-23	Official OpenAI Agents SDK for Node/TS: handoffs, guardrails, voice; the JS counterpart to openai-agents-python.	Multi-agent, Typescript, Voice	Slightly complex	Bounded / Resumable	Open source
pydantic-ai ★ 19,453 · 2026-08-23	Type-safe Python agents with Pydantic I/O; multi-provider, MCP, Logfire observability, and human-in-the-loop.	Mcp, Provider-agnostic, Python, Typed	Slightly complex	Bounded / Durable	Open source
smolagents ★ 28,938 · 2026-08-23	Code-as-action agents: model outputs Python executed in sandbox (E2B, Modal, etc.); ~1k LOC core.	Python, Sandbox	Mostly simple	Bounded / None	Open source
Steel ★ 7,529 · 2026-08-23	Open-source browser API for agents: cloud or self-hosted Chrome sessions with stealth, residential proxies, CAPTCHA solving, and persistent profiles.	Browser, Local, Memory	Slightly complex	N/A / N/A	Open source
strands-agents ★ 6,984 · 2026-08-23	Model-driven Python SDK; decorators for tools, native MCP, multi-agent; "minimal code" without sacrificing provider choice.	Mcp, Multi-agent, Python, Typed	Mostly simple	Bounded / Resumable	Open source
vercel/ai ★ 26,368 · 2026-08-23	React and Node SDK for streaming, tool calls, and agent-style UIs; provider-agnostic.	Provider-agnostic, Typescript	Slightly complex	Bounded / Retry	Open source

Memory and state (5)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
beads ★ 26,534 · 2026-08-23	Portable persistent-memory layer for coding agents: tracks decisions and task state outside the harness's own context window so it survives session resets and model swaps.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	N/A / N/A	?
claude-mem ★ 91,578 · 2026-08-23	Session-memory plugin for Claude Code, Codex, OpenClaw, Gemini, Copilot, and more: captures everything an agent does during a session, AI-compresses it, and injects the...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	Open source
cognee ★ 30,194 · 2026-08-23	Open-source memory layer for agents: an extract-cognify-load pipeline that turns your data into a queryable knowledge graph plus vector store, so agents recall facts and...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	Open source
Graphiti (Zep) ★ 30,212 · 2026-08-23	Zep's open-source memory engine: real-time temporal knowledge graphs that track how facts about users and entities change over time, so agents can answer "what was true when."...	Memory, Python, Rag, Workflow	Slightly complex	N/A / N/A	Open source
Mem0 ★ 63,868 · 2026-08-23	Universal memory layer for AI agents: stores user/org/session memory, retrieves on demand.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	N/A / N/A	Open source

Multi-agent and orchestration (12)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
AG2 ★ 4,883 · 2026-08-23	AG2 (formerly AutoGen): the community-governed continuation of the original AutoGen project after Microsoft's fork diverged: conversable multi-agent groups, code execution, and...	Multi-agent, Python	Complex	N/A / N/A	?
AgentRL ★ 347 · 2026-08-23	Multitask, multiturn RL for LLM agents; Ray-based scaling, rollout/actor workers: for teams that want to train agents, not just run them.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
autogen ★ 60,585 · 2026-08-23	Conversable agents and group chats; code execution and human-in-the-loop; Microsoft origin, AG2 ecosystem.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	Open source
ChatDev ★ 34,103 · 2026-08-23	Multi-agent software-company simulation (CEO, CTO, programmer, tester) built on chat chains with communicative dehallucination; ChatDev 2.0 continues the line.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / None	Open source
crewAI ★ 57,501 · 2026-08-23	Role-based agents (roles, goals, backstories) in Crews; Flows add event-driven and hierarchical control for production.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	Open source
hive ★ 10,949 · 2026-08-23	Self-hosted multi-agent harness aimed at production workloads: human-in-the-loop checkpoints and a self-improving agent loop, distinct from single-session coding-agent shells.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	?
MetaGPT ★ 69,962 · 2026-08-23	The "AI software company" multi-agent framework: role-played PM, architect, and engineer agents turn a one-line requirement into specs, designs, and code along an SOP assembly...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
Microsoft Agent Framework ★ 13,060 · 2026-08-23	Microsoft's convergence of AutoGen and Semantic Kernel: build, orchestrate, and deploy agents and multi-agent workflows in Python and .NET, with graph-based workflows and...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	Open source
omnigent ★ 9,189 · 2026-08-23	Open-source meta-harness: orchestrates Claude Code, Codex, Cursor, Pi, and custom agents behind one policy/sandboxing layer so teams swap harnesses without rewriting workflows.	Ide, Python, Sandbox	Complex	N/A / N/A	?
openai-agents-python ★ 28,887 · 2026-08-23	Handoffs, guardrails, and multi-LLM routing; minimal surface so you own the loop.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / Resumable	Open source
OpenManus ★ 58,048 · 2026-08-23	Open, invite-free general agent from the MetaGPT team: planning plus tool use over a multi-agent loop, aimed at reproducing Manus-style autonomous task completion on your own keys.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / None	Open source
PraisonAI ★ 8,947 · 2026-08-23	Autonomous multi-agent teams with a single entry point; emphasis on minimal config.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	Bounded / None	Open source

Observability and eval-ops (4)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
Arize Phoenix ★ 11,148 · 2026-08-23	Arize's source-available, local-first tracing and eval layer: run it on your laptop or your own infra, and graduate to the managed Arize AX platform only when you need it.	Evals, Python	Slightly complex	N/A / N/A	Restricted (elastic-2.0)
Langfuse ★ 33,575 · 2026-08-23	Open-source LLM engineering platform: full-trace observability, online and offline evals, prompt management, and cost metrics for agent runs in production: the monitoring layer...	Evals, Typescript	Slightly complex	N/A / N/A	Open source
MLflow ★ 27,630 · 2026-08-23	Mature ML platform now covering GenAI: MLflow Tracing captures every agent step, tool call, and token, with built-in LLM evals and prompt versioning: observability for teams...	Evals, Python	Complex	N/A / N/A	Open source
Opik ★ 21,549 · 2026-08-23	Comet's open-source agent observability and evaluation platform: tracing, scoring, and experiment comparison with the whole core feature set free to self-host under Apache-2.0.	Evals, Python	Slightly complex	N/A / N/A	Open source

Personal agent runtimes (10)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
Agent Zero ★ 18,943 · 2026-08-23	Organic, prompt-defined personal agent framework: hierarchical sub-agents, persistent memory, browser and code tools, and self-modifying behavior; runs in Docker with a web UI.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Resumable	?
AIlice ★ 1,411 · 2026-08-23	Fully autonomous general-purpose agent; one binary, Docker-ready, for when you want "set goal and walk away" without a framework.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / None	Open source
CowAgent ★ 46,642 · 2026-08-23	Self-hosted harness (formerly chatgpt-on-wechat) that plans tasks, runs tools/skills, and self-evolves via memory; multi-model, multi-channel (WeChat, Telegram, etc.), one-line...	Memory, Python	Slightly complex	N/A / N/A	?
Eliza ★ 19,137 · 2026-08-23	Open "agentic operating system" (elizaOS): persistent multi-agent runtime with character files, a plugin ecosystem, and social/platform integrations : the harness behind a...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
Hermes ★ 234,688 · 2026-08-23	Nous Research's self-improving agent: a learning loop turns experience into reusable skills, builds a persistent user model across sessions, and checkpoints state to disk with...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / Resumable	Open source
Khoj ★ 36,674 · 2026-08-23	Self-hostable "AI second brain": answers over your docs and the web, custom agents, scheduled automations, and multi-client reach (web, Obsidian, Emacs, WhatsApp).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
nanobot ★ 47,301 · 2026-08-23	Ultra-lightweight, self-hosted personal agent framework: the harness is a Python daemon wiring tools, memory, and MCP into chat/webhook front ends (Telegram, Discord, web)...	Local, Mcp, Memory, Python	Mostly simple	N/A / N/A	?
OpenClaw ★ 387,220 · 2026-08-23	Self-hosted, always-on personal agent (formerly Clawdbot/Moltbot): a gateway + event-loop runtime that treats messages, heartbeats, crons, and webhooks as one input queue...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source
OpenHarness (HKUDS) ★ 15,492 · 2026-08-23	Open agent harness with a built-in personal agent ("Ohmo") that runs across Feishu, Slack, Telegram, and Discord; core tool-use, skills, memory, multi-agent coordination with...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Resumable	Open source
Talon ★ 71 · 2026-08-23	Multi-platform personal agent living in Telegram, Discord, Teams, and the terminal.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Headless / Resumable	Open source

Plugins, MCPs, CLI tools (19)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
Agent Governance Toolkit ★ 6,091 · 2026-08-23	Microsoft's policy-enforcement layer for autonomous agents: zero-trust identity, execution sandboxing, and OWASP Agentic Top-10 coverage sit in front of the agent loop as a...	Python, Sandbox	Slightly complex	N/A / N/A	Open source
agent-vault ★ 2,124 · 2026-08-23	Infisical's HTTP credential proxy that fronts secrets for Claude Code, OpenClaw, and other agent harnesses so the agent's tool calls never see raw credentials: a harness...	?	Mostly simple	N/A / N/A	?
agentlog ★ 1 · 2026-08-23	Persistent decision memory for any project: remember, recall, reflect.	Cli, Memory, Python	Super simple	N/A / N/A	Open source
aider ★ 48,420 · 2026-08-23	Git-aware CLI pair programmer; edits in-repo, supports multiple models and MCP so agents see version control and tools.	Cli, Mcp, Python	Slightly complex	Checkpoint gated / Resumable	Open source
Better-OpenCodeMCP ★ 9 · 2026-08-23	MCP server for OpenCode/Crush: async task execution, model bridging (e.g.	Mcp, Typescript	Mostly simple	N/A / N/A	Open source
chrome-devtools-mcp ★ 49,601 · 2026-08-23	Google's official Chrome DevTools MCP server: exposes console, network, and performance-trace inspection as tool calls agents can drive directly, instead of a human clicking...	Browser, Mcp, Typescript	Mostly simple	N/A / N/A	Open source
cocoindex-code ★ 2,670 · 2026-08-23	Embedded, tree-sitter/AST-based code-search CLI and MCP server that gives coding agents fast semantic lookups over a repo instead of grepping or re-reading whole files into...	Cli, Mcp	Mostly simple	N/A / N/A	?
Context7 ★ 61,104 · 2026-08-23	MCP server that injects up-to-date, version-specific library docs into an agent's context on demand; kills the stale-training-data hallucinations that plague codegen.	Mcp, Training, Typescript	Super simple	N/A / N/A	Open source
continue ★ 35,601 · 2026-08-23	Open-source IDE extension (VS Code, JetBrains); in-editor completion and chat with local or API models.	Ide, Typescript	Complex	Checkpoint gated / Resumable	Open source
Docker MCP Gateway ★ 1,537 · 2026-08-23	Docker's official MCP CLI plugin / gateway; container-aware MCP tooling from Docker (replaces deprecated docker/mcp-servers path).	Cli, Mcp, Sandbox	Slightly complex	N/A / N/A	Open source
github-mcp-server ★ 32,440 · 2026-08-23	GitHub's official MCP server (Go): repos, issues, PRs, code search, Actions.	Mcp	Slightly complex	N/A / N/A	Open source
MCP Inspector ★ 10,730 · 2026-08-23	GUI to test and debug MCP servers; inspect tools, resources, and prompts.	Mcp, Typescript	Super simple	N/A / N/A	Open source
MCP Python SDK ★ 24,090 · 2026-08-23	Official SDK to build and consume MCP servers/clients in Python; stdio and SSE transports.	Mcp, Python	Mostly simple	N/A / N/A	Open source
MCP Registry ★ 7,185 · 2026-08-23	Official, community-driven registry for MCP servers: the "app store" MCP clients use to discover servers.	Mcp	Slightly complex	N/A / N/A	Open source
MCP Servers ★ 89,798 · 2026-08-23	The official reference collection of Model Context Protocol servers (filesystem, git, fetch, memory, time, and more): the canonical, vetted toolset agents connect to, and the...	Mcp, Memory, Typescript	Mostly simple	N/A / N/A	Open source
MCP TypeScript SDK ★ 13,230 · 2026-08-23	Official MCP implementation for Node/TS; reference for the protocol.	Mcp, Typescript	Mostly simple	N/A / N/A	Open source
mcp-context-forge ★ 4,350 · 2026-08-23	IBM's official AI gateway/registry/proxy that sits in front of any MCP, A2A, or REST/gRPC API: unified endpoint, centralized discovery, guardrails, and plugin support: the...	Mcp, Python	Complex	N/A / N/A	?
Playwright MCP ★ 36,382 · 2026-08-23	Playwright's official MCP server: structured browser control (navigate, click, fill, extract) via the accessibility tree rather than screenshots, so web tasks stay fast and...	Browser, Mcp, Typescript, Vision	Mostly simple	N/A / N/A	Open source
puppeteer-real-browser-mcp ★ 26 · 2026-08-23	Puppeteer MCP with real-browser and anti-detection; for agents that need to drive sites that block headless.	Browser, Mcp, Typescript	Mostly simple	N/A / N/A	?

Progressive disclosure harnesses (8)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
agents.md ★ 23,804 · 2026-08-23	Open format for repo-scoped agent briefings; nested AGENTS.md files scope instructions per directory, so agents get a map of what exists and load only what's relevant.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	N/A / N/A	Open source
awesome-cursorrules ★ 40,646 · 2026-08-23	Curated .cursorrules and skills that leverage Cursor's index-then-load model; the canonical collection for rules-as-progressive-disclosure in the IDE.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Super simple	N/A / N/A	Open source
context-mode ★ 20,102 · 2026-08-23	Context-window optimization layer that sandboxes tool output before it reaches the model (claimed 98% reduction) and persists session memory across 17 agent platforms via MCP...	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	N/A / N/A	Restricted (elastic-2.0)
Headroom ★ 67,250 · 2026-08-23	Compresses tool outputs, logs, files, and RAG chunks with content-aware compressors before they reach the model: claimed 20% fewer tokens for coding agents and 60-95% fewer for...	Mcp, Rag	Mostly simple	N/A / N/A	Open source
langgraph-bigtool ★ 556 · 2026-08-23	Build LangGraph agents with large tool sets; retrieval and on-demand tool loading so agents scale beyond context without stuffing every schema upfront.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Slightly complex	Bounded / Durable	Open source
MCP-Zero ★ 511 · 2026-08-23	Active tool discovery for autonomous agents: model requests tools by requirement; hierarchical semantic routing over 308 servers / 2,797 tools with ~98% token reduction (APIBank).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / None	Open source
ToolGen ★ 184 · 2026-08-23	ICLR 2025: unified tool retrieval and calling via generation; 47k+ tools without context stuffing: retrieval and invocation in one generative step.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	N/A / N/A	?
ToolRAG ★ 33 · 2026-08-23	Semantic tool retrieval for LLMs; serves only the tools the user query demands (MCP-compatible), unlimited tool sets with zero context penalty.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Mostly simple	N/A / N/A	Open source

Research and task-specific harnesses (5)

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
AutoResearchClaw ★ 14,071 · 2026-08-23	Fully autonomous, self-evolving research harness: multi-agent debate and citation verification carry a project from idea to a written paper without a human in the loop.	Multi-agent	Complex	N/A / N/A	?
DeerFlow ★ 80,648 · 2026-08-23	ByteDance's long-horizon research harness: a LangGraph-based agent loop that researches, codes, and creates using sandboxes, memory, tools, and subagents behind a message...	Memory, Multi-agent, Python, Sandbox	Complex	Headless / Resumable	Open source
gpt-researcher ★ 29,099 · 2026-08-23	Autonomous deep-research agent: web + local sources, citation-grounded reports, multi-agent and deep-research modes.	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Bounded / Retry	Open source
MiroThinker ★ 8,359 · 2026-08-23	Deep-research harness tuned for long browsing-and-reasoning chains; benchmarked on BrowseComp, GAIA, and HLE by pairing a dedicated agent loop with its own MiroThinker models...	Evals	Slightly complex	Headless / Retry	?
openagents ★ 447 · 2026-08-23	Platform for autonomous agents and autopilot-style workflows; decentralized/Nostr-oriented (Pylon runtime, actively shipped in 2026).	Build vs buy, Lifecycle hooks, Memory, Prompt optimization	Complex	Headless / Resumable	Open source

Guide supplements (32)

These official products and researched candidates are absent from the pinned upstream snapshot. Their inclusion does not change the upstream 160-project count.

Project	Role	Main capabilities	Adoption	Autonomy / Recovery	Licence
Agent Harness ★ 7 · 2026-08-28	The README explicitly describes an adapter-driven control plane that delegates scheduling to the Codex runtime.	?	?	? / ?	?
AgentForge ★ 60 · 2026-08-28	The README documents a ReAct-style loop that repeats model requests, typed tool calls, and observations until completion.	?	?	? / ?	?
agentic-harness ★ 84 · 2026-08-28	The README documents a coding-agent loop that iterates through model tool calls against a workspace.	?	?	? / ?	?
AgentsMesh ★ 2,330 · 2026-08-28	The README documents a control-plane and data-plane system for operating external coding agents.	?	?	? / ?	?
Amp	Amp presents its product as an agentic coding tool.	?	?	? / ?	Proprietary
Augment Code	Augment presents its product as an AI coding platform with agent capabilities.	?	?	? / ?	Proprietary
Autonomous Coding Harness ★ 7 · 2026-08-28	The README claims a milestone-based autonomous coding loop with implementation, verification, and human checkpoints.	?	?	? / ?	?
Autonomous Workstream ★ 32 · 2026-08-28	The README documents a plugin and CLI that sequences bounded tasks through an external coding-agent backend.	?	?	? / ?	?
Claude Code	Anthropic documents Claude Code as an agentic coding tool that reads codebases, edits files, and runs commands.	?	?	? / ?	Proprietary
Cursor Agent	Cursor presents Agent as its coding-agent product across editor and remote surfaces.	?	?	? / ?	Proprietary
DeepSeek Harness ★ 201,064 · 2026-08-28	The official repository describes a developer-preview agent harness built on the Cordis plugin framework.	?	?	? / ?	Mit
Devin	Cognition presents Devin as an AI software engineer.	?	?	? / ?	Proprietary
Factory Droid	Factory presents Droids as software-development agents.	?	?	? / ?	Proprietary
GitHub Copilot CLI	GitHub presents Copilot CLI as a coding agent for the terminal.	?	?	? / ?	Proprietary
Harness ★ 10 · 2026-08-28	The README documents hardened container environments for agents rather than an agent decision loop.	?	?	? / ?	?
harness ★ 20 · 2026-08-28	The README explicitly describes a unified subprocess wrapper for existing coding-agent CLIs.	?	?	? / ?	?
Harness CLI ★ 15 · 2026-08-28	The README describes an orchestration and repair layer over Claude Agent SDK or Codex App Server runtimes.	?	?	? / ?	?
Hermes Autoresearch (Archived) ★ 8 · 2026-08-28	The archived README says the experiment control loop moved into the hermes-loops monorepo.	Historical	?	? / ?	?
Jules	Google presents Jules as an asynchronous coding agent.	?	?	? / ?	Proprietary
Junie	JetBrains presents Junie as its coding agent.	?	?	? / ?	Proprietary
Kimi Code CLI ★ 7,126 · 2026-08-28	The official repository describes Kimi Code CLI as an agentic coding tool for terminals and IDEs.	?	?	? / ?	Mit
Kiro	Kiro presents its product as an agentic development environment.	?	?	? / ?	Proprietary
Liza ★ 363 · 2026-08-28	The source implements a control plane over external coding-agent CLIs, with a persistent YAML blackboard, isolated git worktrees, doer/reviewer roles, leases, recovery...	Adversarial-review, Crash-recovery, Multi-agent, Worktrees	Complex	Checkpoint gated / Durable	Apache-2.0
Open SWE ★ 10,624 · 2026-08-28	The official repository describes Open SWE as an asynchronous coding agent for repository tasks.	?	?	? / ?	Mit
OpenAgents ★ 4,005 · 2026-08-28	The official repository describes OpenAgents as a network for persistent agents and shared workspaces.	?	?	? / ?	Apache-2.0
OpenCode Harness ★ 148 · 2026-08-28	The README documents a coding-agent loop with tools, permissions, traces, evaluation, and repair feedback.	?	?	? / ?	?
OpenHarness ★ 12 · 2026-08-28	The README documents a provider-to-tools agent loop and runtime injection points.	?	?	? / ?	?
Replit Agent	Replit presents Agent as a product that builds applications from user goals.	?	?	? / ?	Proprietary
Spettro ★ 33 · 2026-08-28	The README documents autonomous goal runs, native tool calls, subagent workflows, and verification loops.	?	?	? / ?	?
Vigilante ★ 37 · 2026-08-28	The README explicitly calls Vigilante a control plane over supported headless coding-agent CLIs.	?	?	? / ?	?
Warp Agent Mode	Warp presents its AI surface as an agentic development environment in the terminal.	?	?	? / ?	Proprietary
Windsurf Cascade	Windsurf presents Cascade as its agentic coding assistant.	?	?	? / ?	Proprietary

How to Pick a Harness

Start at the lowest layer that solves the problem. A new framework, runtime, and orchestrator introduced together create three independent failure surfaces before the team has measured one.

1. Job: What must the agent do: generate an answer, edit a repository, run commands, or coordinate several workers?
2. Adoption surface: Can the team support a CLI, IDE extension, local service, container, or managed cloud environment?
3. Autonomy: Which actions need approval, and what can run without a person present?
4. Recovery: What must survive a failed command, process restart, exhausted context window, or interrupted network call?
5. Usage and cost: Is the workload interactive, bursty, continuous, or parallel? Compare accepted-task cost, not token price alone.
6. Exit: Can prompts, repository instructions, state, tools, and audit data move to another runtime?

Feature count and GitHub popularity cannot answer those questions for a specific codebase.

How to Test-Drive the Shortlist

Test two or three candidates on 8 to 12 representative tasks from the same repositories. Use the same model, repository instructions, tool permissions, and resource budget where the products allow it. Repeat critical tasks because one successful run does not establish reliability. Give each run an isolated worktree and write pass criteria before execution.

Record the following measurements for every task:

Measurement	What to record
Human verdict	Accepted, accepted after correction, or rejected
Interventions	Approval prompts, redirects, manual edits, and restarts
Plan drift	Work outside the requested files, contract, or acceptance criteria
Wall time	Time from task start to reviewed result
Accepted-task cost	Total model and platform cost divided by accepted tasks
Recovery	Whether the run resumes after a controlled interruption without repeating or losing work
Setup friction	Time and specialist work needed to reproduce the environment
Turns to accepted completion	Model or agent turns required before the reviewed result passes; Patrick Debois proposes turn count as an enablement signal
Requirement coverage	Passed, failed, and unresolved acceptance criteria, each linked to its evidence
Proof artifacts	Commands, outputs, tests, screenshots, traces, or runtime captures needed to reproduce acceptance
Tail latency	Per-run distribution and slow cases, not only the mean; see Amit Kushwaha's distribution-aware benchmark argument
Model-harness pair	Exact model, version, reasoning mode, harness version, and configuration used for the result
Repeated-run reliability	Success consistency, perturbation tolerance, predictable resource use, and bounded failure severity; see Towards a Science of AI Agent Reliability

Review the produced diff and tests, not the agent's self-report. A green test suite is necessary but may not cover every requirement. Simon Willison's captured command-and-output workflow is one way to preserve a human-reviewable proof artifact, while Shachar Azriel's executable-spec pattern maps verification to individual requirements. Keep consequential actions behind a human or policy gate during the trial. Use the Agent Evaluation framework to define acceptance evidence, Observability to capture traces and interventions, and Security Hardening to test the execution boundary rather than trusting a product label. Apply the Agentic Benchmark Checklist before treating a score change as a product result: task setup and grader defects can exceed the claimed improvement.

Machine-Readable Access

• Normalized dataset on GitHub: four explicit sets, provenance, loop ownership, feature evidence, official URLs, and dated GitHub metadata.
• Machine-readable reference guide: dataset contract, evidence states, rebuild commands, and maintenance rules.
• reference.yaml: stable routes to the engineering page, this landscape, product profiles, evaluation, observability, security, and release evidence.
• claude-code-releases.yaml: version-level evidence for Claude Code runtime behavior. It is release history, not a cross-product feature table.
• Best of Agent Harnesses site: upstream catalog and comparison guides.
• Pinned upstream JSON: the immutable source used for this snapshot.
• Upstream live JSON: useful for discovering changes after the pinned snapshot.
• MCP resource: claude-code-guide://agent-harnesses from this guide's MCP server.
• Upstream MCP package: agent-harnesses-mcp.

The refresh pipeline validates counts, URLs, evidence states, repository metadata, and deterministic output before it replaces the committed dataset. README extraction treats repository content as untrusted data and never publishes model output automatically.

Related Reading

• Agent Harness Engineering: stable architecture, Claude Code implementation checkpoint, meta-harness evidence, and timestamped practitioner video evidence
• Architecture: the master loop, tool arsenal, context, subagents, permissions, and MCP
• Tools Reference and Hooks Events Reference: exact Claude Code interfaces behind several comparison criteria
• Agent Tools: Beyond Claude Code: deeper profiles for selected coding agents
• Agent Evaluation: task sets, graders, human verdicts, and acceptance criteria
• Observability: traces, metrics, errors, costs, and intervention signals
• Security Hardening and Native Sandbox: permissions, prompt injection, filesystem, process, and network controls
• Agent Teams and Agentic Software Factories: coordination above a single runtime loop
• Glossary: runtime, repository, evaluation harness, and orchestrator terminology

Limits of This Map

• The pinned source contains 160 projects and 86 deep-dive profiles. A light record does not support the same feature claims as a deep profile.
• In the upstream snapshot, autonomy is not applicable for 65 entries and recovery for 64. Missing values must not be rewritten as absent features.
• The official DeepSeek Harness repository is a guide supplement because it is absent from the pinned upstream snapshot. DeepSeek-Reasonix is a different community project present upstream.
• Commercial products require direct vendor verification. Pricing, supported models, limits, and product surfaces can change without a repository commit.
• GitHub stars and licence metadata are snapshots captured on the date shown. A star count measures attention, not task success, security, recovery, or maintenance quality.
• owns_loop: claimed records what an official source says. confirmed requires a concrete mechanism or executable evidence. unknown remains unresolved.
• A benchmark score is not a property of a model name. Record the model-harness pair, task version, budget, and run count.
• Meta-harness results remain early. ADAS and AFlow are peer-reviewed; several dedicated 2026 harness-optimization results are preprints and need independent replication.

The upstream data is licensed under CC BY-SA 4.0. This guide records the source commit, preserves attribution, and adds its own classification and verification layer.