A curated, continuously-updated reading list accompanying our paper on long-horizon agents.
The Chinese edition has been reviewed for author names and terminology; please refer to the English OpenReview version for authoritative wording.
⭐ Star us if you find this useful!
- [2026/07] 📣 The brief introduction of our survey can be found on X and Xiaohongshu.
- [2026/07] 📄 Our paper Towards Long-Horizon Agents: A Survey is available on OpenReview.
- [2026/07] 🚀 We released the paper list for Towards Long-Horizon Agents: A Survey, restructured to mirror the paper chapter-by-chapter.
- [2026/07] 🙌 Open PR Request: add a missing work in PR (
[Venue Year] Title. [paper] [code]).
Figure 1. The time horizon of frontier AI agents is growing exponentially, roughly doubling every few months.
Large language models have evolved from single-turn chatbots into the decision-making core of autonomous agents. As Figure 1 shows, the time horizon of tasks they can complete unaided is growing exponentially. This surfaces one decisive requirement we call long horizon: persistent iteration across reasoning, tool use, observation, and revision over many interdependent steps — from tasks within a single context window to those spanning windows, sessions, or open-ended task streams.
Our survey frames long-horizon agency as a system-level capability jointly shaped by two forces:
- Externalized harness engineering: loops and workflows, context and memory, tools and skills, orchestration, hooks, and verification.
- Internalized model optimization: architecture, data and environment synthesis, pre-/mid-training, fine-tuning, agentic reinforcement learning, on-policy distillation, and self-evolution.
The two sides co-evolve through experience and feedback: capabilities first implemented explicitly in the harness may later be internalized into the model policy, while stronger policies in turn enable more capable harnesses. Figure 2 lays out this co-evolutionary landscape end to end.
Figure 2. The landscape of long-horizon agent research, organized around externalized harness engineering and internalized model optimization.
If you find this survey and repository useful for your research, please consider citing:
@article{dong2026longhorizon,
doi = {10.20944/preprints202607.1328.v1},
url = {https://doi.org/10.20944/preprints202607.1328.v1},
year = 2026,
month = {July},
publisher = {Preprints},
author = {Guanting Dong and Xiaoshuai Song and Yuyang Hu and Jiajie Jin and Chenghao Zhang and Yifei Chen and Xiaoxi Li and Huaying Yuan and Xinyu Yang and Tongyu Wen and Jiejun Tan and Hongjin Qian and Shijue Huang and Junting Lu and Zhenyu Li and Wanjun Zhong and Yutao Zhu and Tat-Seng Chua and Zhicheng Dou and Ji-Rong Wen},
title = {Towards Long-Horizon Agents: A Survey},
journal = {Preprints}
}
- Foundations: Formalizing Long-Horizon Agents
- Evolution: From Prompting to Runtime
- Harnesses: Externalizing Long-Horizon Capability (Pillar I)
- Optimization: Internalizing Long-Horizon Capability (Pillar II)
- Applications: Long-Horizon Agents in Practice
- Benchmarks and Resources
- Frontiers: Open Problems
- Citation
- Contributing
Section figure. Three levels of long-horizon tasks (H1 ⊂ H2 ⊂ H3) and their required capabilities (C1 ⊂ C2 ⊂ C3).
We formalize a long-horizon agent as a base policy coupled to a surrounding harness,
| Level | Task horizon | Demanded capability |
|---|---|---|
| H1 | Intra-context, within one window (~minutes) | C1 — Intra-context interactive reasoning |
| H2 | Cross-context, across windows/sessions (~hours–days) | C2 — Cross-context state & memory |
| H3 | Cross-task, open-ended task stream | C3 — Cross-task experience accumulation |
To make the notion of "horizon" concrete, METR measures capability as the length of tasks an agent can complete at a fixed success rate (e.g., the 50%-task-completion time horizon), giving an empirical yardstick that separates long-horizon agency from adjacent notions such as long-running execution, autonomy, and self-evolution.
Section figure. Three stages of co-evolution: from the language of a prompt, to the information per call, to the whole trajectory sustained by a runtime harness.
NeurIPS 2020Language Models are Few-Shot Learners. [paper]NeurIPS 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. [paper]NeurIPS 2022Large Language Models are Zero-Shot Reasoners. [paper]ICLR 2023Self-Consistency Improves Chain of Thought Reasoning in Language Models. [paper]ICLR 2023Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. [paper]ICLR 2023ReAct: Synergizing Reasoning and Acting in Language Models. [paper] [code]ICML 2023PAL: Program-aided Language Models. [paper] [code]TMLR 2023Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. [paper] [code]NeurIPS 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models. [paper] [code]CoRL 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. [paper] [code]NeurIPS 2022Training Language Models to Follow Instructions with Human Feedback. [paper]ICLR 2023Large Language Models Are Human-Level Prompt Engineers. [paper] [code]EMNLP 2023Automatic Prompt Optimization with “Gradient Descent” and Beam Search. [paper]arXiv 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models. [paper]EMNLP-IJCNLP 2019Language Models as Knowledge Bases?. [paper]NeurIPS 2023Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents. [paper] [code]
NeurIPS 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. [paper]ACL 2023Precise Zero-Shot Dense Retrieval without Relevance Labels. [paper]ICLR 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. [paper] [code]ICLR 2024Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [paper] [code]NeurIPS 2023Toolformer: Language Models Can Teach Themselves to Use Tools. [paper]NeurIPS 2024Gorilla: Large Language Model Connected with Massive APIs. [paper] [code]ICLR 2024ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. [paper] [code]arXiv 2021WebGPT: Browser-assisted Question-answering with Human Feedback. [paper]NeurIPS 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. [paper] [code]NeurIPS 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. [paper] [code]TACL 2024Lost in the Middle: How Language Models Use Long Contexts. [paper]arXiv 2023MemGPT: Towards LLMs as Operating Systems. [paper] [code]UIST 2023Generative Agents: Interactive Simulacra of Human Behavior. [paper] [code]EMNLP 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. [paper] [code]arXiv 2023Retrieval-Augmented Generation for Large Language Models: A Survey. [paper]ICML 2020REALM: Retrieval-Augmented Language Model Pre-Training. [paper]arXiv 2024RULER: What's the Real Context Size of Your Long-Context Language Models?. [paper]arXiv 2025ACON: Optimizing Context Compression for Long-horizon LLM Agents. [paper]arXiv 2025A Survey of Context Engineering for Large Language Models. [paper]arXiv 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases. [paper]TACL 2022♫ MuSiQue: Multihop Questions via Single-hop Question Composition. [paper]ACL 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokens. [paper]
NeurIPS 2023Reflexion: Language Agents with Verbal Reinforcement Learning. [paper] [code]NeurIPS 2023Self-Refine: Iterative Refinement with Self-Feedback. [paper] [code]ICML 2024Executable Code Actions Elicit Better LLM Agents. [paper] [code]ICLR 2024MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. [paper] [code]COLM 2024AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. [paper] [code]ACL 2024ChatDev: Communicative Agents for Software Development. [paper] [code]arXiv 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks. [paper] [code]Open standard 2024Model Context Protocol. [paper] [code]ICLR 2025OpenHands: An Open Platform for AI Software Developers as Generalist Agents. [paper] [code]NeurIPS 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. [paper] [code]ICLR 2026Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. [paper]COLM 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [paper] [code]Web specification 2025AGENTS.md. [paper]arXiv 2024AutoFlow: Automated Workflow Generation for Large Language Model Agents. [paper]arXiv 2026Agents' Last Exam. [paper]arXiv 2026Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agents. [paper]arXiv 2026Claw-R1: A Step-Level Data Middleware System for Agentic Reinforcement Learning. [paper]arXiv 2025SEW: Self-Evolving Agentic Workflows for Automated Code Generation. [paper]arXiv 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System. [paper]FSE 2026AgentBound: Securing Execution Boundaries of AI Agents. [paper]arXiv 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning. [paper]Preprints 2026Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime. [paper]arXiv 2026LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks. [paper] [code]
Section figure. An agent harness in action: six components sustain a single goal across many dependent steps.
Linear Workflows
ICLR 2023ReAct: Synergizing Reasoning and Acting in Language Models. [paper] [code]NeurIPS 2023Reflexion: Language Agents with Verbal Reinforcement Learning. [paper] [code]NeurIPS 2023Self-Refine: Iterative Refinement with Self-Feedback. [paper] [code]ICLR 2024Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [paper] [code]TMLR 2024Cognitive Architectures for Language Agents. [paper] [code]IEEE ICAIBD 2025A Survey on Agent Workflow - Status and Future. [paper]
Plan-Execute Workflows
ACL 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. [paper] [code]arXiv 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models. [paper] [code]Findings of NAACL 2024ADaPT: As-Needed Decomposition and Planning with Language Models. [paper] [code]arXiv 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL. [paper]arXiv 2026Toward Generalist Autonomous Research via Hypothesis-Tree Refinement. [paper]arXiv 2026Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution. [paper]
Branching Workflows
NeurIPS 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models. [paper] [code]ICLR 2023Self-Consistency Improves Chain of Thought Reasoning in Language Models. [paper]ICML 2024Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models. [paper] [code]NAACL 2025CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models. [paper]AAMAS 2026ReAcTree: Hierarchical LLM Agent Trees with Control Flow for Long-Horizon Task Planning. [paper]AAAI 2024Graph of Thoughts: Solving Elaborate Problems with Large Language Models. [paper]TMLR 2025Tree Search for Language Model Agents. [paper]Findings of ACL 2026Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search. [paper]arXiv 2025Sherlock: Reliable and Efficient Agentic Workflow Execution. [paper]
Working Context (discard / compress / select)
Anthropic Blog 2025Effective Context Engineering for AI Agents. [paper]arXiv 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization. [paper] [code]ICLR 2026MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent. [paper] [code]ICLR 2026MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents. [paper] [code]ACL 2025HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model. [paper] [code]ICLR 2026Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. [paper]ACL Findings 2026Memory-as-Action: Autonomous Context Curation for Long-Horizon Agentic Tasks. [paper] [code]arXiv 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models. [paper]arXiv 2026MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verification. [paper]arXiv 2025IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling. [paper]arXiv 2026AgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoning. [paper]arXiv 2026ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents. [paper]AAMAS 2026LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automation. [paper]arXiv 2026Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory. [paper]arXiv 2026SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent. [paper]arXiv 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving. [paper]
Persistent Memory (factual / experiential)
ECAI 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. [paper] [code]NeurIPS 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. [paper] [code]NeurIPS 2025A-Mem: Agentic Memory for LLM Agents. [paper] [code]EMNLP 2025Memory OS of AI Agent. [paper] [code]AAAI 2024ExpeL: LLM Agents Are Experiential Learners. [paper] [code]ICML 2025Agent Workflow Memory. [paper] [code]ICLR 2026ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. [paper]TMLR 2024Voyager: An Open-Ended Embodied Agent with Large Language Models. [paper] [code]arXiv 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memory. [paper]Findings of ACL 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms. [paper]NeurIPS 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems. [paper]arXiv 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills. [paper]arXiv 2025MIRIX: Multi-Agent Memory System for LLM-Based Agents. [paper]AAAI 2026PRIME: Planning and Retrieval-Integrated Memory for Enhanced Reasoning. [paper]EACL 2026H-MEM: Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents. [paper]WWW 2025MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation. [paper]ACL 2026EverMemOS: A Self-Organizing Memory Operating System for Structured Long-Horizon Reasoning. [paper]arXiv 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks. [paper]arXiv 2025Memory in the Age of AI Agents. [paper]arXiv 2026Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations. [paper]arXiv 2026From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory. [paper]arXiv 2026Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey. [paper]arXiv 2026MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning. [paper]arXiv 2025From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs. [paper]arXiv 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering. [paper]arXiv 2026Unified Context Evolution for LLM Agents. [paper]arXiv 2026Inference-Time Budget Control for LLM Search Agents. [paper]
Tool interfaces & protocols
NeurIPS 2023Toolformer: Language Models Can Teach Themselves to Use Tools. [paper]ICLR 2024ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. [paper] [code]2024Model Context Protocol (MCP) Specification. [paper] [code]ICML 2025The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models. [paper] [code]ICLR 2025τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. [paper] [code]arXiv 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers. [paper] [code]ICML 2026ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox. [paper]ICLR 2026VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications. [paper]arXiv 2026Schema First Tool APIs for LLM Agents: A Controlled Study of Tool Misuse, Recovery, and Budgeted Performance. [paper]arXiv 2026AutomationBench. [paper]ICML 2026UltraHorizon: Benchmarking LLM-Agent Capabilities in Ultra Long-Horizon Scenarios. [paper]arXiv 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration. [paper]ICLR 2026Benchmarking LLM Tool-Use in the Wild. [paper]
Active tool discovery
arXiv 2025RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation. [paper]ICML 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls. [paper] [code]arXiv 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agents. [paper]ICLR 2025ToolGen: Unified Tool Retrieval and Calling via Generation. [paper] [code]WWW 2026DeepAgent: A General Reasoning Agent with Scalable Toolsets. [paper] [code]arXiv 2026From Tool Orchestration to Code Execution: A Study of MCP Design Choices. [paper]arXiv 2026Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction. [paper]arXiv 2025From REST to MCP: An Empirical Study of API Wrapping and Automated Server Generation for LLM Agents. [paper]arXiv 2025Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use. [paper]arXiv 2026CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents. [paper]arXiv 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward. [paper]
Skill libraries
-
TMLR 2024Voyager: An Open-Ended Embodied Agent with Large Language Models. [paper] [code] -
2025Introducing Agent Skills. [paper] -
arXiv 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization. [paper] -
arXiv 2026Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning. [paper] -
arXiv 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution. [paper] -
COLM 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents. [paper] -
ICML 2026Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis. [paper] -
arXiv 2026SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?. [paper] -
arXiv 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale. [paper] -
arXiv 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail. [paper] -
arXiv 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs. [paper] -
arXiv 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMs. [paper] -
ICML 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents. [paper] -
ICML 2026SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience. [paper] -
arXiv 2026Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning. [paper] -
arXiv 2026GraSP: Graph-Structured Skill Compositions for LLM Agents. [paper] -
arXiv 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI. [paper] -
arXiv 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications. [paper] -
arXiv 2026SkillNet: Create, Evaluate, and Connect AI Skills. [paper]
Decomposition & roles
ICLR 2024MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. [paper] [code]NeurIPS 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society. [paper] [code]ACL 2024ChatDev: Communicative Agents for Software Development. [paper] [code]COLM 2024AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations. [paper] [code]Neural Networks 2025TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation. [paper]ICLR 2025Agent-Oriented Planning in Multi-Agent Systems. [paper]
Coordination topologies
arXiv 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks. [paper] [code]ICLR 2024AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. [paper] [code]ICLR 2025Mixture-of-Agents Enhances Large Language Model Capabilities. [paper] [code]arXiv 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration. [paper] [code]ICLR 2025Scaling Large Language Model-based Multi-Agent Collaboration. [paper] [code]ICLR 2026Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution. [paper]NeurIPS 2024Self-playing Adversarial Language Game Enhances LLM Reasoning. [paper]NeurIPS 2025Agint: Agentic Graph Compilation for Software Engineering Agents. [paper]arXiv 2026AgentRadio: Passive Awareness for Long-Horizon Multi-Agent Collaboration. [paper] [code]arXiv 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs. [paper]
Orchestration optimization
ICLR 2025AFlow: Automating Agentic Workflow Generation. [paper] [code]ICML 2024GPTSwarm: Language Agents as Optimizable Graphs. [paper] [code]ACL 2025MasRouter: Learning to Route LLMs for Multi-Agent Systems. [paper] [code]EMNLP 2025SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence. [paper]NeurIPS 2025Multi-Agent Collaboration via Evolving Orchestration. [paper]arXiv 2025AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent (TEA) Protocol. [paper]ICML 2026AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration. [paper]arXiv 2026CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery. [paper]arXiv 2026Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures. [paper]
Agent protocols
2025Agent2Agent (A2A) Protocol Specification. [paper] [code]2025Agent Communication Protocol (ACP, IBM). [paper]2024Model Context Protocol (MCP) Specification. [paper] [code]arXiv 2025A Survey of AI Agent Protocols. [paper]arXiv 2025A Survey of Agent Interoperability Protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP). [paper]arXiv 2024A Scalable Communication Protocol for Networks of Large Language Models. [paper]IBM Research 2025Agent Communication Protocol (ACP). [paper]arXiv 2026Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks. [paper]arXiv 2026Agentic Test-Time Scaling for WebAgents. [paper]arXiv 2025LOKA Protocol: A Decentralized Framework for Trustworthy and Ethical AI Agent Ecosystems. [paper]
Pre-defined rule-based hooks
ICSA 2025Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agents. [paper]arXiv 2026AEGIS: No Tool Call Left Unchecked — A Pre-Execution Firewall and Audit Layer for AI Agents. [paper]arXiv 2026Authenticated Workflows: A Systems Approach to Protecting Agentic AI. [paper]arXiv 2025Magentic-UI: Towards Human-in-the-loop Agentic Systems. [paper]NDSS 2025IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems. [paper]
Custom user-defined hooks
EMNLP 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. [paper] [code]ICSE 2026AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents. [paper]arXiv 2025Progent: Securing AI Agents with Privilege Control. [paper]ICML 2025GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning. [paper]ICML 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning. [paper]arXiv 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. [paper] [code]arXiv 2025LlamaFirewall: An open source guardrail system for building secure AI agents. [paper]SaTML 2026Defeating Prompt Injections by Design. [paper]ICLR 2026 Workshop (VerifAI-2)Enforcing Temporal Constraints for LLM Agents. [paper]NeurIPS 2025 Workshop (RegML)Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents. [paper]arXiv 2026Formal Policy Enforcement for Real-World Agentic Systems. [paper]arXiv 2025VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation. [paper]
Runtime-adaptive hooks
Nature 2024Detecting hallucinations in large language models using semantic entropy. [paper]arXiv 2025SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems. [paper]ACL 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection. [paper]ASE 2025AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software. [paper]AAAI 2026 Workshop (TrustAgent)Agent-SafetyBench: Evaluating the Safety of LLM Agents. [paper] [code]AISTATS 2026Enhancing LLM Safety Through a Theoretical Minimax Game Lens. [paper]arXiv 2026Quantifying Frontier LLM Capabilities for Container Sandbox Escape. [paper]arXiv 2026Neuro-Symbolic Verification on Instruction Following of LLMs. [paper]ACL 2025Uncertainty Propagation on LLM Agent. [paper]arXiv 2026TrajAD: Trajectory Anomaly Detection for Trustworthy LLM Agents. [paper]arXiv 2025ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety. [paper]arXiv 2026Agentic Uncertainty Quantification. [paper]
Assessment targets
EMNLP 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. [paper] [code]ICML 2024Improving Factuality and Reasoning in Language Models through Multiagent Debate. [paper] [code]ICLR 2025 Workshop (MCDC)Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers. [paper]ICLR 2025AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. [paper]AAAI 2026 Workshop (TrustAgent)Agent-SafetyBench: Evaluating the Safety of LLM Agents. [paper] [code]arXiv 2026AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security. [paper]arXiv 2026AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security. [paper]
Verification levels
ICLR 2024Let's Verify Step by Step. [paper] [code]ICLR 2024CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing. [paper] [code]NeurIPS 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. [paper] [code]ICML 2025Agent-as-a-Judge: Evaluate Agents with Agents. [paper] [code]NeurIPS 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agents. [paper]arXiv 2021Training Verifiers to Solve Math Word Problems. [paper]arXiv 2022Solving math word problems with process- and outcome-based feedback. [paper]ICLR 2025Generative Verifiers: Reward Modeling as Next-Token Prediction. [paper]arXiv 2026SWE-TRACE: Optimizing Long-Horizon SWE Agents Through Rubric Process Reward Models and Heuristic Test-Time Scaling. [paper]arXiv 2026CollabEval: Enhancing LLM-as-a-Judge via Multi-Agent Collaboration. [paper]arXiv 2026Agentic Reward Modeling: Verifying GUI Agent via Online Proactive Interaction. [paper]arXiv 2026Agent-as-a-Judge. [paper]COLM 2025Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers. [paper]
Verifier strategies
ACL 2024Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations. [paper]ICML 2025Free Process Rewards without Process Labels. [paper] [code]ICML 2024Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models. [paper] [code]NeurIPS 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search. [paper] [code]ICLR 2025Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Parameters for Reasoning. [paper]arXiv 2025MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation. [paper]arXiv 2026Scaling Medical Reasoning Verification via Tool-Integrated Reinforcement Learning. [paper]NeurIPS 2025Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search. [paper]
Section figure. The agentic training pipeline: an architectural substrate plus six training stages for internalizing long-horizon capability.
arXiv 2020Longformer: The Long-Document Transformer. [paper] [code]NeurIPS 2020Big Bird: Transformers for Longer Sequences. [paper]COLM 2024Mamba: Linear-Time Sequence Modeling with Selective State Spaces. [paper] [code]ICML 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality. [paper] [code]EMNLP 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. [paper]arXiv 2024DeepSeek-V3 Technical Report. [paper] [code]ICLR 2025Jamba: Hybrid Transformer-Mamba Language Models. [paper]arXiv 2025Kimi Linear: An Expressive, Efficient Attention Architecture. [paper]ICML 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty. [paper] [code]ICML 2023Fast Inference from Transformers via Speculative Decoding. [paper]arXiv 2025MoBA: Mixture of Block Attention for Long-Context LLMs. [paper]arXiv 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference. [paper]arXiv 2019Fast Transformer Decoding: One Write-Head is All You Need. [paper]arXiv 2020Linformer: Self-Attention with Linear Complexity. [paper]arXiv 2023Retentive Network: A Successor to Transformer for Large Language Models. [paper]arXiv 2023Mistral 7B. [paper]arXiv 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. [paper]arXiv 2024Gemma 2: Improving Open Language Models at a Practical Size. [paper]arXiv 2025Gemma 3 Technical Report. [paper]arXiv 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. [paper]arXiv 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models. [paper]arXiv 2025Qwen3 Technical Report. [paper]
ICLR 2026TaskCraft: Automated Generation of Agentic Tasks. [paper] [code]ICLR 2026WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization. [paper] [code]arXiv 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation. [paper] [code]CVPR 2026Socratic-Geo: Synthetic Data Generation and Cross-Modal Geometric Reasoning via Multi-Agent Interaction. [paper] [code]ICLR 2026Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesis. [paper]ICML 2025Training Software Engineering Agents and Verifiers with SWE-Gym. [paper] [code]ICLR 2024WebArena: A Realistic Web Environment for Building Autonomous Agents. [paper] [code]NeurIPS 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. [paper] [code]TMLR 2025Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents. [paper] [code]arXiv 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments. [paper]ICLR 2026AgentGym-RL: An Open-Source Framework to Train LLM Agents for Long-Horizon Decision Making via Multi-Turn RL. [paper] [code]arXiv 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence. [paper]arXiv 2026GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents. [paper]NeurIPS 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge. [paper]arXiv 2026AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis. [paper]arXiv 2026ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas. [paper]NeurIPS 2025 Workshop (MTI-LLM)Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL. [paper]ICLR 2026AgentFold: Long-Horizon Web Agents with Proactive Context Folding. [paper]arXiv 2026AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling. [paper]arXiv 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework. [paper]COLM 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories. [paper]ICLR 2026AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents. [paper]arXiv 2026AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines. [paper]arXiv 2026CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion. [paper]arXiv 2026Code2World: A GUI World Model via Renderable Code Generation. [paper]arXiv 2025Scaling Long-Horizon LLM Agent via Context-Folding. [paper]arXiv 2025Cosmos World Foundation Model Platform for Physical AI. [paper]arXiv 2025CuES: A Curiosity-driven and Environment-grounded Synthesis Framework for Agentic RL. [paper]arXiv 2025Back to the Features: DINO as a Foundation for Video World Models. [paper]arXiv 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use. [paper]CoRL 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models. [paper]arXiv 2026World Action Models are Zero-shot Policies. [paper]arXiv 2026FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation. [paper]arXiv 2026GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation. [paper]arXiv 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators. [paper]CVPR 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agents. [paper]arXiv 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws. [paper]arXiv 2026Computer Environments Elicit General Agentic Intelligence in LLMs. [paper]arXiv 2026The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break. [paper]ICML 2026On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length. [paper]NeurIPS 2025 Workshop (SEA)MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers. [paper]arXiv 2025MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Use. [paper]arXiv 2025Matrix-Game: Interactive World Foundation Model. [paper]arXiv 2025MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science. [paper]arXiv 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft. [paper]arXiv 2026MobileDreamer: Generative Sketch World Model for GUI Agent. [paper]ICLR 2026OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents. [paper]arXiv 2026OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis. [paper]arXiv 2026PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement. [paper]arXiv 2025PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature. [paper]arXiv 2026SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks. [paper]arXiv 2026SWE-World: Building Software Engineering Agents in Docker-Free Environments. [paper]arXiv 2026Immersion in the GitHub Universe: Scaling Coding Agents to Mastery. [paper]arXiv 2026SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents. [paper]arXiv 2026A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks. [paper]arXiv 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. [paper]arXiv 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skills. [paper]arXiv 2025LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training. [paper]arXiv 2026Verifiable Process Rewards for Agentic Reasoning. [paper]arXiv 2026Safe and Scalable Web Agent Learning via Recreated Websites. [paper]arXiv 2025WebSailor: Navigating Super-human Reasoning for Web Agent. [paper]arXiv 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning. [paper]arXiv 2026WebWorld: A Large-Scale World Model for Web Agent Training. [paper]arXiv 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application. [paper]NeurIPS 2025 Workshop (SEA)Environment Scaling for Interactive Agentic Experience Collection: A Survey. [paper]arXiv 2026Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents. [paper]NeurIPS 2025seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models. [paper]ICLR 2026MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use. [paper]arXiv 2026Qwen-AgentWorld: Language World Models for General Agents. [paper]
-
arXiv 2025Kimi K2: Open Agentic Intelligence. [paper] [code] -
ICLR 2024YaRN: Efficient Context Window Extension of Large Language Models. [paper] [code] -
ICLR 2024LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models. [paper] [code] -
ACL 2025How to Train Long-Context Language Models (Effectively). [paper] [code] -
arXiv 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. [paper] [code] -
arXiv 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. [paper] [code] -
NeurIPS 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining. [paper] -
arXiv 2026BabyVision: Visual Reasoning Beyond Language. [paper] -
ICLR 2026Vision Language Models are Biased. [paper] -
ACCV 2024Vision language models are blind. [paper] -
CVPR 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs. [paper] -
NeurIPS 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs. [paper] -
arXiv 2024Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries. [paper] -
arXiv 2026CL-bench: A Benchmark for Context Learning. [paper] -
arXiv 2026GLM-5: from Vibe Coding to Agentic Engineering. [paper] -
arXiv 2023Extending Context Window of Large Language Models via Positional Interpolation. [paper] -
arXiv 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining. [paper] -
arXiv 2025MiMo-VL Technical Report. [paper] -
arXiv 2025Seed1.5-VL Technical Report. [paper] -
arXiv 2026Kimi K2.5: Visual Agentic Intelligence. [paper] -
arXiv 2026Qwen3.5-Omni Technical Report. [paper] -
arXiv 2026Qwen3-Coder-Next Technical Report. [paper] -
arXiv 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency. [paper] -
ICML 2026OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration. [paper] -
NeurIPS 2022Training Compute-Optimal Large Language Models. [paper]
-
ACL Findings 2024AgentTuning: Enabling Generalized Agent Abilities for LLMs. [paper] [code] -
ACL Findings 2025ATLaS: Agent Tuning via Learning Critical Steps. [paper] -
ICML 2024Executable Code Actions Elicit Better LLM Agents. [paper] [code] -
NeurIPS 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets. [paper] [code] -
arXiv 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training. [paper] [code] -
NeurIPS 2025Distilling LLM Agent into Small Models with Retrieval and Code Tools. [paper] [code] -
arXiv 2023FireAct: Toward Language Agent Fine-tuning. [paper] -
arXiv 2025LIMR: Less is More for RL Scaling. [paper] -
arXiv 2026Unified Data Selection for LLM Reasoning. [paper] -
AAMAS 2026Structured Agent Distillation for Large Language Model Agents. [paper] -
arXiv 2026HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation. [paper] -
arXiv 2026AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent. [paper] -
ICLR 2026Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning. [paper] -
ICLR 2026Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents. [paper] -
arXiv 2026Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation. [paper]
Credit assignment · policy optimization · sampling strategy · interaction patterns. GitHub links follow the paper's Table (Agentic RL).
Credit Assignment
arXiv 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. [paper] [code]COLM 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [paper] [code]arXiv 2025DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learning. [paper] [code]NeurIPS 2025ToolRL: Reward is All Tool Learning Needs. [paper] [code]SIGIR 2026Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning. [paper] [code]arXiv 2026Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning. [paper] [code]arXiv 2025DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research. [paper] [code]ACL 2026OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment. [paper] [code]ACL 2026CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic. [paper]arXiv 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents. [paper]arXiv 2025R3: Robust Rubric-Agnostic Reward Models. [paper]arXiv 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains. [paper]arXiv 2026AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization. [paper]arXiv 2025Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training. [paper]arXiv 2025Reinforcement Learning for Long-Horizon Interactive LLM Agents. [paper]arXiv 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. [paper]arXiv 2025EAPO: Enhancing Policy Optimization with On-Demand Expert Assistance. [paper]arXiv 2025Reinforcement Learning with Rubric Anchors. [paper]arXiv 2026SmartSearch: Process Reward-Guided Query Refinement for Search Agents. [paper]arXiv 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use. [paper]arXiv 2025ToRL: Scaling Tool-Integrated RL. [paper]ICML 2026Optimizing Agentic Reasoning with Retrieval via Synthetic Semantic Information Gain Reward. [paper]arXiv 2026Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers. [paper]
Policy Optimization
arXiv 2025REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models. [paper] [code]NeurIPS 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale. [paper] [code]arXiv 2025Understanding R1-Zero-Like Training: A Critical Perspective. [paper] [code]arXiv 2025Group Sequence Policy Optimization. [paper]NIPS 2025Group-in-Group Policy Optimization for LLM Agent Training. [paper] [code]EACL 2026Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs. [paper]arXiv 2026StepPO: Step-Aligned Policy Optimization for Agentic Reinforcement Learning. [paper]arXiv 2025CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention. [paper] [code]arXiv 2025EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning. [paper] [code]EMNLP 2025EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models. [paper]arXiv 2025Agentic Reinforced Policy Optimization. [paper]arXiv 2025On Entropy Control in LLM-RL Algorithms. [paper]arXiv 2025Agentic Entropy-Balanced Policy Optimization. [paper]arXiv 2025BNPO: Beta Normalization Policy Optimization. [paper]arXiv 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models. [paper]arXiv 2025CPGD: Toward Stable Rule-based Reinforcement Learning for Language Models. [paper]arXiv 2026When Denser Credit Is Not Enough: Evidence-Calibrated Policy Optimization for Long-Horizon LLM Agent Training. [paper]arXiv 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning. [paper]arXiv 2026Hindsight Credit Assignment for Long-Horizon LLM Agents. [paper]arXiv 2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks. [paper]arXiv 2026HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents. [paper]arXiv 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention. [paper]arXiv 2025On-Policy RL with Optimal Reward Baseline. [paper]arXiv 2025PORTool: Tool-Use LLM Training with Rewarded Tree. [paper]arXiv 2017Proximal Policy Optimization Algorithms. [paper]
Sampling Strategy
-
NeurIPS 2025WebDancer: Towards Autonomous Information Seeking Agency. [paper] [code] -
ICLR 2025WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning. [paper] [code] -
ICLR 2026Tree Search for LLM Agent Reinforcement Learning. [paper] [code] -
arXiv 2025TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling. [paper] [code] -
EMNLP 2023Reasoning with Language Model is Planning with World Model. [paper] [code] -
arXiv 2026LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent. [paper] [code] -
COLM 2026TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents. [paper] [code] -
arXiv 2025A Survey of Reinforcement Learning for Large Reasoning Models. [paper] -
arXiv 2026AT$^2$PO: Agentic Turn-based Policy Optimization via Tree Search. [paper] -
arXiv 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. [paper] -
arXiv 2025Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning. [paper] -
arXiv 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library. [paper] -
arXiv 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents. [paper] -
arXiv 2026Training Multi-Turn Search Agent via Contrastive Dynamic Branch Sampling. [paper]
Interaction Patterns
ICML 2025Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning. [paper] [code]arXiv 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning. [paper] [code]ICLR 2026THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning. [paper] [code]arXiv 2025Multi-Agent Tool-Integrated Policy Optimization. [paper] [code]arXiv 2025Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs. [paper] [code]ACL 2026Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning. [paper] [code]arXiv 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learning. [paper] [code]arXiv 2025MARS: Optimizing Dual-System Deep Research via Multi-Agent Reinforcement Learning. [paper]arXiv 2025Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPO. [paper]arXiv 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning. [paper]arXiv 2025Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation. [paper]arXiv 2026FlowSteer: Interactive Agentic Workflow Orchestration via End-to-End Reinforcement Learning. [paper]
IJCNN 2024Online Policy Distillation with Decision-Attention. [paper]arXiv 2026MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate. [paper] [code]arXiv 2026KAT-Coder-V2 Technical Report. [paper]arXiv 2026LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning. [paper]arXiv 2026Revisiting DAgger in the Era of LLM-Agents. [paper]arXiv 2026SOD: Step-wise On-policy Distillation for Small Language Model Agents. [paper] [code]arXiv 2026StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning. [paper]arXiv 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation. [paper] [code]arXiv 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents. [paper] [code]arXiv 2026π-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data. [paper] [code]arXiv 2026f-OPD: Stabilizing Long-Horizon On-Policy Distillation with Freshness-Aware Control. [paper]arXiv 2026Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision. [paper] [code]arXiv 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes. [paper] [code]arXiv 2026TIP: Token Importance in On-Policy Distillation. [paper]arXiv 2026Reinforcement Learning via Self-Distillation. [paper]arXiv 2026Skill-Conditioned Gated Self-Distillation for LLM Reasoning. [paper]arXiv 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability. [paper]
-
NeurIPS 2022STaR: Bootstrapping Reasoning with Reasoning. [paper] [code] -
ICML 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI. [paper] [code] -
EMNLP 2025SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection. [paper] -
ICML 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking. [paper] [code] -
ICLR 2026R-Zero: Self-Evolving Reasoning LLM from Zero Data. [paper] [code] -
NeurIPS 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data. [paper] [code] -
ICLR 2026 Workshop (RSI Oral)Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning. [paper] -
ICLR 2026CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards. [paper] -
ICML 2026RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments. [paper] -
arXiv 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence. [paper] -
ICLR 2026Don't Just Fine-tune the Agent, Tune the Environment. [paper] -
arXiv 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems. [paper] -
ICLR 2025 Workshop (Self-Improving Foundation Models Oral)A Self-Improving Coding Agent. [paper] -
NeurIPS 2025Self-Adapting Language Models. [paper] -
arXiv 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning. [paper] -
arXiv 2025Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution. [paper] -
arXiv 2026Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills. [paper] -
arXiv 2025AgentEvolver: Towards Efficient Self-Evolving Agent System. [paper] -
arXiv 2026OpenSkill: Open-World Self-Evolution for LLM Agents. [paper]
Section figure. Representative long-horizon agent applications grouped by the agent–environment interface.
-
arXiv 2026Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testing. [paper] [code] -
arXiv 2026ProgramBench: Can Language Models Rebuild Programs From Scratch? [paper] -
arXiv 2025CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. [paper] -
arXiv 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation. [paper] -
arXiv 2026SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution. [paper] -
arXiv 2026RepoZero: Can LLMs Generate a Code Repository from Scratch? [paper] [code] -
arXiv 2026LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering. [paper] [code]
Repository grounding
NeurIPS 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. [paper] [code]ISSTA 2024AutoCodeRover: Autonomous Program Improvement. [paper] [code]ICLR 2025OpenHands: An Open Platform for AI Software Developers as Generalist Agents. [paper] [code]2023aider is AI pair programming in your terminal. [code]ICLR 2024SWE-bench: Can Language Models Resolve Real-World GitHub Issues? [paper] [code]ICML 2026NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents. [paper] [code] [paper]
Workflow-level planning
2025Claude Code. [code]2025Deep Agents: The Batteries-Included Agent Harness. [code]arXiv 2025Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling. [paper] [code]ICML 2026SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?. [paper] [code]Anthropic Engineering 2026Scaling Managed Agents: Decoupling the brain from the hands. [paper]arXiv 2025AInsteinBench: Benchmarking Coding Agents on Scientific Repositories. [paper]ICML 2025SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? [paper]
Feedback-driven repair
ICML 2025Training Software Engineering Agents and Verifiers with SWE-Gym. [paper] [code]NeurIPS 2025SWE-smith: Scaling Data for Software Engineering Agents. [paper] [code]FSE 2025Demystifying LLM-Based Software Engineering Agents. [paper] [code]NeurIPS 2025 Workshop (SEA Poster)When Agents go Astray: Course-Correcting SWE Agents with PRMs. [paper]arXiv 2025SWE-RM: Execution-free Feedback for Software Engineering Agents. [paper]
NAACL 2025Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation. [paper]
Deep search
EMNLP 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models. [paper] [code]COLM 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [paper] [code]NeurIPS 2025WebDancer: Towards Autonomous Information Seeking Agency. [paper] [code]arXiv 2025WebSeer: Training Deeper Search Agents through RL with Self-Reflection. [paper] [code]ICLR 2026ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization. [paper] [code]arXiv 2025Tongyi DeepResearch Technical Report. [paper] [code]arXiv 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling. [paper] [code]ICLR 2026FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning. [paper]arXiv 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese. [paper]arXiv 2026Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities. [paper]ICLR 2026SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models. [paper]arXiv 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes. [paper]arXiv 2025Deep Research Agents: A Systematic Examination And Roadmap. [paper]arXiv 2025Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward. [paper]arXiv 2025FoldAct: Efficient and Stable Context Folding for Long-Horizon Search Agents. [paper]arXiv 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications. [paper]ICML 2026ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards. [paper]arXiv 2025Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window. [paper]
Wide search
-
ICLR 2026WideSearch: Benchmarking Agentic Broad Info-Seeking. [paper] [code] -
ACL 2026FlowSearch: Advancing Deep Research with Dynamic Structured Knowledge Flow. [paper] [code] -
arXiv 2025Laser: Governing Long-Horizon Agentic Search via Structured Protocol and Context Register. [paper] -
arXiv 2025GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation. [paper] -
EMNLP 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? [paper] [code] -
arXiv 2026WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search. [paper] -
arXiv 2026WideSeek: Advancing Wide Research via Multi-Agent Scaling. [paper] -
arXiv 2026InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seeking. [paper] -
arXiv 2026WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning. [paper]
Multimodal grounding
ACL 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. [paper] [code]ICLR 2025Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent. [paper]ICLR 2026WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent. [paper] [code]arXiv 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents. [paper]arXiv 2026VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learning. [paper]arXiv 2026Towards Long-Horizon Agentic Multimodal Search. [paper]Findings of ACL 2025MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering. [paper]CVPR 2025OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations. [paper]ICCV 2023Document Understanding Dataset and Evaluation (DUDE). [paper]NeurIPS 2025OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning. [paper]NeurIPS 2024MMLONGBENCH-DOC: Benchmarking Long-context Document Understanding with Visualizations. [paper]ACL 2025LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating. [paper]arXiv 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents. [paper]arXiv 2025Agentic Keyframe Search for Video Question Answering. [paper]arXiv 2026Video-Browser: Towards Agentic Open-web Video Browsing. [paper]arXiv 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning. [paper]Visual Intelligence 2025Large multimodal agents: a survey. [paper]SIGIR 2026Towards Mixed-Modal Retrieval for Universal Retrieval-Augmented Generation. [paper]
Research synthesis
-
ICLR 2026WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Research. [paper] [code] -
NeurIPS 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capability. [paper] [code] -
arXiv 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. [paper] [code] -
ICLR 2026Towards Personalized Deep Research: Benchmarks and Evaluations. [paper] -
ICLR 2026DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents. [paper] [code] -
NeurIPS 2024Long-form factuality in large language models. [paper] -
EMNLP 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. [paper] -
ACL 2026FS-Researcher: Test-Time Scaling for Long-Horizon Research Tasks with File-System-Based Agents. [paper] -
arXiv 2025DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence. [paper] -
arXiv 2026AgentCPM-Report: Interleaving Drafting and Deepening for Open-Ended Deep Research. [paper] -
arXiv 2025Deep Research: A Survey of Autonomous Research Agents. [paper] -
arXiv 2026Toward Generalist Autonomous Research via Hypothesis-Tree Refinement. [paper]
arXiv 2026Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies. [paper]arXiv 2026WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation. [paper]
Browser agents
ICLR 2024WebArena: A Realistic Web Environment for Building Autonomous Agents. [paper] [code]NeurIPS 2023Mind2Web: Towards a Generalist Agent for the Web. [paper] [code]ACL 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. [paper] [code]2024browser-use: Make Websites Accessible for AI Agents. [code]TMLR 2026BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions. [paper] [code]EMNLP 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?. [paper] [code]arXiv 2026ClawBench: Can AI Agents Complete Everyday Online Tasks? [paper]arXiv 2025Recon-Act: A Self-Evolving Multi-Agent Browser-Use System via Web Reconnaissance, Tool Generation, and Task Execution. [paper]arXiv 2025WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scale. [paper]ICLR 2025 Workshop (SSI-FM Poster)NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild. [paper]ACL 2026 Industry TrackColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolution. [paper]
Desktop GUI agents
NeurIPS 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. [paper] [code]arXiv 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents. [paper] [code]CVPR 2024CogAgent: A Visual Language Model for GUI Agents. [paper] [code]ACM MM 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use. [paper] [code]2024Anthropic Computer Use (Quickstarts). [code]ICLR 2025Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. [paper] [code]arXiv 2026Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields. [paper]arXiv 2026VISUALSKILL: Multimodal Skills for Computer-Use Agents. [paper]arXiv 2026AliyunConsoleAgent: Training Web Agents in Real-World Cloud Environments via Distillation and Reinforcement Learning. [paper]NeurIPS 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents. [paper]
Mobile agents
ICLR 2024 WorkshopMobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception. [paper] [code]ICLR 2025AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents. [paper] [code]ACL 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents. [paper] [code]arXiv 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation. [paper] [code]NeurIPS 2025 Workshop (SEA Oral)Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks. [paper] [code]ACL 2026MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments. [paper]MobiCom 2026V-Droid: Advancing Mobile GUI Agent Through Generative Verifiers. [paper]arXiv 2025MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment. [paper]arXiv 2025AutoEval: A Practical Framework for Autonomous Evaluation of Mobile Agents. [paper]arXiv 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents. [paper]
arXiv 2024Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models. [paper]ICCV 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models. [paper]CVPR 2024HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models. [paper]NeurIPS 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly. [paper]AAAI 2026Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models. [paper]ICLR 2025CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding. [paper]TMLR 2026VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation. [paper]AAAI 2026CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models. [paper]CVPR 2024 Workshop (SynData4CV; Best Short Paper)GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation. [paper]arXiv 2024Agent AI: Surveying the Horizons of Multimodal Interaction. [paper]
Multimodal understanding
ECCV 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent. [paper] [code]NeurIPS 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding. [code] [paper]KDD 2026VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos. [paper]CVPR 2025Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. [paper] [code]arXiv 2025Qwen2.5-VL Technical Report. [paper] [code]CVPR 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking. [paper] [code]NeurIPS 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memory. [paper]NeurIPS 2025AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding. [paper]ICLR 2026A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answering. [paper]arXiv 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios. [paper]arXiv 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs. [paper]
Multimodal generation
arXiv 2025MM-StoryAgent: Immersive Narrated Storybook Video Generation with a Multi-Agent Paradigm across Text, Image and Audio. [paper] [code]arXiv 2026Qwen-Image-2.0 Technical Report. [paper]arXiv 2025Seedream 3.0 Technical Report. [paper]arXiv 2026MUSE: A Multi-agent Framework for Unconstrained Story Envisioning via Closed-Loop Cognitive Orchestration. [paper]arXiv 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation. [paper]arXiv 2026Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation. [paper] [code]arXiv 2025Generative Universal Verifier as Multimodal Meta-Reasoner. [paper]arXiv 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editing. [paper]arXiv 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning. [paper]arXiv 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. [paper]arXiv 2026MemoGen: Can Past Experience Improve Future Text-to-Image Generation?. [paper]arXiv 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis. [paper]arXiv 2025Emerging Properties in Unified Multimodal Pretraining. [paper]arXiv 2026Seedance 2.0: Advancing Video Generation for World Complexity. [paper]arXiv 2026Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing. [paper]
Omnimodal agency
arXiv 2025Qwen3-Omni Technical Report. [paper] [code]arXiv 2025Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anything. [paper] [code]arXiv 2026OmniGAIA: Towards Native Omni-Modal AI Agents. [code]TrustCom 2025OmniNova: A General Multimodal Agent Framework. [paper]arXiv 2026Orchestra-o1: Omnimodal Agent Orchestration. [paper] [code]arXiv 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning. [paper]ICML 2026Native Active Perception as Reasoning for Omni-Modal Understanding. [paper]arXiv 2025Qwen2.5-Omni Technical Report. [paper]
Personal assistants
2023AutoGPT: Autonomous Goal Pursuit. [code]2025Manus: A General Autonomous Agent Product. [paper]ICLR 2024GAIA: a benchmark for General AI Assistants. [paper]arXiv 2026SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineering. [paper]arXiv 2026APEX-Agents. [paper]2025OpenManus: An Open-Source Framework for Building General AI Agents. [code]2025Claude Code overview. [paper]2025Introducing Codex. [paper]arXiv 2026PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation. [paper]NeurIPS 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation. [paper]arXiv 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents. [paper]arXiv 2026ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents. [paper]arXiv 2026MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers. [paper]arXiv 2026OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning. [paper]ICLR 2026The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution. [paper]
Embodied agents and world models
-
arXiv 2025Gemini Robotics: Bringing AI into the Physical World. [paper] -
arXiv 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. [paper] [code] -
CoRL 2024OpenVLA: An Open-Source Vision-Language-Action Model. [paper] [code] -
arXiv 2025π₀.₅: a Vision-Language-Action Model with Open-World Generalization. [paper] [code] -
arXiv 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. [paper] [code] -
ICML 2025DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning. [paper] [code] -
AAAI 2026Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs. [paper] -
ICLR 2026MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence. [paper] -
NeurIPS 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics. [paper] -
arXiv 2025PhyX: Does Your Model Have the "Wits" for Physical Reasoning? [paper] -
ICML 2024ContPhy: Continuum Physical Concept Learning and Reasoning from Videos. [paper] -
ICLR 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding. [paper] -
ACL 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models. [paper] -
arXiv 2025World Simulation with Video Foundation Models for Physical AI. [paper] -
arXiv 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models. [paper] -
arXiv 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System. [paper] -
arXiv 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments. [paper] -
arXiv 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI. [paper] -
arXiv 2026OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics. [paper] -
arXiv 2026Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy. [paper] -
arXiv 2026$\tau_0\text{-}\mathbf{WM}$ : A Unified Video-Action World Model for Robotic Manipulation. [paper] -
ICML 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents. [paper]
Productive agents
arXiv 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. [paper] [code]arXiv 2025AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. [paper] [code]arXiv 2025Kosmos: An AI Scientist for Autonomous Discovery. [paper] [code]ICAIF 2024 WorkshopFinRobot: AI Agent for Equity Research and Valuation with Large Language Models. [paper] [code]ACL 2025LegalAgentBench: Evaluating LLM Agents in Legal Domain. [paper] [code]NEJM AI 2025MedAgentBench: A Virtual EHR Environment to Benchmark Medical LLM Agents. [paper] [code]arXiv 2026OneMillion-Bench: How Far are Language Agents from Human Experts?arXiv 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. [paper]arXiv 2026PostTrainBench: Can LLM Agents Automate LLM Post-Training? [paper]arXiv 2026HorizonMath: Measuring AI Progress Toward Mathematical Discovery with Automatic Verification. [paper]arXiv 2026FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks. [paper]arXiv 2026BABE: Biology Arena BEnchmark. [paper]ICLR 2022miniF2F: a cross-system benchmark for formal Olympiad-level mathematics. [paper]NeurIPS 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competition. [paper]arXiv 2025HealthBench: Evaluating Large Language Models Towards Improved Human Health. [paper]ICML 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding. [paper]arXiv 2025Robin: A multi-agent system for automating scientific discovery. [paper]arXiv 2025InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification. [paper]arXiv 2025Prover Agent: An Agent-Based Framework for Formal Mathematical Proofs. [paper]arXiv 2025Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experience. [paper]arXiv 2026LawThinker: A Deep Research Legal Agent in Dynamic Environments. [paper]arXiv 2026MedCTA: A Benchmark for Clinical Tool Agents. [paper]arXiv 2025Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks. [paper]arXiv 2026Parthenon Law: A Self-Evolving Legal-Agent Framework. [paper]arXiv 2026Benchmark Test-Time Scaling of General LLM Agents. [paper]arXiv 2026PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments. [paper]arXiv 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery. [paper]ICML 2026APE-Bench: Evaluating Automated Proof Engineering for Formal Math Libraries. [paper]arXiv 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search. [paper]
A consolidated set of open-source benchmarks and reusable systems, organized by application domain (matching the paper's resource table). Links point to public code repositories.
OpenAI 2024SWE-bench Verified. [code]arXiv 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?. [code]arXiv 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. [code]ACL 2026OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding. [code]Findings of ACL 2024DebugBench: Evaluating Debugging Capability of Large Language Models. [code]arXiv 2026RepoZero: Can LLMs Generate a Code Repository from Scratch? [code]
EMNLP 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?. [code]arXiv 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. [code]ICLR 2026WideSearch: Benchmarking Agentic Broad Info-Seeking. [code]ICLR 2026DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents. [code]arXiv 2025Tongyi DeepResearch Technical Report. [code]GitHub Resource 2025DeepConsult: A Deep Research Benchmark for Consulting / Business Queries [code]Findings of ACL 2025MMInA: Benchmarking Multihop Multimodal Internet Agents. [code]arXiv 2026GISA: A Benchmark for General Information-Seeking Assistant. [code]
ICLR 2024WebArena: A Realistic Web Environment for Building Autonomous Agents. [code]NeurIPS 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. [code]ACL 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents. [code]ICLR 2025AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents. [code]ACL 2025AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents. [code]arXiv 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointing. [code]
CVPR 2025Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. [code]NeurIPS 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. [code]NeurIPS 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. [code]CVPR 2024 WorkshopEvaluating and Improving Compositional Text-to-Visual Generation. [code]arXiv 2026OmniGAIA: Towards Native Omni-Modal AI Agents. [code]CVPR 2024MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. [code]ICASSP 2026StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding. [code]ICLR 2026OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs. [code]
ICLR 2025τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. [code]ACL 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents. [code]ICML 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents. [code]NeurIPS 2025WorldModelBench: Judging Video Generation Models As World Models. [code]ACL 2025LegalAgentBench: Evaluating LLM Agents in Legal Domain. [code]NEJM AI 2025MedAgentBench: A Virtual EHR Environment to Benchmark Medical LLM Agents. [code]arXiv 2026OneMillion-Bench: How Far are Language Agents from Human Experts?arXiv 2026YC-Bench: Benchmarking AI Agents for Long-Term Planning and Consistent Execution. [code]arXiv 2026Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity. [code]ECCV 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments. [code]Nature Machine Intelligence 2024Augmenting large language models with chemistry tools. [code]
Additional autonomy-stressing suites: MLE-bench (ML engineering) [code], PaperBench (paper replication) [code], and the METR time-horizon measurements [code].
We group open problems into four axes spanning nine concrete directions. A recurring thread: the harness, not the model alone, is where much of the next advance must happen.
| Axis | Frontier | Core open challenge |
|---|---|---|
| I. Evolution | Self-evolving harness & agents | Objective is a hand-set metric; gains stay in-distribution; long runs overfit/drift |
| Harness transferability | Models bind to one harness; rankings swing across providers; no standard protocol | |
| Continual & lifelong learning | External memory is shallow; internal updates risk forgetting | |
| II. Effectiveness | Real-world environment interaction | No direct training in live systems; synthesis & world-models face a fidelity test |
| From digital to embodied agents | Timescale conflict; physics/dimensionality gap; coarse-vs-fine feedback | |
| III. Efficiency | Cost- & budget-aware agency | Budget-blind; no calibrated cost sense; no runtime ceilings; no budget↔success law |
| Multimodal & omni harness | Multimodality bolted on; heuristic visual-token budgeting; unreliable cross-modal verification | |
| IV. Trustworthiness | Reflection & error robustness | Late failure detection; unreliable intrinsic self-correction; errors compound into goal drift |
| Safety & governance | Injected error/hazardous experience reuse; no unified safety standard; self-evolution erodes invariants |
Representative references for the frontiers:
ICLR 2024Large Language Models Cannot Self-Correct Reasoning Yet. [paper]ICLR 2025Scaling LLM Test-Time Compute Optimally. [paper]ICLR 2025RouteLLM: Learning to Route LLMs with Preference Data. [paper] [code]NeurIPS 2025TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. [paper] [code]ICLR 2026Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. [paper]
Contributions are very welcome! Please open a Pull Request to add a missing paper or fix a link. When adding a work, keep the per-line format consistent:
- **`Venue Year`** Title. [[paper](URL)] [[code](URL)]
Guidelines:
- Prefer the acceptance venue (e.g.,
ICML 2024,NeurIPS 2023,ICLR 2025); usearXiv YYYYonly when a work has no conference venue. - Place each paper under the subsection that best matches its primary contribution.
- Prioritize representative, high-impact works to keep each list readable.