Rate limits: count them and adapt how many calls run at once

- Providers report a refusal for rate or capacity reasons (HTTP 429,
  overloaded) as ProviderError::RateLimited, apart from other transient
  errors. It is retried with the same backoff.
- One Throttle per session: max_parallel is the ceiling; a rate limit
  halves the window (never below one); ten calls in a row that go through
  give a slot back. Plan tasks and sub-agent calls start new work only
  while the window has room; running work is never stopped.
- Counters provider_retries and rate_limit_retries; a guard event of kind
  rate_limited; a plan event window once a rate limit has narrowed it,
  which the task board shows ("slowed to 2 at a time after 3 rate limits").
- The score is shown as 93.0% ±0.8, the run-to-run standard error across
  the five runs, the measure the official leaderboard uses; the across-tasks
  interval stays, labelled as such.

Author: chandan25809-lyzr

BLUEPRINT.md

@@ -6,7 +6,7 @@ This document says **what** gitagent is built to do and **why** each part exists
 
 ## 0. Goal in one paragraph
 
-gitagent 3 is a single static Rust binary that runs **agents defined as git repositories**. An agent is a folder of plain files (identity, rules, memory, skills, tools, hooks, schedules, sub-agents), so its behaviour is versioned, reviewed and diffed like code. One loop runs every agent, the same way in a terminal, behind an SDK, in a fresh clone of a repository, on a timetable, or inside a benchmark container. That loop is built to finish real work: it shapes what the model reads, keeps the provider's cache warm, guards against loops and running out of time, and refuses "done" until the work has been verified. The proof is public: 93.03% on Terminal-Bench 2.1 with Claude Opus 5.5, above every published harness on that benchmark.
+gitagent 3 is a single static Rust binary that runs **agents defined as git repositories**. An agent is a folder of plain files (identity, rules, memory, skills, tools, hooks, schedules, sub-agents), so its behaviour is versioned, reviewed and diffed like code. One loop runs every agent, the same way in a terminal, behind an SDK, in a fresh clone of a repository, on a timetable, or inside a benchmark container. That loop is built to finish real work: it shapes what the model reads, keeps the provider's cache warm, guards against loops and running out of time, and refuses "done" until the work has been verified. The proof is public: 93.0% ±0.8 on Terminal-Bench 2.1 with Claude Opus 5.5, above every published harness on that benchmark.
 
 ---
 
@@ -43,7 +43,7 @@ gitagent 3 is a single static Rust binary that runs **agents defined as git repo
 
 gitagent is measured on Terminal-Bench 2.1, and Terminal-Bench runs through **Harbor**, an open framework for running agents against containerised tasks. Harbor matters for four reasons.
 
-1. **It is the same yardstick as the published results.** The leaderboards for Terminal-Bench 2.1 were produced with Harbor: the same task images, the same time limits, the same hidden tests, the same scoring. Running gitagent through it is what makes 93.03% comparable with the published 87.6% for Terminus 2 on the same model, rather than a number from a home-made runner.
+1. **It is the same yardstick as the published results.** The leaderboards for Terminal-Bench 2.1 were produced with Harbor: the same task images, the same time limits, the same hidden tests, the same scoring. Running gitagent through it is what makes 93.0% ±0.8 comparable with the published 87.6% for Terminus 2 on the same model, rather than a number from a home-made runner.
 2. **Grading is independent of the agent.** Harbor starts each task's container, hands the agent the instruction, stops it at the task's time limit, and when the agent has finished it runs the task's hidden tests itself and records the reward. gitagent never sees, reads or writes the tests. The result is a pass or a fail decided outside the agent, which is the only kind worth publishing.
 3. **It fits a static binary.** Harbor's "installed agent" model uploads the agent into the task container and runs it there. gitagent is one binary with no runtime, so installing it is a single file copy, and nothing in the container has to change. The shim that does this, `gitagent_harbor/agent.py`, is about two hundred lines.
 4. **It produces evidence others can check.** Harbor runs many trials in parallel, resumes trials that failed for reasons outside the agent (a revoked key, a full Docker network), keeps every trajectory and verifier log, and uploads a job publicly. The headline run's nine jobs, one per time limit, are public, so anyone can read every trial behind the number.
@@ -130,6 +130,7 @@ Steps 4 to 9 repeat until the stop check accepts, the agent replies in conversat
 | D24 | Everyday defaults keep the benchmark's guards, except the snapshot; the console has no time limit unless one is set | The loop detector, stop check and context clearing help in any repository; a person at the console stops a turn with Ctrl-C, while runs nobody watches keep a 30-minute ceiling; the snapshot costs up to 15 seconds per start in a project the user already knows |
 | D25 | Sub-agents are warned as their turns run low, may be given their own turn limit (up to 100), and come back marked partial when they run out | A review cut off mid-step wastes its whole cost; a warned one hands back what it found, and the lead re-runs only what is marked partial |
 | D26 | The `/tests` write guard applies to benchmark runs only | It protects graders' tests; in a user's project `/tests` is an ordinary path and a pattern such as `grep -v /tests/` must not be refused |
+| D27 | One throttle per session: `max_parallel` is a ceiling, a rate-limit refusal halves the window, ten clean calls in a row give a slot back | The right number of parallel calls depends on the account's rate limit, which gitagent cannot know; backing off beats retrying blindly, and the ceiling stays the user's choice |
 
 **Left out on purpose:** critics that rewrite answers; best-of-N on benchmarks; memory or learning during a scored run; automated rewriting of the harness; masking every turn; third-party multi-provider frameworks; a web interface; voice; remote sandboxes.
 
@@ -223,7 +224,7 @@ Dependencies are kept few: `tokio`, `reqwest` with rustls, `serde`, `serde_json`
 
 ## 9. Observability
 
-Counters are written to `counters.json` after every turn and exported with telemetry when it is on. They cover the loop (`turns`, `tool_calls`, `malformed_calls`, `repairs`, `refusals`, `fallback_turns`), the shell (`exec_timeouts`, `background_sessions`, `interrupts`, `shell_restarts`), shaping and context (`shaped_bytes_in`, `spill_reads`, `masks_applied`, `mask_checkpoints`, `compactions`, `cache_read_tokens`, `cache_write_tokens`), the guards (`stop_accepts`, `stop_rejections`, `loop_triggers`, `protected_writes_blocked`), hooks and compliance (`hook_blocks`, `hook_modifies`, `approvals_granted`, `approvals_denied`, `compliance_blocks`), memory, sub-agents and plans (`subagent_calls`, `plan_tasks`, `plan_failed_tasks`), workflows and MCP.
+Counters are written to `counters.json` after every turn and exported with telemetry when it is on. They cover the loop (`turns`, `tool_calls`, `malformed_calls`, `repairs`, `refusals`, `fallback_turns`, `provider_retries`, `rate_limit_retries`), the shell (`exec_timeouts`, `background_sessions`, `interrupts`, `shell_restarts`), shaping and context (`shaped_bytes_in`, `spill_reads`, `masks_applied`, `mask_checkpoints`, `compactions`, `cache_read_tokens`, `cache_write_tokens`), the guards (`stop_accepts`, `stop_rejections`, `loop_triggers`, `protected_writes_blocked`), hooks and compliance (`hook_blocks`, `hook_modifies`, `approvals_granted`, `approvals_denied`, `compliance_blocks`), memory, sub-agents and plans (`subagent_calls`, `plan_tasks`, `plan_failed_tasks`), workflows and MCP.
 
 A feature whose counters stay at zero across a measurement run is either broken or useless, and is investigated before it is kept.
 

README.md

@@ -10,7 +10,7 @@
 </p>
 
 <p align="center">
-  <img src="https://img.shields.io/badge/Terminal--Bench_2.1-93.03%25-e05a44?style=flat-square" alt="Terminal-Bench 2.1: 93.0%" />
+  <img src="https://img.shields.io/badge/Terminal--Bench_2.1-93.0%25_%C2%B10.8-e05a44?style=flat-square" alt="Terminal-Bench 2.1: 93.0% ±0.8" />
   <img src="https://img.shields.io/badge/version-3.0.0-2a1f18?style=flat-square" alt="version 3.0.0" />
   <img src="https://img.shields.io/badge/built_with-Rust-2a1f18?style=flat-square" alt="Rust" />
   <img src="https://img.shields.io/github/license/open-gitagent/gitagent?style=flat-square&color=2a1f18" alt="license" />
@@ -32,7 +32,7 @@
   <img src="./docs/media/console.gif" alt="A gitagent session in the terminal" width="820" />
 </p>
 
-**Terminal-Bench 2.1: 93.03%** with Claude Opus 5.5, all 89 tasks, five runs each, code frozen at the published commit. The published harnesses on the same benchmark: Terminus 2 with the same model 87.6%, Codex CLI with GPT-6 Astra 87.4%, and the highest published entry, Artificial Analysis's own harness with Claude Fable 5.1 at maximum effort, 91.4%. The numbers are under [Results](#results); the reason is under [How gitagent works](#how-gitagent-works).
+**Terminal-Bench 2.1: 93.0% ±0.8** with Claude Opus 5.5, all 89 tasks, five runs each, code frozen at the published commit; the ± is the run-to-run variation, as on the official leaderboard. The published harnesses on the same benchmark: Terminus 2 with the same model 87.6%, Codex CLI with GPT-6 Astra 87.4%, and the highest published entry, Artificial Analysis's own harness with Claude Fable 5.1 at maximum effort, 91.4%. The numbers are under [Results](#results); the reason is under [How gitagent works](#how-gitagent-works).
 
 ## Why gitagent
 
@@ -49,7 +49,7 @@ Most agent frameworks treat the agent as code scattered across your application.
 
 Fork an agent. Branch a personality. `git log` its memory. Diff its rules. Review a change to its behaviour like a code change. Every result traces back to a commit.
 
-**gitagent 3** is the Rust rewrite: one static binary with no runtime to install, the harness that scored 93.03% on Terminal-Bench 2.1, a console in your terminal, parallel sub-agents with a live task board, and SDKs for Python, TypeScript and any other language over a simple line protocol. Agents, hooks, schedules and SDK code written for the TypeScript gitagent (1.x and 2.x) run unchanged, and agent folders in the OpenGAP format load too. See [Coming from the TypeScript gitagent](#coming-from-the-typescript-gitagent).
+**gitagent 3** is the Rust rewrite: one static binary with no runtime to install, the harness that scored 93.0% on Terminal-Bench 2.1, a console in your terminal, parallel sub-agents with a live task board, and SDKs for Python, TypeScript and any other language over a simple line protocol. Agents, hooks, schedules and SDK code written for the TypeScript gitagent (1.x and 2.x) run unchanged, and agent folders in the OpenGAP format load too. See [Coming from the TypeScript gitagent](#coming-from-the-typescript-gitagent).
 
 ## Install
 
@@ -161,6 +161,14 @@ the next task starts as soon as one finishes. Use it for work that splits
 into similar pieces: many files to read, many items to check. It costs
 tokens in proportion, so give it scoped tasks.
 
+`max_parallel` is a ceiling. When the model provider refuses a call for
+rate limits, the call is retried after a backoff and the number of tasks
+running at once is halved; after ten calls in a row go through, one slot
+comes back, up to the ceiling. The task board says when this happens
+("slowed to 2 at a time after 3 rate limits"), and every run counts its
+`rate_limit_retries`. So set `max_parallel` to what you want when the
+account allows it, and let gitagent slow down when it does not.
+
 Each task gets 30 model calls unless the agent gives it its own
 `max_turns` (up to 100), which it can size to the task: more for a large
 review, fewer for a quick look-up. A task is warned when 80% of its calls
@@ -214,7 +222,7 @@ The headline run's logs are public: every trial of it, with its trajectory and v
 
 | Harness | Model | Terminal-Bench 2.1 | Runs per task | Source |
 |---|---|---|---|---|
-| **gitagent 3** | Claude Opus 5.5 | **93.03%** (414/445; 75 of 89 tasks passed every run) | 5 | [job logs on the Harbor hub](#the-same-run-by-time-limit) |
+| **gitagent 3** | Claude Opus 5.5 | **93.0% ±0.8** (414/445; 75 of 89 tasks passed every run) | 5 | [job logs on the Harbor hub](#the-same-run-by-time-limit) |
 | Terminus 2 | Claude Opus 5.5 | 87.6% | 1 | [Vals.ai](https://www.vals.ai/benchmarks/terminal-bench-2-1) |
 | Codex CLI | GPT-6 Astra | 87.4% ±0.9 | — | [Snorkel, the official leaderboard](https://snorkel.ai/leaderboard/terminal-bench-2-1/) |
 | Artificial Analysis's own harness | Claude Fable 5.1, max effort | 91.4% | — | [Artificial Analysis](https://artificialanalysis.ai/evaluations/terminalbench-2-1) |
@@ -234,9 +242,9 @@ Pass@k is the chance that at least one of k runs passes:
 | 60 min | 65 | 95.4% | 97.7% | 100.0% | [job](https://hub.harborframework.com/jobs/e1ea8dbd-7a55-4fa3-95de-b9338a3c1223) |
 | 2 h | 5 | 100.0% | 100.0% | 100.0% | [job](https://hub.harborframework.com/jobs/127445ee-63ec-44c7-8206-189eb22cd912) |
 | 3 h 20 min | 5 | 100.0% | 100.0% | 100.0% | [job](https://hub.harborframework.com/jobs/1d8e0bdb-7ae5-4ec7-8e54-329007590e9d) |
-| **all** | **445** | **93.03%** | | | |
+| **all** | **445** | **93.0% ±0.8** | | | |
 
-Two of the 89 tasks start a virtual machine inside the task container, which this host cannot do, and no harness has passed them here; without those two the mean is 95.2%. The 95% interval for the headline run is 88.8–96.9%. It cost $112.85, about $0.25 per run, with 92% of the input tokens served from the prompt cache.
+Two of the 89 tasks start a virtual machine inside the task container, which this host cannot do, and no harness has passed them here; without those two the mean is 95.2%. The ±0.8 is the standard error across the five runs (93.3, 95.5, 92.1, 92.1 and 92.1%), the measure the official leaderboard shows; across tasks, the 95% interval is 88.8–96.9%, which answers the wider question of how the score would hold on a different set of tasks. It cost $112.85, about $0.25 per run, with 92% of the input tokens served from the prompt cache.
 
 ### Why it scores higher than the other harnesses
 

crates/cli/src/console.rs

@@ -463,6 +463,9 @@ struct BoardState {
     phase_total: usize,
     /// Columns to draw in; longer lines are cut so each takes one row.
     width: usize,
+    /// (running at once, ceiling, rate limits so far), once a rate limit
+    /// has narrowed the window.
+    window: Option<(usize, usize, u64)>,
     tasks: Vec<TaskRow>,
     reconcile: Option<(bool, Option<bool>, String, std::time::Instant)>,
     drawn: usize,
@@ -543,6 +546,13 @@ impl Board {
                     t.took = Some(now.duration_since(t.started));
                 }
             }
+            "window" => {
+                b.window = Some((
+                    e["at_once"].as_u64().unwrap_or(0) as usize,
+                    e["ceiling"].as_u64().unwrap_or(0) as usize,
+                    e["rate_limited"].as_u64().unwrap_or(0),
+                ));
+            }
             "reconcile_start" => b.reconcile = Some((true, None, String::new(), now)),
             "reconcile_end" => {
                 b.reconcile = Some((false, Some(e["ok"] == true), s("result"), now));
@@ -624,12 +634,23 @@ impl Board {
         let finished = b.tasks.len() - running;
         let total = b.phase_total.max(b.tasks.len());
         let elapsed = b.started.map(|s| s.elapsed().as_secs()).unwrap_or(0);
+        let slowed = match b.window {
+            Some((w, c, hits)) if w < c => format!(
+                " · slowed to {w} at a time after {hits} rate limit{}",
+                if hits == 1 { "" } else { "s" }
+            ),
+            Some((_, c, hits)) => format!(
+                " · {hits} rate limit{}, back to {c} at a time",
+                if hits == 1 { "" } else { "s" }
+            ),
+            None => String::new(),
+        };
         out.push(format!(
             "{} {} {}",
             st.orange("⇶"),
             st.bold(&format!("phase {}/{} {}", b.phase, b.of, b.phase_name)),
             st.dim(&format!(
-                "· {running} running · {finished} of {total} done · {elapsed}s"
+                "· {running} running · {finished} of {total} done · {elapsed}s{slowed}"
             ))
         ));
         const SHOW: usize = 14;

crates/cli/src/ipc.rs

@@ -1014,6 +1014,7 @@ async fn session(cli: Cli, out: Out) -> i32 {
             kill: Some(ctl.kill.clone()),
             steer: Some(ctl.steering.clone()),
             external_tools: external.clone(),
+            throttle: None,
         };
         let run = gitagent_core::run(&s.spec, s.provider.as_ref(), opts);
         tokio::pin!(run);
@@ -1142,6 +1143,7 @@ async fn session(cli: Cli, out: Out) -> i32 {
             kill: None,
             steer: None,
             external_tools: external.clone(),
+            throttle: None,
         };
         match gitagent_core::teardown(&s.spec, s.provider.as_ref(), opts).await {
             Some(Ok(done)) => out.system(

crates/cli/src/main.rs

@@ -542,6 +542,7 @@ async fn single(cli: Cli, instruction: String) -> ExitCode {
                 kill: None,
                 steer: None,
                 external_tools: None,
+                throttle: None,
             },
         )
         .await?;
@@ -620,6 +621,7 @@ async fn run_workflow(cli: Cli, name: String) -> ExitCode {
             kill: None,
             steer: None,
             external_tools: None,
+            throttle: None,
         };
         let report =
             gitagent_core::run_workflow(&s.spec, s.provider.as_ref(), &name, &inputs, opts).await?;
@@ -666,6 +668,7 @@ pub(crate) async fn teardown(
         kill: None,
         steer: None,
         external_tools: None,
+        throttle: None,
     };
     match gitagent_core::teardown(&s.spec, s.provider.as_ref(), opts).await {
         Some(Ok(mut done)) => {

crates/cli/src/repl.rs

@@ -108,6 +108,7 @@ pub(crate) async fn run(s: Setup, out_dir: &Path) -> Result<()> {
                 kill: None,
                 steer: None,
                 external_tools: None,
+                throttle: None,
             };
             match gitagent_core::run_workflow(&s.spec, s.provider.as_ref(), &name, &inputs, opts)
                 .await
@@ -160,6 +161,7 @@ pub(crate) async fn run(s: Setup, out_dir: &Path) -> Result<()> {
             kill: None,
             steer: None,
             external_tools: None,
+            throttle: None,
         };
         let spinner = console::Spinner::start(st, "thinking");
         tokio::select! {
@@ -203,6 +205,7 @@ pub(crate) async fn run(s: Setup, out_dir: &Path) -> Result<()> {
             kill: None,
             steer: None,
             external_tools: None,
+            throttle: None,
         };
         match gitagent_core::teardown(&s.spec, s.provider.as_ref(), opts).await {
             Some(Ok(done)) => eprintln!(

crates/cli/src/repo_mode.rs

@@ -99,6 +99,7 @@ async fn run_inner(cli: &mut Cli, url: String, task: String) -> Result<ExitCode>
             kill: None,
             steer: None,
             external_tools: None,
+            throttle: None,
         },
     )
     .await?;

crates/cli/src/schedule_cmd.rs

@@ -343,6 +343,7 @@ impl Runner {
             kill: None,
             steer: None,
             external_tools: None,
+            throttle: None,
         };
         let (success, result) =
             match gitagent_core::run(&self.setup.spec, self.setup.provider.as_ref(), opts).await {