Skip to content

fix(anthropicsdk): a spent quota fails the turn instead of sleeping through it - #18

Closed
jack-work wants to merge 1 commit into
fix/wirelog-round-ledgerfrom
fix/anthropicsdk-retry-cap
Closed

jack-work wants to merge 1 commit into
fix/wirelog-round-ledgerfrom
fix/anthropicsdk-retry-cap

Conversation

@jack-work

Copy link
Copy Markdown
Owner

Stacked on #17 (base: fix/wirelog-round-ledger). The second commit is this PR.

The bug

anthropic-sdk-go computes its own retry delay:

if retryAfterDelay, ok := parseRetryAfterHeader(res); ok {
        return max(0, retryAfterDelay)   // no ceiling
}
maxDelay := 8 * time.Second              // guards only the FALLBACK path

When the server sends a Retry-After the SDK sleeps for exactly that long, inside RequestConfig.Execute, with no callback, no log line and no stream event. We set no MaxRetries either, so the ceiling was the SDK default.

That is right for a per-minute throttle. It is wrong for a usage window — a spent balance or a quota period — where Retry-After is "seconds until your window resets" and arrives in the thousands.

Observed on 94f0752b: two consecutive turns, 22 and 77 minutes of total silence, each ended by the user giving up rather than by the retry completing. One 429 at 17:09:36 was the entire content of a 77-minute span. figaro status said idle throughout. The account was simply out of credit for that model, and figaro responded by holding the conversation hostage until the balance renewed.

Two things were wrong

1. It waited at all. Sleeping cannot make a spent quota unspent. What it can do is hold the agent loop, so the human who could fork, downshift the model, or top up the account cannot get a prompt in edgewise.

Past maxRetryAfter (60s) the transport now sets x-should-retry: false — the SDK's own documented override, checked ahead of the status code in shouldRetry — so Execute returns the 429 immediately. The turn ends, and the aria is idle and promptable within the second. Retry-After is clamped as well, belt and braces. A short throttle is still retried: the cap is a ceiling, not a refusal.

2. It said nothing. A rate limit produced no log record of any kind. Now every refusal writes a slog.Warn with the status, the wait, the request id, the provider's reset hint and what we did about it — and the error carries it too:

anthropic: rate_limit_error (429): the provider asked us to wait 1h0m0s
(limit resets 2026-08-14T20:00:00Z); that is longer than figaro will hold a
turn open (1m0s), so the turn was ended rather than slept through - the aria
is free, send again after the reset, or fork to a different model
[req_011Ce3TTTY6NjCj66TBPgp3K]

Tests

  • TestQuotaRefusalEndsTheTurnAtOnce — the requirement, directly.
  • TestWorstCaseSilentWaitIsBounded — asserts maxRetries * maxRetryAfter, so the worst case is arithmetic this repo controls.
  • sdkWouldRetry mirrors requestconfig.shouldRetry, pinning the x-should-retry contract so we find out if a future SDK stops honoring it.

Ordering note

retryCapTransport sits above wirelog in the chain, so the ledger and the span still record what the server really sent. A diagnostic that recorded our own policy back to us would be worthless.

…hrough it

anthropic-sdk-go computes its own retry delay:

    if retryAfterDelay, ok := parseRetryAfterHeader(res); ok {
            return max(0, retryAfterDelay)   // no ceiling
    }

The 8-second cap in that function guards only the fallback exponential path.
When the server sends a Retry-After the SDK sleeps for exactly that long,
inside RequestConfig.Execute, with no callback, no log line and no stream
event. We set no MaxRetries either, so the ceiling was the SDK default.

That is right for a per-minute throttle. It is wrong for a usage WINDOW - a
spent balance or a quota period - where Retry-After is "seconds until your
window resets" and arrives in the thousands. Observed: one aria, two
consecutive turns, 22 and 77 minutes of total silence, each ended by the user
giving up rather than by the retry completing. `figaro status` said "idle"
throughout. The account was simply out of credit for that model, and figaro
responded by holding the conversation hostage until the balance renewed.

Two things were wrong and both are fixed here.

1. It waited at all. Sleeping cannot make a spent quota unspent. What it CAN
   do is hold the agent loop, so the human who could fork, downshift the
   model, or top up the account cannot get a prompt in edgewise. Past
   maxRetryAfter (60s) the transport now sets x-should-retry:false - the
   SDK's own documented override, checked ahead of the status code in
   shouldRetry - so Execute returns the 429 immediately. The turn ends, and
   the aria is idle and promptable within the second. Retry-After is clamped
   as well, belt and braces, so no other consumer of that response can be
   talked into an hour-long nap either. A short throttle is still retried:
   the cap is a ceiling, not a refusal.

2. It said nothing. A rate limit produced no log record of any kind - the SDK
   swallowed it and slept - and the error that eventually surfaced said
   "rate_limit_error (429)" with no hint of the scale. Now every refusal
   writes a slog.Warn with the status, the wait asked for, the request id,
   the provider's own reset hint and what we did about it; and the error
   carries the wait, the reset time and what to do:

     anthropic: rate_limit_error (429): the provider asked us to wait 1h0m0s
     (limit resets 2026-08-14T20:00:00Z); that is longer than figaro will
     hold a turn open (1m0s), so the turn was ended rather than slept through
     - the aria is free, send again after the reset, or fork to a different
     model [req_011Ce3TTTY6NjCj66TBPgp3K]

option.WithMaxRetries is also now set explicitly, so the worst-case wait is
arithmetic this repo controls rather than a property of a dependency's
release notes. There is a test asserting that product, one asserting a quota
refusal returns at once, and one pinning the x-should-retry contract so we
find out if a future SDK stops honoring it.

retryCapTransport sits ABOVE wirelog in the chain so the ledger and the span
still record what the server really sent; a diagnostic that recorded our own
policy back to us would be worthless.
@jack-work
jack-work force-pushed the fix/wirelog-round-ledger branch from 67397cd to c9a59f1 Compare August 15, 2026 00:34
@jack-work
jack-work force-pushed the fix/anthropicsdk-retry-cap branch from 7239196 to 1d804e2 Compare August 15, 2026 00:34
@jack-work

Copy link
Copy Markdown
Owner Author

Merged into main as 6d70b52 (merge fix/anthropicsdk-retry-cap: a spent quota fails the turn instead of sleeping through it, plus the round-trip ledger), shipped in v0.26.1.

Closing by hand rather than by branch deletion so the reason survives: GitHub did not auto-close it because main took the work through a merge commit this PR's head does not point at.

The behaviour that shipped: past maxRetryAfter the transport sets x-should-retry: false, the SDK returns the 429 immediately, the turn ends and the aria is promptable within the second — rather than sleeping out a usage window that no amount of retrying can shorten.

@jack-work jack-work closed this Aug 15, 2026
@jack-work
jack-work deleted the fix/anthropicsdk-retry-cap branch August 15, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant