Skip to content

[Bug]: connect retries ignore Retry-After on 429 (remaining ask from #1013) #1040

Closed

Description

@Dumbris

Context

#1013 asked for three things on the upstream reconnect path. #1014 landed exponential backoff in the supervisor's reconcile, and #1039 closes the OAuth-error hole in that gate plus real PendingAuth parking. The third ask is still open:

Honor Retry-After when the upstream answers 429 (the OAuth resource-detect path already parses hints; connect retries should too).

Why it can't be done in the backoff layer

The 429 response never reaches us as a response. mcp-go flattens every non-2xx into a string:

// client/transport/streamable_http.go
return nil, fmt.Errorf("request failed with status %d: %s", resp.StatusCode, body)

so managed.Client.Connect sees text, and the Retry-After header is gone before any state-manager code runs. internal/oauth/config.go can parse it only because the OAuth resource-detect path issues its own http.Do and holds the *http.Response.

Sketch

  1. A RetryAfterRecorder http.RoundTripper in internal/transport (next to LoggingTransport) that records Retry-After (RFC 7231 seconds and HTTP-date, like internal/oauth/config.go:379-440 already does) on a 429, keyed per upstream.
  2. Thread it into every client-construction branch in CreateHTTPClient / CreateSSEClient. Two branches already build an http.Client; the plain streamable-HTTP branches need WithHTTPBasicClient (preserving today's 180s timeout and SSE's deliberately absent Timeout); the two client.NewOAuth*Client branches build their own client inside mcp-go and need OAuthConfig.HTTPClient verified to cover transport requests, not just the OAuth ones.
  3. Surface the hint (e.g. ConnectionInfo.RetryAfter time.Time, set alongside the connect error) and treat it as a floor in ConnectionInfo.ShouldAutoReconnect — the one gate the supervisor consults.

Blast radius

Step 2 changes the transport for all upstream HTTP traffic, tool calls included — not just connect. That is why it was kept out of #1039: it needs its own testing pass (SSE long-lived streams and connection pooling in particular), and the storm risk is bounded meanwhile. After #1039 a 429-ing upstream is paced 1s → … → 5min, then one probe per 30min, so the worst case is ~48 requests/day rather than the ~2880 that started #1013. Honoring the header matters for the case where the vendor asks for a wait longer than that.

Acceptance

  • A 429 with Retry-After: 3600 during connect parks the server for an hour; the supervisor's reconcile plans no ActionConnect in that window.
  • Both Retry-After forms parse (delta-seconds and HTTP-date); a malformed value falls back to the existing ladder.
  • Coverage/limitations of the mcp-go-built OAuth client paths documented in the PR.

Refs #1013, #1014, #1039

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions