Repository navigation
Commit 5774ba9
authored
feat(upstream): honor Retry-After on 429 via transport RoundTripper (#1042)
* feat(upstream): honor Retry-After on 429 via transport RoundTripper
mcp-go flattens every non-2xx upstream response into an error string, so a
429's Retry-After header was gone before any connection-state code ran and the
reconnect ladder paced a rate-limited upstream on its own schedule.
Capture the hint where the *http.Response still exists: a RetryAfterTransport
RoundTripper installed under every HTTP/SSE client we hand to mcp-go, feeding a
per-server RetryAfterRecorder. The managed client stamps the resulting deadline
onto the state machine, and ConnectionInfo.ShouldAutoReconnect / ShouldRetry
treat it as a floor under the existing backoff — effective delay is
max(backoff, Retry-After), capped at one hour.
- Both RFC 7231 forms parse (delta-seconds and HTTP-date); a malformed, absent,
or already-elapsed value falls back to the existing ladder untouched.
- 503 counts only when it carries a parseable hint; a bare 503 is an outage the
ladder already paces.
- The wrapper composes with the existing tracing transport and sits below
mcp-go's OAuth handling, so the three authorize paths are unchanged. The two
OAuth client constructors now receive our basic client explicitly, since
OAuthConfig.HTTPClient covers only metadata/DCR/token requests.
- The window is cleared on a successful connect and on Reset, so a stale hint
cannot hold back a later, unrelated reconnect.
Closes #1040. Refs #1013, #1039.
* fix(upstream): scope the Retry-After window to its attempt and gate the OAuth ladder
Cross-model review of #1042 found four real holes:
- The hint only reached the state machine after a failed Connect, so a 429
answered to the health-check ping was recorded and then ignored.
syncRetryAfterFromTransport is now called from every path that turns an
upstream failure into Error state.
- The recorder was cleared only on success, so a window from an earlier attempt
could be re-attributed to a later, unrelated failure — re-parking a server the
upstream had not rate-limited again (notably right after a manual reconnect
reset the state machine). Each attempt now starts with a clean slate.
- ShouldRetryOAuth (the monitoring loop's reconnect path) ignored the window,
and its rungs can be shorter than what the upstream asked for.
- Converting delta-seconds to a Duration before capping overflowed int64
nanoseconds past ~292 years, turning a huge header into a tiny or negative
delay instead of the cap. The bound check now precedes the multiplication.
Tests: OAuth streamable-HTTP and SSE clients now have wire-level coverage (a
pre-seeded token store makes the request actually go out, so the 429 is
observable) — verified to fail when either constructor loses the basic-client
option; plus per-attempt window scoping at the core level, the OAuth-ladder
gate, and the two overflow-shaped Retry-After inputs.
Refs #1040.
* fix(upstream): make the 429 sync reachable from the request paths and swap recorders per attempt
Round 2 of the cross-model review found the round-1 health-path fix was dead
code and the per-attempt clear was too blunt:
- performHealthCheck only reached the sync after isConnectionError, which
matches transport failures ("connection refused", "deadline exceeded") and
not HTTP statuses — so a ping answered 429 never handed its window to the
state machine. The sync now runs on any failed ping, before classification;
it does not change the health verdict, it only stops the automatic reconnect
paths from returning early. The same sync now also runs on a failed
tools/call, the other surface that can be rate-limited with no connect
attempt in sight.
- Clearing one shared recorder at the start of an attempt let a request still
in flight on a superseded transport write into the new attempt's slate (and
discarded a hint recorded moments before the clear). The core client now
holds a generation pointer and swaps in a fresh recorder per attempt, so each
transport keeps the recorder it was built with.
Tests: the health-path test is verified to fail when the sync moves back behind
isConnectionError, plus tools/call coverage.
Refs #1040.1 parent ca00071 commit 5774ba9
14 files changed
Lines changed: 1311 additions & 95 deletions
File tree
- internal
- runtime/supervisor
- transport
- upstream
- core
- managed
- types
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
137 | 137 | | |
138 | 138 | | |
139 | 139 | | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
140 | 170 | | |
141 | 171 | | |
142 | 172 | | |
| |||
202 | 232 | | |
203 | 233 | | |
204 | 234 | | |
205 | | - | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
206 | 249 | | |
207 | 250 | | |
208 | 251 | | |
| |||
219 | 262 | | |
220 | 263 | | |
221 | 264 | | |
222 | | - | |
223 | | - | |
224 | | - | |
225 | | - | |
226 | | - | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
238 | | - | |
239 | | - | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
| 265 | + | |
253 | 266 | | |
254 | 267 | | |
255 | | - | |
256 | | - | |
257 | | - | |
258 | | - | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
259 | 289 | | |
260 | | - | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
261 | 296 | | |
262 | 297 | | |
263 | | - | |
264 | | - | |
| 298 | + | |
265 | 299 | | |
266 | 300 | | |
267 | 301 | | |
| |||
313 | 347 | | |
314 | 348 | | |
315 | 349 | | |
316 | | - | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
317 | 362 | | |
318 | 363 | | |
319 | 364 | | |
| |||
330 | 375 | | |
331 | 376 | | |
332 | 377 | | |
333 | | - | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | | - | |
338 | | - | |
339 | | - | |
340 | | - | |
341 | | - | |
342 | | - | |
343 | | - | |
344 | | - | |
345 | | - | |
346 | | - | |
347 | | - | |
348 | | - | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
356 | | - | |
357 | | - | |
358 | | - | |
359 | | - | |
360 | | - | |
361 | | - | |
362 | | - | |
363 | | - | |
364 | | - | |
365 | | - | |
366 | | - | |
367 | | - | |
368 | | - | |
369 | | - | |
370 | | - | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | 378 | | |
376 | 379 | | |
377 | 380 | | |
| |||
385 | 388 | | |
386 | 389 | | |
387 | 390 | | |
388 | | - | |
389 | 391 | | |
390 | 392 | | |
391 | | - | |
392 | 393 | | |
393 | 394 | | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
394 | 398 | | |
395 | | - | |
| 399 | + | |
396 | 400 | | |
397 | 401 | | |
398 | 402 | | |
399 | 403 | | |
400 | 404 | | |
401 | 405 | | |
| 406 | + | |
402 | 407 | | |
403 | 408 | | |
404 | 409 | | |
| |||
411 | 416 | | |
412 | 417 | | |
413 | 418 | | |
414 | | - | |
415 | | - | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
416 | 426 | | |
417 | 427 | | |
418 | 428 | | |
| |||
0 commit comments