Repository navigation
Description
Summary
After a laptop sleeps overnight, remote HTTP upstreams (e.g. https://mcp.atlassian.com/v2/mcp, https://api.githubcopilot.com/mcp/) end up in Error state and stay there after the machine wakes and the network is back. The retry backoff has grown to ~30 min, nothing triggers an earlier retry, and the UI shows a red, misleading error. A manual Restart connects instantly (OAuth token refresh works fine), so the server was reachable the whole time — mcpproxy just didn't try.
Environment
- MCPProxy v0.73.0 (personal), darwin/arm64, tray app (
mcpproxy serverunning for ~7 days) - macOS 26 (Darwin 25.5), on AC power, normal sleep with periodic DarkWake / Maintenance Sleep
- Upstreams affected: 2 ×
protocol: http(one OAuth/DCR, one staticAuthorizationheader). Stdio/docker upstreams were unaffected.
Timeline (from server-<name>.log, main.log, pmset -g log; local time UTC+2)
| Time | Event |
|---|---|
| Day 1 09:18 | Upstream connects OK (Successfully connected and initialized) |
| Day 1 ~17:00 → Day 2 04:00 | Machine sleeping / dark-waking. Errors: dial tcp: lookup mcp.atlassian.com: no such host, ... i/o timeout, Post ...: context deadline exceeded |
| Day 2 04:00 → 07:23 | Retries continue, but spaced out: 04:58, 05:04, 05:27, 05:33, 06:04, 06:51, 07:22 (backoff ≈ 30 min). retry_count reaches 23 (38 for the second upstream) |
| Day 2 07:23 | Last automatic attempt — fails (machine still in DarkWake) |
| Day 2 07:28 | pmset: DarkWake to FullWake ... due to UserActivity — user is back, network is up |
| Day 2 07:34 | curl -X POST to both MCP URLs from the same host: HTTP 401 in ~0.3 s (i.e. reachable) |
| Day 2 07:35 | Manual Restart → token refreshed from persistent store → MCP initialization completed successfully in < 1 s |
Without the manual restart, the next automatic attempt would have been ~07:53, i.e. ~25 min of the server being unusable after wake while the UI showed it as broken.
Problems
- No reconnect trigger on wake / network change. Recovery depends solely on the backoff timer, which is at its maximum exactly when the user returns.
- Retries during sleep inflate the backoff. DarkWake windows have no/flaky network, so every attempt fails and pushes the backoff higher.
- Transient network errors are presented as hard failures. The tray shows red
failed to connect: MCP initialize failed during no-auth strategy: server did not respond to MCP initialize within 30s and produced no stderr output (check that the command starts an MCP server and not a help banner). For an HTTP upstream the "stderr"/"help banner" hint doesn't apply, and the real cause (DNS failure / timeout) is hidden in the log. - Tool calls don't trigger a reconnect by default.
reconnect_on_use(Add optional on-demand reconnect when a tool is invoked for a disconnected upstream #354) exists but is opt-in per server, so a typical remote HTTP upstream fails fast instead of attempting a just-in-time connection.
Related: #1013 / #1014 / #1039 added the backoff on purpose (to stop hammering dead remotes) — this issue is not asking to remove it, only to reset it when the local network situation demonstrably changes.
Proposed behavior
In rough order of impact:
- Reset backoff and retry immediately on system wake and on network-path change (macOS:
NSWorkspaceDidWakeNotification/NWPathMonitor; Linux: NetworkManager/systemd-logind signals; Windows: power/network events). This alone would have recovered both upstreams within seconds of 07:28. It's compatible with [Bug]: Upstream reconnect has no backoff — supervisor re-dials failed servers every 30s forever, hammering dead remotes (~9k req/day) #1013: one attempt per wake/network event, then the normal backoff resumes; failures caused by local network unavailability arguably shouldn't advance the backoff at all. - Default
reconnect_on_useto on for HTTP/SSE upstreams (or globally), and make it also apply toretrieve_tools, so a client using an Error-state upstream gets one bounded connection attempt before an error. - Classify errors: DNS failure, connect/dial timeout, connection refused,
context deadline exceededon HTTP → transient → show as "Reconnecting…" (neutral/amber), not red Error. Keep red for auth failures, 4xx config problems, or persistent failure while other upstreams/hosts are reachable. - Pause the retry loop while offline (no default route / path unsatisfied) instead of burning attempts and growing backoff.
- Lower / configurable max backoff for HTTP upstreams (e.g.
max_reconnect_backoff, default ~5 min). Stdio servers may warrant a different policy. - Transport-specific error text: for HTTP, e.g.
mcp.atlassian.com unreachable (DNS lookup timed out); will retry, without the stdio-only hint.
Workaround
Manual Restart from tray / upstream_servers restart, or an external wake hook (sleepwatcher / launchd) that restarts Error-state upstreams.
Log excerpts
ERROR core/connection_http.go:227 MCP initialize JSON-RPC call failed {"server": "<atlassian>", "error": "transport error: failed to send request: failed to send request: Post \"https://mcp.atlassian.com/v2/mcp?tools=all\": dial tcp: lookup mcp.atlassian.com: i/o timeout"}
ERROR core/connection_http.go:227 MCP initialize JSON-RPC call failed {"server": "<atlassian>", "error": "transport error: ... Post \"https://mcp.atlassian.com/v2/mcp?tools=all\": context deadline exceeded"}
ERROR core/connection.go:345 Connection failed {"server": "<atlassian>", "transport": "http", "error": "MCP initialize failed during no-auth strategy: server did not respond to MCP initialize within 30s and produced no stderr output (check that the command starts an MCP server and not a help banner)"}
ERROR upstream/manager_generation.go:392 Connection error {"component": "managed_client", "server": "<atlassian>", "error": "failed to connect: ...", "retry_count": 23}
# after manual restart at 07:35
INFO oauth ✅ Found existing token in persistent storage {"expired": true}
INFO core/connection_oauth.go:476 MCP initialization completed successfully {"server_name": "atlassian-mcp-server", "protocol_version": "2025-11-25"}
INFO managed/client.go:623 Successfully connected and initialized
Error counts in the affected upstream's log over the period: ~324 transport errors (deadline exceeded / DNS i/o timeout / no such host), 0 non-network errors after the initial OAuth setup.
Activity
fix(upstream): budget network-recovery reconnects, refresh expired to…
fix(upstream): restore the recovery budget on any connect, space it o…
fix(upstream): a backwards clock step does not defer network recovery
fix(upstream): recover a connected server whose token refresh stalled…
fix(upstream): keep following an outage the health check noticed first
test(upstream): exercise the health-check-noticed outage for real
fix(upstream): recognise Windows socket errors as network failures
fix(upstream): classify socket errors by type in the health check
fix(server): republish /mcp/s/<server> after a transient tool-listing…
fix(server): bound the direct rebuild retry and republish on every to…
fix(server): bound the direct rebuild retry and republish on every to…
fix(upstream): reconnect remote servers as soon as the network returns (