reverse_proxy: slow-start warm-up for newly added or recovered upstreams

[rajvarun77]

Issue Details

Problem. When an upstream joins the pool (dynamic upstreams via SRV/A lookups, a config reload that adds a host) or flips back to healthy after failing health checks, every selection policy gives it its full traffic share immediately. A cold upstream (JIT not warmed, empty caches, connection pools not yet established) serves worse latency for its first seconds, and with least_conn (and least_latency, #7965) the cold host looks idle, so it is the one that gets hammered first. Envoy addresses this with slow_start (a ramp of the effective weight with slow_start_window, aggression, min_weight_percent).

Proposal. A lb_slow_start <duration> option on reverse_proxy, off by default and zero-cost when unset.

• Host records a join timestamp, set when the host is created in the shared hosts pool and again on an unhealthy → healthy transition (both passive and active health checks).
• During the window, the host's effective weight ramps from a floor (10%, tunable) to 100%, linearly by default with an optional aggression exponent, mirroring Envoy's parameters so operators can reuse their intuition.
• Policies apply it in the shared selection path: random, random_choose, least_conn, least_latency scale their score by the ramp; round_robin / weighted_round_robin probabilistically skip a warming host; hash policies (ip_hash, client_ip_hash, uri_hash, query, header, cookie) are exempt because stickiness matters more than warm-up there (Envoy makes the same choice).
• A warming host is never skipped when it is the only available upstream, so the ramp can never produce a 503 that would not have happened anyway.

Scope. Roughly 250 lines plus tests, entirely inside modules/caddyhttp/reverseproxy (hosts.go, selectionpolicies.go, healthchecks.go, Caddyfile adapter). I have the same design implemented for brpc (apache/brpc, cross-policy warm-up ramp with duration / curve / min-weight flags) and would be happy to open the PR here if maintainers want the feature. Happy to adjust the config shape — e.g. nesting it under lb_policy instead of a sibling option — to whatever fits Caddy's conventions.

Assistance Disclosure

AI used

If AI was used, describe the extent to which it was used.

I wrote this proposal and the design myself. If implemented, an AI coding assistant would help draft the implementation and tests from that design, which I would review and verify; this will be disclosed in the PR body as required.

[faiyaz032]

nice idea, but i read through the reverse_proxy code against this design and a few things don't hold:

• scaling the score does nothing for least_conn / random_choose. leastRequests in selectionpolicies.go returns early on reqs == 0. a cold host has 0 active requests, and scaling 0 is still 0, so the warming host still wins every time. that's the exact case the issue opens with. envoy doesn't scale the score either, least_request uses max(weight / (active+1)^k).

• there's no shared selection path. each policy is its own module implementing Selector, with one call site in reverseproxy.go. only 3 helpers are shared and the main policies each inline their own loop. the namespace is public too, so out of tree policies get skipped.

• the join timestamp has nowhere correct to live. Host is keyed by dial address and shared process wide, so one site adding an upstream throttles it for another site already using it. Upstream is worse: allNew() in upstreams.go builds a fresh struct every request for dynamic upstreams, so SRV/A hosts look brand new each time and stay stuck at the 10% floor forever. that kills the main use case.

• probabilistic skipping can cause the 503 you're avoiding. if the whole pool is warming (fresh start, rolling deploy), round_robin returns nil, that hits tryAgain, and with the default lb_try_duration of 0 it's an instant 503. the "only available upstream" guard doesn't cover it.

• passive health has no healthy transition to hook. fails just decay on a timer in healthchecks.go, and MaxFails is per handler while fails is per Host. active checks also never run on dynamic upstreams. the one clean hook is the setHealthy(true) CAS.

• envoy nits: it only supports slow start in round robin, least request and client side WRR. hash policies aren't exempt there, the feature just doesn't exist for them, and ramping random goes past envoy rather than matching it. also aggression is an inverse exponent, TimeFactor^(1/Aggression).

instead of editing every policy, you may wrap the configured Selector in a decorator at provision time and drop warming hosts from the pool with probability 1-ramp. skips the first two problems and the fact that Upstream has no weight field, covers third party policies, and stays zero cost when unset.

either way i'd wait for a maintainer to confirm they want this in core and settle the config shape first, before writing code against it.

[mohammed90]

I agree with @faiyaz032. Can't this live as a plugin/external module? Why does it have to be in core?