A Server Action that revalidatePaths its own route + returns a value can leave the appended RSC segment never draining (useActionState hangs) — strong production evidence, help finding the co-factor
#95916
Replies: 4 comments
|
Your interpretation of the response shape matches the 16.2.10 source. The action and refreshed route are not two independent HTTP responses that happen to be concatenated; they become two fields in the same RSC payload/stream. The relevant path is:
That explains why seeing the action-return row does not imply that Given your 379 clean-room passes, I would not change the harness again yet. I would run the real production build, route, fixture and action through a layer-removal matrix. It gives a stronger answer than adding more synthetic flow-control cases:
For B, preserve the production hostname/SNI/cookies rather than changing the URL. For example, use I would add a temporary nginx access-log format that records at least: log_format flight '$request_id $status rt=$request_time '
'urt=$upstream_response_time uaddr=$upstream_addr '
'ubs=$upstream_bytes_received bs=$body_bytes_sent '
'conn=$connection creq=$connection_requests';After aborting a stalled request, the useful discriminator is:
For one diagnostic route/location, test the response path without nginx's three moving parts: proxy_http_version 1.1;
proxy_buffering off;
proxy_cache off;
gzip off;
I would test Cloudflare second, not first. First establish whether nginx received the complete upstream response. If nginx received all bytes, repeat A with a cache-bypass rule and response transformations/compression disabled for the affected route. Also confirm the actual Cloudflare-to-origin protocol; browser HTTP/2 does not prove the origin leg is HTTP/2. Finally, log the PM2 worker PID on every action start, action return and render return. If a stalled response always belongs to one PID, drain that worker and compare its open handles/socket state. If it moves randomly and disappears at So my first experiment would be C versus B on the exact production app, with the nginx upstream-byte timings enabled. It is the shortest test that distinguishes framework/PM2 from proxy buffering, and it does not require modifying Next internals. Your If this isolation matrix identifies the failing layer, please mark this as the accepted answer; the resulting nginx/PM2/Cloudflare discriminator will also make a much stronger upstream bug report. |
|
Why this happens (RSC Flight Stream Deadlock) Emits the action's return tuple (e.g., 1:{"ok":true}) immediately. Re-renders the current route, serializing the newly rendered RSC segment directly into the same streaming response via React Server Components (Flight) stream. The post-render flush hangs when backpressure from downstream proxies (Cloudflare/Nginx) causes Node's underlying WritableStream to fill its high-water mark buffer while React's server renderer is waiting for a stream drain event that never fires or gets swallowed. Which co-factor to instrument first Nginx proxy_max_temp_file_size: When an RSC response exceeds Nginx's in-memory buffers (proxy_buffers), Nginx attempts to buffer the remaining stream to a temporary file on disk. If proxy_buffering is enabled and proxy_max_temp_file_size is reached or throttled by Cloudflare's HTTP/2 window updates, Node's stream socket pauses emitting data. React 19 Stream Drain Listener: Under high backpressure, Node's res.write() returns false (signaling buffer full). React 19 waits for the 'drain' event to continue enqueuing chunks. If Cloudflare/Nginx closes or halts the HTTP window update without closing the TCP connection, the 'drain' event never fires on Node's socket, leaving the HTTP response permanently pending. Recommended Debugging / Next Steps Bypass Cloudflare temporarily: Test the endpoint directly against your Nginx / PM2 setup. If it stops stalling, the issue is directly tied to Cloudflare's edge-to-origin HTTP/2 multiplexing window size limit (initial_window_size). Disable Nginx Disk Buffering on RSC endpoints: Add proxy_max_temp_file_size 0; to your Nginx configuration for Next.js Flight responses (Header match Content-Type: text/x-component) to see if preventing disk spill resolves the flush deadlock. Instrument the Response Stream: Add a lightweight wrapper around res.write in custom middleware/server to log when res.write() returns false and whether the subsequent drain event is ever emitted during the stall. Note: Your client-side workaround (router.refresh() post-action or redirect()) remains the safest production strategy for large RSC segments until the upstream Flight stream backpressure handling is patched in next/server. |
|
Thanks — the backpressure explanation is plausible, but I think it is too early to present it as the confirmed root cause. The current evidence confirms that:
However, it does not yet prove that:
Those are useful hypotheses to test, but they should remain hypotheses until the failing layer is isolated. Important nginx distinctionThese two directives test different things:
This disables nginx response buffering for the location.
This only prevents buffered responses from being written to temporary files. It does not disable the in-memory buffering controlled by directives such as:
Therefore, the following tests should be run independently:
Test | proxy_buffering | proxy_max_temp_file_size | What it isolates
-- | -- | -- | --
Baseline | on | Current production value | Existing production behavior
Memory-only buffering | on | 0 | Temporary-file spill
No response buffering | off | Irrelevant | nginx buffering and downstream backpressure as a whole
Recommended isolation matrixI would test the exact production build, route, fixture and action through this matrix:
For test B, preserve the production hostname, cookies and TLS/SNI behavior. For example:
Changing the hostname may also change middleware behavior, action-origin validation, cache keys or application routing. Cloudflare testingCloudflare HTTP/2 flow control is worth testing, but only after determining whether nginx received the complete upstream response. The browser using HTTP/2 does not prove that the Cloudflare-to-origin connection also uses HTTP/2. The actual origin protocol and zone configuration should be verified directly. If nginx receives the full upstream response, compare:
If bypassing Cloudflare fixes the issue while the exact same nginx and PM2 path remains stable, Cloudflare becomes the first demonstrated failing layer. About instrumenting
|
|
Additional evidence from a different application: the redirect variant can stall too, without nginx, Cloudflare, or PM2 in the path. Environment:
Action shape:
One of two comparable CI executions stalled. The same targeted flow and its actual predecessors completed in all 62 controlled local attempts. I am reporting those rates separately rather than calling this flaky: CI 1/2 stalled; local 0/62 stalled. Sanitized Playwright trace entry for the action POST at artifact closure: The 303 headers arrived and Playwright observed the response, but five seconds later:
The mutation definitely completed before the stall: the x-action-redirect message contains a calculated value read from the upstream mutation response, so the action could not have constructed that header before consuming the successful upstream result. This seems to narrow the stuck boundary to the Flight body used to commit the app-relative redirect. In the Next action handler this path internally fetches the redirect target as RSC and returns its response body as FlightRenderResult. The trace had bodySize=-1 and transferSize=-1 for that POST when the test artifact closed. We also checked two plausible application-side races:
This extends the original report in two useful ways: redirect() does not always supersede the problematic Flight path, and the failure reproduced on a direct standalone localhost path without the proxy/cluster topology from the original report. Our product mitigation is a truthful 15-second uncertain-result state plus a verification link, backed by an idempotent mutation so retry cannot duplicate the operation. That contains user risk but does not fix the stream. The most useful next instrumentation point appears to be between the internal redirect-target RSC fetch / FlightRenderResult body and the client router commit. Is there an existing debug hook or trace point in 16.2.x that can distinguish an origin body that never closes from a body that closes but is never committed by the client reducer? |
Uh oh!
There was an error while loading. Please reload this page.
Next.js 16.2.10 · React 19.2.7 · App Router
We have a reproducible-in-production, intermittent deadlock and a lot of evidence isolating it, but a clean-room minimal app does not yet trigger it — so we're posting this as a discussion to ask where in the App Router / Flight pipeline the failure could live, and which environmental co-factor to instrument next. Full evidence and a minimal harness below; happy to share the harness repo and raw captures.
What we observe
A Server Action that calls
revalidatePath(currentRoute)and then returns a value (rather thanredirect()-ing) makes Next append the freshly re-rendered RSC segment of the current route to that action's own POST flight response, after the1:{…}action-return row. When that appended segment is large (≳800 KB), it intermittently never drains/closes: the HTTP response stays open, so the client'suseActionStatepromise never settles and the form is stuck pending forever. The page render always completes — only the post-render flush of the appended segment hangs.Production stack where we see it:
output: 'standalone'server under PM2 cluster mode, behind nginx (proxy_buffering on,proxy_cache), behind the Cloudflare proxy (HTTP/2 to the browser). Route isforce-dynamic; action bound viauseActionState.Production evidence
1. The discriminator — HAR body size + whether the response ever finished
Same route, same fixture, same worker pool; three identical fires of the action:
contentSize/bodySize200 text/x-component200 text/x-component200 text/x-component200 text/x-component…1:{"ok":true})revalidatePath)2 of 3 identical fires stalled; 1 streamed the full 872 KB. The only variable is timing ⇒ a race, not a deterministic path. The control fire (same action shape minus
revalidatePath) closes in ~1 ms / 99 bytes — isolatingrevalidatePath-into-returnas the trigger.2. The render always completes (rules out the app's own await graph)
Server-side render spans for each fire (the re-render triggered by
revalidatePath), same worker, immediately after the action-return:render-returnBoth stalled fires logged
render-return— the page component ran to completion (sub-second) before the stream hung. The hang is in the framework's post-render flush of the appended segment, not in the render or our data layer.3. The DB is idle during the hang
pg_stat_activitysampled ~15 s into each stalled fire's pending window: 0 active (non-idle) backends — every pooled connection idle. No query is in flight while the stream hangs, so the wait isn't in our data access.4. Size sensitivity
The same
revalidatePath(currentRoute) + return {ok:true}shape on a ~647 KB route completed 2/2 (never stalled); the ~872 KB route lost the flush race ~2/3 of the time. The failing cell is(large revalidated RSC segment) × (appended to a Server Action's own POST response), and it is size/latency-sensitive.Minimal shape
Fire
revalidateActionrepeatedly (real click, browserfetch, or a raw POST to the page path with theNext-Actionheader) and watch whether the ~1.3 MB+ response drains and the form settles.Clean-room reproduction status — the shape alone did not reproduce it
On a bare Next 16.2.10 app we could not reproduce the deadlock across 379 fires, spanning every transport/runtime/proxy combination we could think of:
next startfetch+ realuseActionStateclicknext startEvery configuration — including write-backpressure (deliberately slow readers), concurrency, a buffering reverse proxy, HTTP/2 flow control, and an async/awaited render — drained the full ~1.3–3 MB appended segment and closed cleanly. So the shape+size looks necessary but not sufficient: the production deadlock has an additional environmental co-factor absent from the minimal surface.
Leading co-factor candidates (in order)
proxy_bufferingdisk-spill at the configuredproxy_buffers/proxy_max_temp_file_sizethresholds (a local pass-through proxy doesn't spill to disk).AsyncLocalStoragerequest context, etc.) subtly changing the flush timing.Workaround (shipped, reliable)
For any Server Action that would
revalidatePath(currentRoute)and then return a value: drop the in-action revalidate and instead return the result and callrouter.refresh()on the client after the action settles.redirect()afterrevalidatePathis also safe (the redirect supersedes the appended segment). Only therevalidatePath(currentRoute) + return valueshape on a large route is affected.What we're asking
Happy to share the full minimal harness (repro app + the probe scripts behind the 379-fire table) and the raw production captures (HAR files, server render-span logs,
pg_stat_activitysnapshots) if useful.All reactions