Skip to content

fix: bound JSON output shutdown after interrupt - #7578

Open
harsh21234i wants to merge 5 commits into
reflex-dev:mainfrom
harsh21234i:fix/7510-bounded-json-drain
Open

harsh21234i wants to merge 5 commits into
reflex-dev:mainfrom
harsh21234i:fix/7510-bounded-json-drain

Conversation

@harsh21234i

@harsh21234i harsh21234i commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Bound shutdown after the first interrupt, including while output is being drained.
  • Isolate the POSIX child process group so Ctrl-C is forwarded exactly once.
  • Keep shutdown handlers active through drain and ignore repeated signals.
  • Bound oversized JSON records to the pipe's atomic-write limit so every emitted line remains parseable.
  • Add regression coverage for repeated Ctrl-C/SIGTERM, oversized records, drain cleanup, and forced test-process cleanup.
  • Add the Reflex Base bugfix news fragment.

Tests

  • tests/units/reflex_base/utils/test_log.py: 81 passed, 5 skipped on Windows (POSIX-only cases).
  • Linux Docker smoke tests: group SIGINT forwarded once; oversized output stayed valid JSON; blocked drain remained bounded after a second SIGTERM.
  • Ruff formatting, focused lint, and git diff --check passed.

Fixes #7510

@harsh21234i
harsh21234i requested a review from a team as a code owner October 15, 2026 13:57
@greptile-apps

greptile-apps Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] Fixes JSON output shutdown when interrupted.

Fix the nested-JSON output-thread failure before merging.

Findings

  1. P1 Nested logs stop output ▶

Summary

This PR bounds JSON-output shutdown from the first interrupt.

  • Interrupts start a bounded shutdown and reach the child group once.
  • Oversized JSON records stay parseable and fit pipe limits.

Both numbered previous findings are fixed. The new recursive truncation can stop an output thread on deeply nested JSON.

Reviews (3) · Last reviewed commit: "fix: preserve fields in oversized log re..." · Reviewed by Greptile

Comment thread packages/reflex-base/src/reflex_base/utils/log.py Outdated
Comment thread packages/reflex-base/src/reflex_base/utils/log.py
Comment thread packages/reflex-base/src/reflex_base/utils/log.py
Comment thread tests/units/reflex_base/utils/test_log.py Outdated
@codspeed

codspeed Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ 21 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

✅ 148 untouched benchmarks


Comparing harsh21234i:fix/7510-bounded-json-drain (e7f0078) with main (7ccaa4e)

Open in CodSpeed

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread tests/units/reflex_base/utils/test_log.py Outdated
Comment thread packages/reflex-base/news/7510.bugfix.md Outdated
Comment thread packages/reflex-base/src/reflex_base/utils/log.py Outdated
Comment thread packages/reflex-base/src/reflex_base/utils/log.py
Comment thread packages/reflex-base/src/reflex_base/utils/log.py

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread packages/reflex-base/src/reflex_base/utils/log.py Outdated
Comment thread packages/reflex-base/src/reflex_base/utils/log.py Outdated
Comment thread tests/units/reflex_base/utils/test_log.py

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread tests/units/reflex_base/utils/test_log.py
Comment on lines +424 to +429
if isinstance(value, dict):
return {
key: truncate_strings(item, max_chars) for key, item in value.items()
}
if isinstance(value, list):
return [truncate_strings(item, max_chars) for item in value]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Nested logs stop output

On supported Python 3.11, an oversized JSON record with roughly 600 nested containers can pass json.loads but fail inside truncate_strings. Each level adds both a function call and a comprehension frame, exhausting the allowed call depth sooner.

The resulting RecursionError escapes _write, which only catches OSError, and stops the output thread. Later child output can fill the unread pipe and hang the command. Use a walk that does not recurse, or catch this failure and emit a bounded fallback record.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/reflex-base/src/reflex_base/utils/log.py">

<violation number="1" location="packages/reflex-base/src/reflex_base/utils/log.py:426">
P1: Replace this recursive traversal with an iterative walk so deeply nested JSON records cannot raise `RecursionError` and terminate the output pump.</violation>

<violation number="2" location="packages/reflex-base/src/reflex_base/utils/log.py:434">
P2: A near-limit record dominated by non-string fields can trigger thousands of full copies and serializations before reaching the fallback, stalling output draining. Detect when retries cannot shrink the record or cap the retry work.</violation>

<violation number="3" location="packages/reflex-base/src/reflex_base/utils/log.py:435">
P2: Oversized records containing `NaN` or `Infinity` are emitted as non-standard JSON because this serialization uses the default `allow_nan=True`. Normalize non-finite values before dumping so strict JSON consumers can parse every emitted line.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

return value if len(value) <= max_chars else value[:max_chars] + marker
if isinstance(value, dict):
return {
key: truncate_strings(item, max_chars) for key, item in value.items()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Replace this recursive traversal with an iterative walk so deeply nested JSON records cannot raise RecursionError and terminate the output pump.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/reflex-base/src/reflex_base/utils/log.py, line 426:

<comment>Replace this recursive traversal with an iterative walk so deeply nested JSON records cannot raise `RecursionError` and terminate the output pump.</comment>

<file context>
@@ -416,32 +416,30 @@ def _truncate_json_record(record: bytes, max_bytes: int) -> bytes:
+            return value if len(value) <= max_chars else value[:max_chars] + marker
+        if isinstance(value, dict):
+            return {
+                key: truncate_strings(item, max_chars) for key, item in value.items()
+            }
+        if isinstance(value, list):
</file context>


payload["truncated"] = True
max_chars = max_bytes
while True:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A near-limit record dominated by non-string fields can trigger thousands of full copies and serializations before reaching the fallback, stalling output draining. Detect when retries cannot shrink the record or cap the retry work.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/reflex-base/src/reflex_base/utils/log.py, line 434:

<comment>A near-limit record dominated by non-string fields can trigger thousands of full copies and serializations before reaching the fallback, stalling output draining. Detect when retries cannot shrink the record or cap the retry work.</comment>

<file context>
@@ -416,32 +416,30 @@ def _truncate_json_record(record: bytes, max_bytes: int) -> bytes:
+
+    payload["truncated"] = True
+    max_chars = max_bytes
+    while True:
+        result = json.dumps(truncate_strings(payload, max_chars)).encode() + b"\n"
+        if len(result) <= max_bytes:
</file context>

payload["truncated"] = True
max_chars = max_bytes
while True:
result = json.dumps(truncate_strings(payload, max_chars)).encode() + b"\n"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Oversized records containing NaN or Infinity are emitted as non-standard JSON because this serialization uses the default allow_nan=True. Normalize non-finite values before dumping so strict JSON consumers can parse every emitted line.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/reflex-base/src/reflex_base/utils/log.py, line 435:

<comment>Oversized records containing `NaN` or `Infinity` are emitted as non-standard JSON because this serialization uses the default `allow_nan=True`. Normalize non-finite values before dumping so strict JSON consumers can parse every emitted line.</comment>

<file context>
@@ -416,32 +416,30 @@ def _truncate_json_record(record: bytes, max_bytes: int) -> bytes:
+    payload["truncated"] = True
+    max_chars = max_bytes
+    while True:
+        result = json.dumps(truncate_strings(payload, max_chars)).encode() + b"\n"
+        if len(result) <= max_bytes:
+            return result
</file context>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reflex run --json: the 30 s drain cap can end the stream mid-record, and a consumer that stops reading blocks shutdown forever

1 participant