Skip to content

Commit edb86b6

Browse files
Hardening v2.7claude
andcommitted
perf(llm): load the two stream-guard modules in parallel
Both output-cap sites awaited liveDeadlines and streamFaultInjection in sequence. The two imports are independent, so that costs an extra microtask hop on every streamed turn for nothing (react-doctor server-sequential-independent-await). This is the hot path: every answer passes through it. Both pairs are now a single Promise.all. The lone streamFaultInjection import in trackCommit has no partner and is left alone. Ported from the unmerged fix/stream-fault-injection (PR #464, closed). Its sibling commit — the fault-injection switches themselves — is already on main by another route; only this follow-up was left behind. A straight cherry-pick would have conflicted: the streamChat site now also destructures MAX_SUMMARY_OUTPUT_CHARS, which was added after that branch was cut, so the first hunk had to be re-derived rather than replayed. Behaviour is unchanged — same modules, same bindings, same order of use. Validation: - Build validated on macOS: tsc -p electron/tsconfig.json --noEmit, 0 errors - Covered by automated tests: StreamFaultInjection 8/8 (including "both output-cap sites honour the test ceiling", which pins exactly these two sites), RunawayStreamOutputCap 20/20, CodingRegenCeiling 3/3, PostCommitNoProviderSwitch 10/10 - Caveat: RunawayStreamOutputCap imports the built bundle from dist-electron, which predates this edit, so its runtime assertions exercised the pre-change build. Its source-assertion half did read the patched .ts. A rebuild would close that gap; the change is a semantically identical refactor, so the residual risk is low. - Platform-neutral: no platform-specific code path is touched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JJS2xjF8uDPMa4EpXey5FL
1 parent aa9748a commit edb86b6

1 file changed

Lines changed: 17 additions & 4 deletions

File tree

electron/LLMHelper.ts

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5529,8 +5529,17 @@ let isMultimodal = !!(imagePaths?.length);
55295529
// The ceiling is per-SURFACE: the live cap is calibrated from what_to_answer
55305530
// answers and is far too tight for a whole-meeting summary, which the batch
55315531
// callers reach through streamChatLongForm.
5532-
const { MAX_STREAM_OUTPUT_CHARS, MAX_SUMMARY_OUTPUT_CHARS } = await import('./llm/liveDeadlines');
5533-
const { testOutputCharCeiling } = await import('./llm/streamFaultInjection');
5532+
// Loaded TOGETHER: the two imports are independent, so awaiting them in
5533+
// sequence costs an extra microtask hop on every streamed turn for nothing
5534+
// (react-doctor server-sequential-independent-await). This is the hot path —
5535+
// every answer passes through it.
5536+
const [
5537+
{ MAX_STREAM_OUTPUT_CHARS, MAX_SUMMARY_OUTPUT_CHARS },
5538+
{ testOutputCharCeiling },
5539+
] = await Promise.all([
5540+
import('./llm/liveDeadlines'),
5541+
import('./llm/streamFaultInjection'),
5542+
]);
55345543
// Test switch (dev-only, opt-in) lets the cap be provoked without waiting
55355544
// for a model to actually loop. Null in any packaged build.
55365545
const outputCeiling = testOutputCharCeiling()
@@ -5629,8 +5638,12 @@ let isMultimodal = !!(imagePaths?.length);
56295638
state: { chars: number; truncated?: boolean },
56305639
label: string,
56315640
): AsyncGenerator<string, void, unknown> {
5632-
const { MAX_STREAM_OUTPUT_CHARS } = await import('./llm/liveDeadlines');
5633-
const { testOutputCharCeiling } = await import('./llm/streamFaultInjection');
5641+
// Same independent-await pair as streamChat above — one Promise.all rather
5642+
// than two sequential awaits on the per-chunk cap path.
5643+
const [{ MAX_STREAM_OUTPUT_CHARS }, { testOutputCharCeiling }] = await Promise.all([
5644+
import('./llm/liveDeadlines'),
5645+
import('./llm/streamFaultInjection'),
5646+
]);
56345647
const ceiling = testOutputCharCeiling() ?? MAX_STREAM_OUTPUT_CHARS;
56355648
for await (const chunk of inner) {
56365649
yield chunk;

0 commit comments

Comments
 (0)