fix(compression): drain the worker pool queue when worker construction throws - #12755
Open
insoln wants to merge 14 commits into
Open
fix(compression): drain the worker pool queue when worker construction throws#12755insoln wants to merge 14 commits into
insoln wants to merge 14 commits into
Conversation
…ronously A synchronous spawn failure (Turbopack standalone moduleContext MODULE_NOT_FOUND) escaped dispatch() before queue.shift(), stranding every queued job — each holding a full parsed request body — in pool.queue for the process lifetime (unbounded main-isolate heap leak, #2). - dispatch() now fail-opens the whole queue on a spawn throw and marks the pool broken: subsequent run() calls resolve unchanged immediately instead of re-jamming - constructor accepts a workerFactory test seam - strategySelector fail-open catch now logs (rate-limited): the leak was invisible precisely because the catch was silent - llmlingua worker passes a URL object to new Worker (ERR_WORKER_PATH on Node >= 21 for file:// strings)
…and dark llmlingua catches - extract notifyCompressionFailOpen into failOpenNotifier.ts (import-cycle-free) - broken-pool run() now notifies instead of failing open silently forever - rate limiter keys on detail so distinct new failure modes are never swallowed - llmlingua pump/postMessage catches log through the notifier
insoln
marked this pull request as ready for review
September 4, 2026 23:28
- errorText/strategySelector/llmlingua now pass the raw unknown to sanitizeErrorMessage (never String(error), which throws on hostile toString/valueOf) so run() cannot reject on a non-Error spawn failure - add an injectable worker harness to the llmlingua backend and cover the spawn-throw and postMessage-throw fail-open paths (original text, queue continuation, respawn, rate-limited warn) - add hostile-throwable regression test for the pool
Open
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prevents the compression worker pool from retaining queued request bodies when worker construction throws synchronously.
This is deliberately a containment fix: it removes the queue-retention failure mode and makes fail-open behavior observable, but it does not repair the standalone build's worker resolution or worker dependency packaging. Compression still returns the original body when no worker can be created.
Changes
postMessagethrows, and pool shutdown.URLobject to the LLMLingua worker constructor on Node.js 21+.tap.testFiles.Validation
node --import tsx/esm --test tests/unit/compression/compression-worker-pool-jam.test.ts— 14 passedcomplexity9 → 9;cognitive-complexity3 → 3)git diff --check— passedThe release branch is currently red independently of this change. The inherited failures include the
glm.tsTS2554 error, invalid changelog fragment, provider-asset provenance object, mutation-list drift forquotaScoring.ts, and unrelated unit-test failures.Tests Added
tests/unit/compression/compression-worker-pool-jam.test.tsOut of scope