Skip to content

fix(js/ts): decompress F#-compressed signature data in fcs-fable - #4684

Merged
dbrattli merged 4 commits into
mainfrom
fix/fcs-fable-compressed-sigdata
Jun 25, 2026
Merged

dbrattli merged 4 commits into
mainfrom
fix/fcs-fable-compressed-sigdata

Conversation

@dbrattli

@dbrattli dbrattli commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Problem

fable-standalone / fable-compiler-js (which run the F# Compiler Service compiled to JavaScript via fcs-fable) could not read referenced assemblies built by a modern (F# 8+) compiler.

Those compilers compress an assembly's embedded signature/optimization data into FSharpSignatureCompressedData. / FSharpOptimizationCompressedData. resources using raw DEFLATE. fcs-fable recognizes these resources but its decompressResource was a no-op stub under #if FABLE_COMPILER:

#if FABLE_COMPILER
    r.GetBytes() // no support for gunzip
#else
    ... DeflateStream ...
#endif

So it fed the still-compressed bytes straight to the unpickler, which misread a bogus ~461M "array length" → Array.zeroCreate/fill → Node JavaScript heap out of memory (invalid table size). Raising --max-old-space-size does not help — the bogus allocation exceeds V8's hashtable cap regardless of heap size. Native Fable.Cli is unaffected because it uses the prebuilt FCS DLL with a real DeflateStream.

How it surfaced

The build-standalone CI job started OOMing after Expecto was bumped 10.2.3 → 11.1.0 (#4680). Expecto 11 ships FSharpSignatureCompressedData; 10.2.3 shipped uncompressed FSharpSignatureData, which is why it had worked until now. This is a general latent bug — any modern assembly reference would hit it.

Fix

Add a dependency-free, synchronous pure-F# raw-DEFLATE (RFC 1951) inflater (src/fcs-fable/Inflate.fs) and use it in the #if FABLE_COMPILER branch of decompressResource.

A pure-managed inflate was chosen over pako/Node zlib because fable-standalone runs in the browser and is deliberately zero-runtime-dependency (everything is F# compiled to JS). It works identically in the browser REPL and Node, where System.IO.Compression is unavailable.

Verification

./build.sh test standalone (recompiles fcs-fable → JS and runs the suite against tests/Js/Main/Fable.Tests.fsproj, which references Expecto 11):

  • No OOM
  • 2918 tests pass, 0 fail

Once merged, dependabot PR #4680 needs a rebase/re-run to pick this up; its build-standalone job will then pass.

🤖 Generated with Claude Code


Alternatives considered (note to reviewers)

The other viable way to get inflate in the JS path is to use pako (pako.inflateRaw) from NPM instead of Inflate.fs. It's small, battle-tested, synchronous, and works in browser + Node.

The tradeoff: pako becomes a runtime dependency that consumers inherit (and must be bundled into fable-standalone's bundle.min.js). @fable-org/fable-standalone currently has zero runtime dependencies — everything is F# compiled to JS — so this PR keeps that property by shipping a self-contained Inflate.fs.

(A browser-native DecompressionStream('deflate-raw') was also considered but is async, and decompressResource is a synchronous call deep in the unpickler, so it doesn't fit without a larger refactor.)

If reviewers prefer the smaller diff and are comfortable adding the dependency, swapping Inflate.fs for pako is straightforward.


Demonstrated by this PR's own CI

This PR also bumps Expecto 10.2.3 → 11.1.0 in tests/Js/Main/Fable.Tests.fsproj (the project the build-standalone job compiles with fcs-fable). That makes this PR self-proving: its build-standalone job traverses the compressed-FSharpSignatureCompressedData path, so it would OOM without Inflate.fs and passes with it. Reviewers can confirm the fix by reverting just Inflate.fs + the decompressResource change and watching build-standalone fail.

(This Expecto bump overlaps with dependabot #4680 and reconciles harmlessly on rebase.)

F# 8+ compilers compress an assembly's embedded signature/optimization
data into FSharpSignatureCompressedData./FSharpOptimizationCompressedData.
resources (raw DEFLATE). fcs-fable recognized these resources but its
decompressResource was a no-op under #if FABLE_COMPILER ("no support for
gunzip"), so it fed the still-compressed bytes to the unpickler. That
misread a bogus ~461M array length and OOM'd Node when fable-standalone /
fable-compiler-js compiled a project referencing such an assembly. Native
Fable.Cli was unaffected since it uses the prebuilt FCS DLL with a real
DeflateStream.

This surfaced as the build-standalone CI job running out of memory after
Expecto was bumped 10.2.3 -> 11.1.0 (11 ships compressed signature data).

Add a dependency-free, synchronous pure-F# raw-DEFLATE (RFC 1951) inflater
(src/fcs-fable/Inflate.fs) and use it in the FABLE_COMPILER branch of
decompressResource. A pure-managed inflate keeps fable-standalone's
zero-runtime-dependency design and works identically in the browser REPL
and Node, where System.IO.Compression is unavailable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli dbrattli changed the title fix(standalone): decompress F#-compressed signature data in fcs-fable fix(js): decompress F#-compressed signature data in fcs-fable Jun 24, 2026
@dbrattli
dbrattli requested a review from MangelMaxime June 24, 2026 19:12
@dbrattli dbrattli changed the title fix(js): decompress F#-compressed signature data in fcs-fable fix(js|ts): decompress F#-compressed signature data in fcs-fable Jun 24, 2026
@dbrattli dbrattli changed the title fix(js|ts): decompress F#-compressed signature data in fcs-fable fix(js/ts): decompress F#-compressed signature data in fcs-fable Jun 24, 2026
Expecto 11.1.0 ships compressed signature data
(FSharpSignatureCompressedData). The standalone test compiles
tests/Js/Main/Fable.Tests.fsproj with fcs-fable, so bumping Expecto here
makes this PR's build-standalone CI actually traverse the compressed-data
path: it OOMs without the preceding Inflate.fs change and passes with it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@MangelMaxime MangelMaxime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of maintaining our own decompression library, I think it is ok to re-use an existing JavaScript library.

Now days it seems like fflate is the smallest and fastest library even compared to pako so I think it ok to make fable-standalone depends on it. fable-compiler-js will inherit it automatically.

@ncave

ncave commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

@MangelMaxime Idk, I like the one @dbrattli added in the PR, it's so small. It's unlikely decompressing this particular format will change much.

Thanks for the PR @dbrattli! ❤️

@ncave
ncave self-requested a review June 25, 2026 15:03

@ncave ncave left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, it can be merged here until the FCS fork is updated.

dbrattli and others added 2 commits June 25, 2026 18:17
Add direct unit tests for the pure-managed RFC 1951 inflate used by
fcs-fable in JS. They compress with DeflateStream and assert inflateRaw
round-trips it back, deliberately covering stored (btype 0), fixed
Huffman (btype 1), dynamic Huffman (btype 2), multi-block streams, and
the overlapping LZ77 copy path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Inflate.fs is original Fable-authored code, not copied from the upstream
F# compiler, so the boilerplate Microsoft copyright line was inaccurate.
Remove it; the existing Fable-only banner comment describes the file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli

Copy link
Copy Markdown
Collaborator Author

Thanks for the feedback @ncave and @MangelMaxime. I added some tests to be more confident. I'll merge it and it's quick to swap it for fflate later if we want to.

@dbrattli
dbrattli merged commit 62bfc61 into main Jun 25, 2026
23 checks passed
@dbrattli
dbrattli deleted the fix/fcs-fable-compressed-sigdata branch June 25, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants