fix(js/ts): decompress F#-compressed signature data in fcs-fable#4684
Open
dbrattli wants to merge 2 commits into
Open
fix(js/ts): decompress F#-compressed signature data in fcs-fable#4684dbrattli wants to merge 2 commits into
dbrattli wants to merge 2 commits into
Conversation
F# 8+ compilers compress an assembly's embedded signature/optimization
data into FSharpSignatureCompressedData./FSharpOptimizationCompressedData.
resources (raw DEFLATE). fcs-fable recognized these resources but its
decompressResource was a no-op under #if FABLE_COMPILER ("no support for
gunzip"), so it fed the still-compressed bytes to the unpickler. That
misread a bogus ~461M array length and OOM'd Node when fable-standalone /
fable-compiler-js compiled a project referencing such an assembly. Native
Fable.Cli was unaffected since it uses the prebuilt FCS DLL with a real
DeflateStream.
This surfaced as the build-standalone CI job running out of memory after
Expecto was bumped 10.2.3 -> 11.1.0 (11 ships compressed signature data).
Add a dependency-free, synchronous pure-F# raw-DEFLATE (RFC 1951) inflater
(src/fcs-fable/Inflate.fs) and use it in the FABLE_COMPILER branch of
decompressResource. A pure-managed inflate keeps fable-standalone's
zero-runtime-dependency design and works identically in the browser REPL
and Node, where System.IO.Compression is unavailable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Expecto 11.1.0 ships compressed signature data (FSharpSignatureCompressedData). The standalone test compiles tests/Js/Main/Fable.Tests.fsproj with fcs-fable, so bumping Expecto here makes this PR's build-standalone CI actually traverse the compressed-data path: it OOMs without the preceding Inflate.fs change and passes with it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
fable-standalone/fable-compiler-js(which run the F# Compiler Service compiled to JavaScript viafcs-fable) could not read referenced assemblies built by a modern (F# 8+) compiler.Those compilers compress an assembly's embedded signature/optimization data into
FSharpSignatureCompressedData./FSharpOptimizationCompressedData.resources using raw DEFLATE.fcs-fablerecognizes these resources but itsdecompressResourcewas a no-op stub under#if FABLE_COMPILER:So it fed the still-compressed bytes straight to the unpickler, which misread a bogus ~461M "array length" →
Array.zeroCreate/fill→ NodeJavaScript heap out of memory(invalid table size). Raising--max-old-space-sizedoes not help — the bogus allocation exceeds V8's hashtable cap regardless of heap size. NativeFable.Cliis unaffected because it uses the prebuilt FCS DLL with a realDeflateStream.How it surfaced
The
build-standaloneCI job started OOMing after Expecto was bumped 10.2.3 → 11.1.0 (#4680). Expecto 11 shipsFSharpSignatureCompressedData; 10.2.3 shipped uncompressedFSharpSignatureData, which is why it had worked until now. This is a general latent bug — any modern assembly reference would hit it.Fix
Add a dependency-free, synchronous pure-F# raw-DEFLATE (RFC 1951) inflater (
src/fcs-fable/Inflate.fs) and use it in the#if FABLE_COMPILERbranch ofdecompressResource.A pure-managed inflate was chosen over
pako/Nodezlibbecausefable-standaloneruns in the browser and is deliberately zero-runtime-dependency (everything is F# compiled to JS). It works identically in the browser REPL and Node, whereSystem.IO.Compressionis unavailable.Verification
./build.sh test standalone(recompilesfcs-fable→ JS and runs the suite againsttests/Js/Main/Fable.Tests.fsproj, which references Expecto 11):Once merged, dependabot PR #4680 needs a rebase/re-run to pick this up; its
build-standalonejob will then pass.🤖 Generated with Claude Code
Alternatives considered (note to reviewers)
The other viable way to get inflate in the JS path is to use
pako(pako.inflateRaw) from NPM instead ofInflate.fs. It's small, battle-tested, synchronous, and works in browser + Node.The tradeoff:
pakobecomes a runtime dependency that consumers inherit (and must be bundled intofable-standalone'sbundle.min.js).@fable-org/fable-standalonecurrently has zero runtime dependencies — everything is F# compiled to JS — so this PR keeps that property by shipping a self-containedInflate.fs.(A browser-native
DecompressionStream('deflate-raw')was also considered but is async, anddecompressResourceis a synchronous call deep in the unpickler, so it doesn't fit without a larger refactor.)If reviewers prefer the smaller diff and are comfortable adding the dependency, swapping
Inflate.fsforpakois straightforward.Demonstrated by this PR's own CI
This PR also bumps Expecto 10.2.3 → 11.1.0 in
tests/Js/Main/Fable.Tests.fsproj(the project thebuild-standalonejob compiles withfcs-fable). That makes this PR self-proving: itsbuild-standalonejob traverses the compressed-FSharpSignatureCompressedDatapath, so it would OOM withoutInflate.fsand passes with it. Reviewers can confirm the fix by reverting justInflate.fs+ thedecompressResourcechange and watchingbuild-standalonefail.(This Expecto bump overlaps with dependabot #4680 and reconciles harmlessly on rebase.)