Skip to content

Don't keep side-module wasm bytes alive after dlopen - #28002

Open
rustyconover wants to merge 2 commits into
emscripten-core:mainfrom
rustyconover:dylink-free-binary
Open

rustyconover wants to merge 2 commits into
emscripten-core:mainfrom
rustyconover:dylink-free-binary

Conversation

@rustyconover

@rustyconover rustyconover commented Oct 17, 2026 •

Copy link
Copy Markdown

After a synchronous dlopen, the side module's wasm bytes are now released once the module is compiled, instead of being kept in memory for the life of the program.

Cause

Inside loadWebAssemblyModule, loadModule captures binary. When a side module imports a symbol that isn't defined yet, proxyHandler creates a lazy stub that becomes one of the instance's imports. The stub does not reference binary directly: it uses resolveSymbol, which captures localScope from the same outer scope. In V8, these captured bindings can share a context, so the surviving stub's context chain can retain the full binary Uint8Array. This concerns captured bindings; it does not mean every local variable is retained. V8 documents this shared-environment behavior. The async path already replaces binary with the compiled WebAssembly.Module.

Effect

I found this while profiling extension loading in Haybarn, a DuckDB-wasm fork, where extensions are side modules loaded with dlopen. In Node 26 on aarch64, after LOAD spatial (a 26 MB side module) and two forced GCs, process.memoryUsage().arrayBuffers stayed 25.9 MB higher. With this change it's +0.03 MB, and RSS growth went from about 120–150 MB to about 92–118 MB. LOAD time didn't change (~387 ms). Every extension we loaded showed the same thing (iceberg 13.6 MB retained, ducklake 8.2 MB, autocomplete 3.7 MB).

Standalone repro (the new test): a MAIN_MODULE=2 program dlopens a side module with one not-yet-defined import and a 32 MB initialized EMSCRIPTEN_KEEPALIVE C array. The test checks that the compiled side module actually contains at least 32 MB of bytes and reserves 64 MB of initial Wasm memory for loading it. It runs gc() and compares process.memoryUsage().arrayBuffers before and after loading. Before: 32 MB retained. After: 0 MB.

Change

In the sync path, replace binary with the compiled module before instantiating, the same way the async path does. getDylinkMetadata(binary) runs before this, so nothing reads the bytes afterwards. A WebAssembly.Module passed in directly (for example from sharedModules under pthreads) is used as before.

Testing

  • Revised other.test_dlopen_binary_not_retained on Node 24.19.0 with --expose-gc: fails with the pre-fix loader ("retained: 32MB") and passes with the fix ("retained: 0MB"). Five consecutive repeated runs passed.
  • Fresh-cache EC2 run of core0/core2 × *dylink*/*dlfcn*/*dlopen*, plus other.*dylink*/*dlfcn*/*dlopen*/*MAIN_MODULE*/*main_module*/*side_module* and both dylink codesize tests: 393 tests total, 370 passed and 23 skipped.
  • Codesize relative to the pre-PR baseline: hello_dylink JS unchanged (gzip −3 bytes); hello_dylink_all JS unchanged.
  • Ruff and git diff --check passed.

🤖 Generated with Claude Code

In the synchronous path of `loadWebAssemblyModule`, the `binary` bytes stayed
referenced by the function's closure scope.  Any lazy import stub created by
`proxyHandler` (for a symbol not yet defined at load time) lives as long as the
module and shares that scope, so the full wasm bytes of every such side module
were kept in memory for the life of the program.

Replace `binary` with the compiled `WebAssembly.Module` before instantiating,
as the async path already does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread test/test_other.py Outdated
Comment thread test/test_other.py
Comment thread src/lib/libdylink.js Outdated
Comment thread src/lib/libdylink.js Outdated
Comment thread src/lib/libdylink.js
// though they do not reference it directly.
binary = binary instanceof WebAssembly.Module ? binary : new WebAssembly.Module(binary);
var instance = new WebAssembly.Instance(binary, info);
return postInstantiation(binary, instance);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually can you just add binary = null;, keeping the module local separate. I think this is explictly obvious what the intent it. Then the comment can be more clear too.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants