Turbopack production build duplicates shared client modules across chunks (no splitChunks-equivalent extraction) #96040
Replies: 6 comments 4 replies
|
This is a well-written proposal, and the reproduction makes the trade-off very clear. The metrics demonstrate that Turbopack currently optimizes large individual modules into separate chunks, but doesn't perform a second pass to extract shared dependency graphs that are referenced across multiple chunk groups. As a result, many small shared modules are emitted multiple times, increasing the total client JavaScript shipped. The comparison with webpack's
That provides an objective demonstration of the current behavior rather than relying solely on bundle size comparisons. I also agree with the scope you've outlined:
A configurable extraction threshold (similar in spirit to webpack's Since the Turbopack documentation already notes that production chunking is still evolving, this proposal feels like a natural optimization for future work. The included reproduction repository and quantitative measurements should make it much easier for the team to evaluate the impact and compare alternative chunking strategies. Overall, this seems like a valuable enhancement request, especially for applications with many lazily loaded features that share substantial dependency trees composed of smaller modules. |
|
This proposal matches what we've been measuring on Next.js 16.2.11, Pages Router, exactly — the "buckets by the exact set of chunk groups, no usage-count extraction" framing is spot on, and it accounts for every behaviour we saw. Adding a corroborating repro from a slightly different angle plus the user-facing cost, in case it's useful for the case. Repro: https://github.com/cultureamp/turbopack-chunk-duplication-repro Corroboration — and a static-route repro (no
|
| Turbopack | webpack | |
|---|---|---|
date-fns submodule copies (Σ embeddings over chunks) |
312 | 166 |
core helpers (constructFrom, toDate, constants) |
in all 12 route chunks | ≤ 8, in shared chunks |
| Total client JS | 284 KB gzip | 164 KB gzip |
Same shape as your 83-vs-40 physical-copy count.
The user-facing cost: multi-route navigation
One dimension worth adding to the motivation — the impact isn't on first load, it's on navigation:
| Turbopack | webpack | |
|---|---|---|
| First page visit | 117 KB gzip | 146 KB gzip |
| Full session (visit all 12 routes, browser cache reuse) | 269 KB gzip | 164 KB gzip (+64%) |
| — per additional route navigated | ~14 KB gzip | ~2 KB gzip |
First-page-load is actually smaller under Turbopack (webpack's shared chunk carries the union of every route's usage). The cost lands on multi-route sessions: because the shared code isn't extracted into a cacheable shared chunk, each new route re-downloads its own copy (~14 KB/route here) instead of reusing one already cached (~2 KB/route under webpack). So the proposal's shared-extraction pass would primarily improve cross-route navigation payload and total client JS, not initial load.
At real-world scale
On a 13-route production app (same Next 16, same config, only the bundler differs) the same mechanism inflates total client JS ~+84% gzip, with the heavy many-small-module deps copied per route — consistent with the "scales with (shared-module count) × (chunk groups sharing them)" scaling you describe.
Strong +1 on the proposal — a splitChunks-style extraction pass (with a minChunks-like threshold, at the production.rs bucketing step you point to) would resolve this for us. In the meantime next build --webpack is our workaround.
|
Thanks @lkm36486938 for the original post! And @sentience for the expansion. I've just put up an experimental PR to give users more options here: #96398. This allows you to configure I found that setting This was my module.exports = {
productionBrowserSourceMaps: true,
experimental: {
turbopackChunking: {
minChunkSize: 0,
},
turbopackSharedRuntime: true,
}
}And this was the results: Compared to Webpack: |
|
Thanks, @sampoder, @Mr-Nilarnab, and @sentience! |
|
Maybe this is relevant in the discussion https://github.com/project-millipede/next-slug-splitter |
|
This fixes a particular case with Pages router: #97664 |
Uh oh!
There was an error while loading. Please reload this page.
Goals
Non-Goals
instanceof#89192); that is the same underlying behavior but tracked separately as a bug.Background
webpack's
splitChunksextracts modules shared across chunk groups into shared chunks, so each module is emitted once. Turbopack's production chunking instead groups modules by the exact set of chunk groups that reference them, and has no usage-count-based extraction. As a result, when features import an overlapping-but-distinct subset of a shared dependency, the shared modules are duplicated across chunks.Minimal reproduction (no external deps beyond Next/React — 40 small shared modules imported by four
next/dynamicfeatures, each using a different overlapping window):https://github.com/lkm36486938/turbopack-shared-chunk-dup
Measured on
next@16.2.10(objective copy counts from.next/static/chunks):next build)next build --webpack)The effect scales with (shared-module count) × (number of chunk groups sharing them): a large dependency tree of small modules (e.g. a rich-text / math editor bundling KaTeX) imported by ~15–20 lazily-loaded feature entries duplicates the same modules ~15–20×, adding tens of MB of client JS. Turbopack's docs also note that production-optimized JS chunking is not yet fully implemented.
Current alternatives, all unsatisfactory:
next build --webpack— deduplicates, but gives up Turbopack.optimizePackageImports/ barrel-splitting — partial, high-maintenance, and doesn't reliably deduplicate shared modules across chunk groups.Proposal
Add a production chunking pass that extracts a module into a shared chunk when it is referenced by ≥ N chunk groups (or when extraction saves more than a byte threshold), even if the referencing chunk-group sets are not identical — analogous to webpack splitChunks minChunks / cacheGroups. In the current implementation this is where modules are bucketed by the exact chunk-group RoaringBitmap in turbopack-core/src/chunk/chunking/production.rs; the extraction would run before/alongside that bucketing. A config flag to opt in (and tune the threshold) would be a reasonable first step.
All reactions