Skip to content

perf(electron): build the Next standalone once and hydrate natives per leg (#10321 stage 8/8) - #10390

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
backryun:perf/electron-shared-web-build
Aug 16, 2026
Merged

perf(electron): build the Next standalone once and hydrate natives per leg (#10321 stage 8/8)#10390
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
backryun:perf/electron-shared-web-build

Conversation

@backryun

Copy link
Copy Markdown
Contributor

Summary

Stage 8 of the Electron efficiency roadmap (#10321): the desktop release matrix currently runs the full Next.js standalone build on all four legs (windows, macos-intel, macos-arm64, linux), duplicating the platform-neutral majority of that work four times, and re-exposing every leg to the hosted-runner RAM failure class that took the linux leg out of v3.8.49.

This PR builds the standalone once in a new web-build job (ubuntu, webpack — the same RAM escape hatch the linux leg already documents) and has each leg restore + hydrate it:

  • scripts/build/standaloneTarball.mjs — deterministic, dependency-free tar.gz writer/reader (uid/gid/mtime pinned, sorted entries, symlink + exec-bit preservation; GNU-tar interop covered by tests)
  • scripts/build/standaloneManifest.mjs — byte-level manifest of .build/next (sha256 + size + symlink target per entry, plus the archive's own digest); catches artifact-transfer corruption before extraction and re-verifies the restored tree byte-for-byte (unlisted/smuggled files rejected)
  • scripts/build/standaloneBundle.mjspack / restore / hydrate CLI
  • scripts/build/hydrateNativeDeps.mjs — swaps install-machine-forked native optionals (@img/sharp-*, @ngrok/ngrok-*, fsevents) from the leg's own npm ci into the restored tree, then asserts the bundled-native closure (koffi triplets, better-sqlite3 prebuilds, wreq-js, onnxruntime — with its documented darwin-x64 exemption) services the leg's matrix.os/matrix.arch before packaging starts
  • .github/workflows/electron-release.yml — new web-build job; legs download → restore → hydrate and skip the per-leg build

Rollback & failure semantics

  • Rollback: the legacy per-leg build is kept and runs when the ELECTRON_SHARED_STANDALONE workflow_dispatch input is disabled (or the shared job was skipped) — a single-input revert to the v-prior behavior.
  • Fail-closed: if web-build ran and failed, legs do not silently fall back to four per-leg builds; the matrix job condition treats it as a hard failure so the regression the shared job exists to surface stays visible.
  • Signing / smoke / packaging steps are untouched — they consume .build/next exactly as before; restore verifies the tree is byte-identical to what web-build produced, then hydrate makes it platform-correct.

Measurements (issue gate 4 baseline)

  • Current four-build matrix: 111.4 runner-minutes per release run (intel 39.7 / arm64 37.5 / win 19.6 / linux 14.6), dominated by 4× duplicated Next standalone builds.
  • Post-change expected compute: one ubuntu standalone build (~7 min) + 4× (install-only + restore + hydrate). Hydrate itself is 31 ms-class (checksums + copies, no compile) — the same class of saving validated in the stage-6 prebuild work (perf(electron): verify better-sqlite3 v13 Node-API prebuilds instead of source rebuild (6/8) #10367).
  • Actual runner-minute/wall-clock comparison will be recorded on the first release run after merge (issue gate 4 completion).

Local verification

  • node --test tests/unit/build/standalone-bundle.test.ts — 7/7 (roundtrip incl. symlinks/exec bits, byte determinism, GNU-tar interop, tamper + smuggle detection, forked-native swap, native-closure serviceability + darwin-x64 exemption)
  • Full lifecycle smoke against a real dependency tree: pack → restore → tree equality → hydrate (darwin/arm64: copied=3 replaced=2 removed=2) → natives verified
  • prettier/eslint/tsc clean (scripts dir is eslint-ignored, same as existing build scripts)
  • Workflow YAML validated with both js-yaml and python3 yaml.safe_load

Notes for reviewers

Closes the stage-8 implementation gates of #10321 (runner-minute comparison to be appended after the first release run).

@backryun
backryun requested a review from diegosouzapw as a code owner August 14, 2026 14:46
@backryun
backryun force-pushed the perf/electron-shared-web-build branch from 6dfbcaf to f0e7ae2 Compare August 14, 2026 14:51
@backryun
backryun force-pushed the perf/electron-shared-web-build branch from f0e7ae2 to 99b3d58 Compare August 14, 2026 15:54
@backryun backryun changed the title perf(electron): build the Next standalone once and hydrate natives per leg (#10321 stage 8) perf(electron): build the Next standalone once and hydrate natives per leg (#10321 stage 8/8) Aug 14, 2026
@backryun
backryun force-pushed the perf/electron-shared-web-build branch 2 times, most recently from a766fe4 to 8856988 Compare August 14, 2026 17:06
diegosouzapw added a commit that referenced this pull request Aug 14, 2026
…#9985) (#10396)

`check:dead-code` reports 410 dead symbols against a 409 baseline on the
pristine `release/v3.8.50` tip, so every PR on the branch is born red on that
gate (#10386, #10393, #10390, #10388, #10382 all fail it).

Isolated the +1 by diffing knip 6.32 reports between the rebaseline commit
97aac6a (409) and the tip (410): `resolveOpencodeConfigDir` in
`src/shared/services/cliRuntime.ts`. #10246 moved the canonical resolvers into
`opencodeConfigPath.ts` and left this wrapper behind; the same commit removed
its last consumer.

The wrapper was not just unused, it was divergent: it returned
`path.dirname()` of the canonical value — `~/.config` rather than
`~/.config/opencode` — so any future caller reaching for it by name would have
written the OpenCode config one directory too high.

Removed the wrapper and its now-unused import. A new test pins the canonical
resolver's contract and asserts the divergent re-export stays gone; the guard
was mutation-validated (re-adding the wrapper fails it).

check:dead-code: 409 = baseline, PASS.
cliRuntime/opencode suites: 51 pass, 0 fail. New guard: 3 pass, 0 fail.
lint / typecheck:core / file-size / complexity-ratchets / test-discovery: green.

Co-authored-by: Xiangzhe <bakryun0718@proton.me>
@backryun
backryun force-pushed the perf/electron-shared-web-build branch 7 times, most recently from 6e5904f to 3fd0d93 Compare August 15, 2026 07:20
@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks for this — Stage 8 closes out the roadmap nicely and the engineering here is solid. I ran the new test suite independently against your head commit (tests/unit/build/standalone-bundle.test.ts) and confirmed all 7 cases pass, including the tamper/smuggle-detection and native-fork-hydration paths. The rollback lever (ELECTRON_SHARED_STANDALONE=disabled) and fail-closed semantics are exactly the right shape for a change this central to the release pipeline.

One thing worth fixing before merge: runHydrate() in standaloneBundle.mjs was clearly designed to accept a comma-separated --arch list — the doc comment says "the linux leg ships x64+arm64" — but the workflow's linux matrix entry only sets arch: x64. Since electron/package.json's build.linux.target config actually packages both x64 and arm64 AppImage/deb in that same job, the new bundled-native serviceability check never actually verifies the arm64 target it was built to protect. Could you change that matrix entry to arch: x64,arm64? Should be a one-line fix and the existing test already covers the multi-arch path.

Everything else looks ready as-is. Nice work on the deterministic tar/manifest layer — the zip-slip protection and byte-level tamper detection are a genuinely good addition on top of the runner-minute savings.

@backryun
backryun force-pushed the perf/electron-shared-web-build branch from 3fd0d93 to bc5bee5 Compare August 15, 2026 10:48
@backryun

Copy link
Copy Markdown
Contributor Author

Addressed the requested item in commit bc5bee5: the Linux release matrix now passes x64,arm64 to the hydration verification, covering both architectures packaged by that job. The standalone bundle suite passes 7/7, and the workflow YAML validates with the updated matrix value. Please take another look when you have a chance.

@backryun
backryun force-pushed the perf/electron-shared-web-build branch 2 times, most recently from 480c372 to 08c98bd Compare August 15, 2026 17:16
…r leg (diegosouzapw#10321 stage 8)

The desktop release matrix ran the full Next.js standalone build on all four legs (windows, macos-intel, macos-arm64, linux), duplicating the platform-neutral majority of that work four times and re-exposing every leg to the hosted-runner RAM class of failure that took the linux leg out of v3.8.49.

- scripts/build/standaloneTarball.mjs: deterministic, dependency-free tar.gz writer/reader (uid/gid/mtime pinned, sorted entries, symlink + exec-bit preservation; GNU-tar interop covered by tests).
- scripts/build/standaloneManifest.mjs: byte-level manifest of .build/next (sha256 + size + symlink target per entry, plus the archive's own digest) catching artifact-transfer corruption before extraction and re-verifying the restored tree byte-for-byte, smuggling included.
- scripts/build/standaloneBundle.mjs: pack / restore / hydrate CLI over the two modules above.
- scripts/build/hydrateNativeDeps.mjs: swaps install-machine-forked native optionals (@img/sharp-*, @ngrok/ngrok-*, fsevents) from the leg's own npm ci into the restored tree, then verifies the bundled-native closure (koffi triplets, better-sqlite3 prebuilds, wreq-js, onnxruntime with its documented darwin-x64 exemption) services the leg's platform/arch before packaging starts.
- .github/workflows/electron-release.yml: new web-build job builds the standalone once on ubuntu with webpack and uploads the bundle; legs download, restore, and hydrate it, skipping the per-leg build. The legacy per-leg build remains as a rollback path via the ELECTRON_SHARED_STANDALONE workflow_dispatch input, and legs fail closed if web-build ran and failed.

Regression tests cover archive roundtrip, byte determinism, manifest tamper/smuggle detection, forked-native swaps, and native-closure serviceability.
@backryun
backryun force-pushed the perf/electron-shared-web-build branch from 08c98bd to b31c629 Compare August 16, 2026 03:33
@diegosouzapw
diegosouzapw merged commit 6b85413 into diegosouzapw:release/v3.8.50 Aug 16, 2026
4 checks passed
@diegosouzapw

Copy link
Copy Markdown
Owner

Validated in local merge-train .claude/worktrees/merge-train-20260816-002347-suite.log (train7, 8be62df6) / train8 1e5c976f on devbox-vm-06-dev002 (FAST gates green: static + changed tests + vitest; daily full-suite run still required). Merged under batch merge-train protocol (merge-gates §7). ⚠️ base-red inherited: #10523 (ServiceSupervisor order-dependent test, unrelated to this batch).

@backryun
backryun deleted the perf/electron-shared-web-build branch August 16, 2026 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants