Skip to content

Commit 2162289

Browse files
authored
perf(electron): verify better-sqlite3 v13 Node-API prebuilds instead of source rebuild (#10367)
better-sqlite3 v13 ships Node-API prebuilds for every packaged platform (darwin/linux/linuxmusl/win32 x x64/arm64) inside the npm tarball, so the Electron-ABI node-gyp source rebuild in prepare-electron-standalone.mjs is obsolete. Replace it with a fail-fast prebuild verification that mirrors better-sqlite3 lib/binding.js selection, and strip build/deps/src so the packaged loader can only resolve the prebuild. Verified locally on darwin-arm64: the same darwin-arm64.node prebuild loads under both Node 24 (NODE_MODULE_VERSION 137) and Electron 43.3.0 under ELECTRON_RUN_AS_NODE (148); DB create/migrate/read/write/close/reopen pass in both runtimes and cross-runtime on each other's database files. Issue #10321 Stage 6.
1 parent 6d93360 commit 2162289

7 files changed

Lines changed: 217 additions & 133 deletions

File tree

.github/workflows/ci.yml

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -697,11 +697,12 @@ jobs:
697697
runs-on: ${{ matrix.os }}
698698
timeout-minutes: 30
699699
needs: build
700-
# WS1.5 (v3.8.49 plan): the Electron rebuild/spawn path previously executed for
700+
# WS1.5 (v3.8.49 plan): the Electron native-module path previously executed for
701701
# the FIRST time on the release tag — the v3.8.48 Windows bug (npx.cmd spawned
702702
# without shell, CVE-2024-27980 behavior change) could only surface at release.
703-
# windows-latest runs prepare:bundle (the ABI rebuild + spawn plan) per release
704-
# PR; ubuntu keeps the full pack + headless smoke.
703+
# windows-latest runs prepare:bundle (better-sqlite3 prebuild verification since
704+
# v13 — the node-gyp rebuild is gone) per release PR; ubuntu keeps the full
705+
# pack + headless smoke.
705706
strategy:
706707
fail-fast: false
707708
matrix:
@@ -738,7 +739,7 @@ jobs:
738739
# precedent): its first-ever real run (2026-07-15, run 29457533565) died in
739740
# 0.7s with the error swallowed by pwsh — bash shell captures stderr and
740741
# continue-on-error keeps the heavy gate green while we harden it (#7336).
741-
- name: Prepare Electron standalone (Windows ABI rebuild + spawn path)
742+
- name: Prepare Electron standalone (Windows prebuild verification)
742743
if: runner.os == 'Windows'
743744
working-directory: electron
744745
continue-on-error: true

docs/guides/ELECTRON_GUIDE.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -39,15 +39,15 @@ system tray, auto-updater, IPC bridge, and zero-config secret bootstrap.
3939

4040
Confirmed from `electron/package.json`:
4141

42-
| Package | Version |
43-
| ------------------ | -------------------------- |
44-
| `electron` | `^41.5.1` |
45-
| `electron-builder` | `^26.10.0` |
46-
| `electron-updater` | `^6.8.5` |
47-
| `better-sqlite3` | `^12.9.0` |
48-
| App version | `3.8.0` |
49-
| App id | `online.omniroute.desktop` |
50-
| Product name | `OmniRoute` |
42+
| Package | Version |
43+
| ------------------ | --------------------------------------------------------- |
44+
| `electron` | `^41.5.1` |
45+
| `electron-builder` | `^26.10.0` |
46+
| `electron-updater` | `^6.8.5` |
47+
| `better-sqlite3` | root `^13.0.2` (Node-API prebuilds — no Electron rebuild) |
48+
| App version | `3.8.0` |
49+
| App id | `online.omniroute.desktop` |
50+
| Product name | `OmniRoute` |
5151

5252
## Scripts (root `package.json`)
5353

@@ -260,14 +260,14 @@ Releases are published to GitHub Releases (`diegosouzapw/OmniRoute`), which is a
260260

261261
## Troubleshooting
262262

263-
| Symptom | Fix |
264-
| --------------------------------------------------------------- | --------------------------------------------------------------------------- |
265-
| `Cannot find module 'better-sqlite3'` after Electron major bump | `cd electron && npm rebuild` |
266-
| `ERR_DLOPEN_FAILED` for native module | Re-run `prepare:bundle` and verify ABI matches Electron's Node |
267-
| Window appears blank on Linux | Confirm Next.js server actually bound to PORT (check `[Server]` logs) |
268-
| macOS notarization stalls | Ensure `APPLE_*` vars are exported, not just in `.env` |
269-
| Windows SmartScreen warning | Sign with EV cert, or users right-click → "Run anyway" |
270-
| Smoke test fails with port-in-use | Stop any local dev server on 20128 before running `electron:smoke:packaged` |
263+
| Symptom | Fix |
264+
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
265+
| `Cannot find module 'better-sqlite3'` after Electron major bump | better-sqlite3 v13 ships Node-API prebuilds — re-run `npm install` at the root and `prepare:bundle` (it verifies the prebuild for the current platform) |
266+
| `ERR_DLOPEN_FAILED` for native module | Re-run `prepare:bundle` — it fails fast when the Node-API prebuild for the current platform is missing |
267+
| Window appears blank on Linux | Confirm Next.js server actually bound to PORT (check `[Server]` logs) |
268+
| macOS notarization stalls | Ensure `APPLE_*` vars are exported, not just in `.env` |
269+
| Windows SmartScreen warning | Sign with EV cert, or users right-click → "Run anyway" |
270+
| Smoke test fails with port-in-use | Stop any local dev server on 20128 before running `electron:smoke:packaged` |
271271

272272
## See Also
273273

scripts/build/assembleStandalone.mjs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
* prune + validate (pack-artifact-policy) - Y - UNIQUE (prepublish)
4040
* data/ dir creation - Y - UNIQUE (prepublish)
4141
* --- electron-UNIQUE ---
42-
* better-sqlite3 native strip + Electron-ABI rebuild - - Y UNIQUE (electron)
42+
* better-sqlite3 prebuild verify + compile-input strip - - Y UNIQUE (electron)
4343
* Turbopack hashed-module symlink materialize (node_modules) - - Y SHARED (opt-in: materializeSymlinks)
4444
* symlink guard (assertBundleIsPackagable) - - Y UNIQUE (electron)
4545
* removeGeneratedElectronArtifacts - - Y UNIQUE (electron)
Lines changed: 69 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,73 @@
11
/**
2-
* Spawn plan for the better-sqlite3 Electron-ABI rebuild (pure — import-safe for tests).
2+
* better-sqlite3 Node-API prebuild planning (pure — import-safe for tests).
33
*
4-
* On Windows, `npx.cmd` MUST be spawned through a shell: since Node's
5-
* CVE-2024-27980 hardening, spawning `.cmd`/`.bat` shims without `shell: true`
6-
* fails outright (spawnSync returns `status: null`), which broke the v3.8.47
7-
* tag build ("better-sqlite3 rebuild against electron 43.1.0 failed (exit null)").
8-
* The args are a fixed literal list — no untrusted input reaches the shell.
4+
* Since better-sqlite3 v13 the packaged app no longer compiles the addon from
5+
* source against the Electron headers: v13 ships Node-API (NAPI_VERSION=10)
6+
* prebuilds for every platform we package, and Node-API addons are
7+
* ABI-independent, so the same prebuild runs under plain Node and under the
8+
* packaged app's ELECTRON_RUN_AS_NODE server (verified against electron 43 /
9+
* NODE_MODULE_VERSION 148 — issue #10321 Stage 6). The historical
10+
* `npx node-gyp rebuild` spawn plan existed because better-sqlite3@12 only
11+
* shipped prebuilds up to electron-v146; v13 makes it obsolete.
12+
*
13+
* This module mirrors better-sqlite3's own `lib/binding.js` selection logic so
14+
* the build fails fast when the prebuild the runtime loader would pick is
15+
* missing, instead of shipping an app that falls back to sql.js and OOMs on a
16+
* user machine.
17+
*/
18+
19+
import { existsSync } from "node:fs";
20+
import { join } from "node:path";
21+
22+
export const SQLITE_PREBUILD_PLATFORMS = ["darwin", "linux", "linuxmusl", "win32"];
23+
export const SQLITE_PREBUILD_ARCHS = ["x64", "arm64"];
24+
25+
/**
26+
* Resolve the prebuild file name better-sqlite3's loader would pick for the
27+
* given platform/arch. Mirrors lib/binding.js: linux without a glibc runtime
28+
* version resolves to the linuxmusl prebuild.
29+
*
30+
* @param {string} platform - process.platform ("linux", "darwin", "win32")
31+
* @param {string} arch - process.arch ("x64", "arm64")
32+
* @param {{ glibcVersionRuntime?: string | null }} [reportHeader] - parsed
33+
* process.report.getReport().header (injectable for tests)
34+
*/
35+
export function sqlitePrebuildFileName(platform, arch, reportHeader) {
36+
const isMusl = platform === "linux" && !reportHeader?.glibcVersionRuntime;
37+
const target = `${isMusl ? "linuxmusl" : platform}-${arch}`;
38+
return `${target}.node`;
39+
}
40+
41+
/**
42+
* Whether a prebuild check applies for this platform/arch combination.
43+
* Unsupported combos (e.g. freebsd-ia32) are skipped rather than failed: the
44+
* runtime loader falls back to node-gyp build/ locations for those, which we
45+
* do not package.
46+
*/
47+
export function isSqlitePrebuildSupported(platform, arch) {
48+
return SQLITE_PREBUILD_PLATFORMS.includes(platform) && SQLITE_PREBUILD_ARCHS.includes(arch);
49+
}
50+
51+
/**
52+
* Assert that the runtime-selected prebuild exists in a staged module.
53+
* Unsupported platform/arch combinations retain the historical fallback path.
54+
*
55+
* @returns {string | null} selected prebuild path, or null when unsupported
956
*/
10-
export function buildRebuildSpawnPlan(platform) {
11-
const win = platform === "win32";
12-
return {
13-
command: win ? "npx.cmd" : "npx",
14-
args: ["--yes", "node-gyp", "rebuild"],
15-
shell: win,
16-
};
57+
export function assertSqlitePrebuildExists(moduleDir, platform, arch, reportHeader) {
58+
if (!isSqlitePrebuildSupported(platform, arch)) return null;
59+
60+
const expected = join(
61+
moduleDir,
62+
"prebuilds",
63+
sqlitePrebuildFileName(platform, arch, reportHeader)
64+
);
65+
if (!existsSync(expected)) {
66+
throw new Error(
67+
`[electron] better-sqlite3 prebuild missing for ${platform}-${arch} ` +
68+
`(${expected}). The packaged app would fall back to sql.js and OOM. ` +
69+
`Restore the prebuilds/ directory (npm cache / registry tarball) before packaging.`
70+
);
71+
}
72+
return expected;
1773
}

scripts/build/prepare-electron-standalone.mjs

Lines changed: 39 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
11
#!/usr/bin/env node
22

3-
import { cpSync, existsSync, lstatSync, readFileSync, readdirSync, rmSync } from "node:fs";
3+
import { existsSync, lstatSync, readdirSync, rmSync } from "node:fs";
44
import { basename, dirname, join, relative } from "node:path";
55
import { fileURLToPath } from "node:url";
6-
import { spawnSync } from "node:child_process";
76
import { assembleStandalone } from "./assembleStandalone.mjs";
8-
import { buildRebuildSpawnPlan } from "./electronRebuildPlan.mjs";
7+
import { assertSqlitePrebuildExists } from "./electronRebuildPlan.mjs";
98
import { pruneElectronRuntimeDocs } from "./electronRuntimeDocs.mjs";
109

1110
const __filename = fileURLToPath(import.meta.url);
@@ -90,9 +89,7 @@ function removeNativeModules(baseDir, prefixes = ["keytar"]) {
9089
// user machine as "Internal Server Error" on every route.
9190
function assertNoStaleHashedNatives(baseDir, prefixes) {
9291
if (!existsSync(baseDir)) return;
93-
const leftovers = readdirSync(baseDir).filter((dir) =>
94-
prefixes.some((p) => dir.startsWith(p))
95-
);
92+
const leftovers = readdirSync(baseDir).filter((dir) => prefixes.some((p) => dir.startsWith(p)));
9693
if (leftovers.length > 0) {
9794
throw new Error(
9895
`[electron] stale native module copies survived cleanup in ${baseDir}: ` +
@@ -102,77 +99,43 @@ function assertNoStaleHashedNatives(baseDir, prefixes) {
10299
}
103100
}
104101

105-
// --- Electron-UNIQUE: rebuild better-sqlite3 against the Electron ABI --------
102+
// --- Electron-UNIQUE: verify better-sqlite3 Node-API prebuilds ----------------
106103
//
107-
// The `npm ci` at the repo root compiles better-sqlite3 for the CI *Node* ABI
108-
// (e.g. 137 for Node 24). The packaged app runs its Next.js server via
109-
// ELECTRON_RUN_AS_NODE, so it needs the *Electron* ABI (146 for electron 42,
110-
// 148 for electron 43). We cannot rely on electron-builder's @electron/rebuild
111-
// here: it searches `electron/node_modules` (where better-sqlite3 does not live)
112-
// and, with the default prebuild path, tries to fetch a prebuilt binary — but
113-
// better-sqlite3@12.11.1 only ships prebuilds up to electron-v146, so electron
114-
// 43 (v148) silently gets no rebuild and the app dies with "Nenhum driver
115-
// SQLite disponível — better-sqlite3 (falhou)".
104+
// better-sqlite3 >= 13 ships Node-API (NAPI_VERSION=10) prebuilds for every
105+
// platform we package (darwin/linux/linuxmusl/win32 × x64/arm64) inside the
106+
// npm tarball. Node-API addons are ABI-independent, so the same prebuild runs
107+
// under plain Node (CI, CLI) and under the packaged app's ELECTRON_RUN_AS_NODE
108+
// server (verified against electron 43 / NODE_MODULE_VERSION 148 — issue
109+
// #10321 Stage 6). The historical source rebuild below existed because
110+
// better-sqlite3@12 only shipped prebuilds up to electron-v146 and electron 43
111+
// (v148) silently got no binary; v13 makes that obsolete.
116112
//
117-
// Instead we copy the *full* module (source + binding.gyp) from the root into
118-
// the standalone and compile it from source against the Electron headers, so
119-
// `bindings` finds a correct build/Release/better_sqlite3.node regardless of
120-
// prebuild availability. Robust to any current/future electron version.
121-
122-
function readElectronVersion() {
123-
const pkg = JSON.parse(readFileSync(join(ROOT, "electron", "package.json"), "utf8"));
124-
const raw = pkg.devDependencies?.electron || pkg.dependencies?.electron || "";
125-
return String(raw).replace(/^[\^~]/, "");
126-
}
113+
// Instead of compiling from source on every build (tens of seconds to minutes
114+
// per platform), we fail fast when the prebuild for the CURRENT build platform
115+
// is missing — a missing prebuild must kill the build here, not the app on a
116+
// user machine with "Nenhum driver SQLite disponível — better-sqlite3 (falhou)".
127117

128-
function rebuildBetterSqlite3ForElectron(standaloneNodeModules) {
129-
const srcMod = join(ROOT, "node_modules", "better-sqlite3");
130-
if (!existsSync(srcMod)) {
131-
console.warn("[electron] better-sqlite3 not found at repo root — skipping ABI rebuild.");
132-
return;
133-
}
134-
const electronVersion = readElectronVersion();
135-
if (!electronVersion) {
136-
throw new Error("[electron] could not resolve electron version for better-sqlite3 rebuild.");
137-
}
118+
function verifyBetterSqlite3Prebuilds(standaloneNodeModules) {
138119
const destMod = join(standaloneNodeModules, "better-sqlite3");
139-
// copyNatives only copies build/; we need the full module (src + binding.gyp)
140-
// to compile from source. Overwrite the copied Node-ABI build in the process.
141-
cpSync(srcMod, destMod, { recursive: true, force: true });
142-
rmSync(join(destMod, "build"), { recursive: true, force: true });
143-
144-
console.log(`[electron] rebuilding better-sqlite3 against electron ${electronVersion} ABI…`);
145-
const plan = buildRebuildSpawnPlan(process.platform);
146-
const result = spawnSync(
147-
plan.command,
148-
plan.args,
149-
{
150-
cwd: destMod,
151-
stdio: "inherit",
152-
// .cmd shims must go through a shell on Windows (CVE-2024-27980 hardening
153-
// makes a shell-less spawn fail with status null); args are fixed literals.
154-
shell: plan.shell,
155-
// Compile against the Electron headers (not Node's) so the .node lands in
156-
// build/Release with the Electron NODE_MODULE_VERSION. No shell interpolation.
157-
env: {
158-
...process.env,
159-
npm_config_runtime: "electron",
160-
npm_config_target: electronVersion,
161-
npm_config_disturl: "https://electronjs.org/headers",
162-
npm_config_arch: process.arch,
163-
npm_config_build_from_source: "true",
164-
},
165-
}
166-
);
167-
if (result.status !== 0) {
168-
throw new Error(
169-
`[electron] better-sqlite3 rebuild against electron ${electronVersion} failed (exit ${result.status}).`
170-
);
120+
if (!existsSync(destMod)) {
121+
console.warn("[electron] better-sqlite3 not found in standalone — skipping prebuild check.");
122+
return;
171123
}
172-
// Drop the now-unneeded compile inputs to keep the packaged app lean.
173-
for (const dir of ["deps", "src", "build/Debug", "build/obj.target"]) {
124+
125+
// Fail fast when the loader would find no prebuild for THIS build platform.
126+
// Mirrors better-sqlite3's own lib/binding.js selection logic.
127+
const reportHeader = process.report?.getReport?.().header;
128+
assertSqlitePrebuildExists(destMod, process.platform, process.arch, reportHeader);
129+
130+
// Drop compile inputs and stale Node-ABI build outputs to keep the packaged
131+
// app lean and to guarantee the loader resolves the prebuild, not a leftover
132+
// build/Release/better_sqlite3.node compiled for a different ABI.
133+
for (const dir of ["build", "deps", "src"]) {
174134
rmSync(join(destMod, dir), { recursive: true, force: true });
175135
}
136+
console.log(
137+
`[electron] better-sqlite3 Node-API prebuilds verified for ${process.platform}-${process.arch}.`
138+
);
176139
}
177140

178141
function logContextualError(error) {
@@ -217,12 +180,12 @@ if (docsPrune.removedFiles > 0) {
217180
// Electron-UNIQUE post-assembly steps
218181
removeGeneratedElectronArtifacts();
219182

220-
// Rebuild better-sqlite3 from source against the Electron ABI in the primary
221-
// node_modules (where the standalone server resolves it). keytar is still
222-
// stripped so electron-builder's @electron/rebuild handles it (it has electron
223-
// prebuilds); also drop any stray Node-ABI better-sqlite3 under .next/node_modules
224-
// so it cannot shadow the rebuilt one.
225-
rebuildBetterSqlite3ForElectron(join(ELECTRON_STANDALONE_DIR, "node_modules"));
183+
// Verify better-sqlite3 Node-API prebuilds in the primary node_modules (where
184+
// the standalone server resolves it). keytar is still stripped so
185+
// electron-builder's @electron/rebuild handles it (it has electron prebuilds);
186+
// also drop any stray better-sqlite3 under .next/node_modules so it cannot
187+
// shadow the prebuild-backed one.
188+
verifyBetterSqlite3Prebuilds(join(ELECTRON_STANDALONE_DIR, "node_modules"));
226189
removeNativeModules(join(ELECTRON_STANDALONE_DIR, "node_modules"), ["keytar"]);
227190
removeNativeModules(join(ELECTRON_STANDALONE_DIR, NEXT_DIST_DIR, "node_modules"), [
228191
"better-sqlite3",

tests/unit/electron-rebuild-spawn-win.test.ts

Lines changed: 0 additions & 22 deletions
This file was deleted.

0 commit comments

Comments
 (0)