Skip to content

Commit ea9680b

Browse files
committed
Count the build's allocations on generated projects in the benchmarks
Why: The benchmarks compile runtime and Belt files one at a time, whose dependency graphs are small, and build tests/tests, whose modules use a few small modules. Costs that grow with how many modules a build has or loads barely show: a change that avoids copying the signatures of large dependencies that a module hardly reads moved them by 0.4%, and opening the map of a namespaced package, which is quadratic in its modules per build, doesn't run at all. With `--project`, `measure` now also builds two projects that the script generates, clean and after an edit, on one worker domain and counting allocated words like tests/tests: - WideDeps: 240 modules that each use one or two values of four modules with 600 types and 1,200 values each. The edit adds a value to one of them, which compiles its users again. - Namespace: a package with `"namespace": true` and 1,000 one-line modules. The edit changes a module that nothing uses. Their counts are larger than all the others together, so the report adds them up on a line of their own, `Generated projects`, and `In total` covers the other benchmarks as before. Each has its row and threshold like any other benchmark. Every module of a project build loads the runtime's interfaces, so the inputs of the generated projects and of tests/tests now include a hash of the runtime sources: a pull request that changes the runtime marks them as inputs changed rather than as regressions. Locally, the signature copy pull requests take WideDeps clean from 505M to 333M words and its edit from 232M to 146M, and building the namespace map on demand takes Namespace clean from 611M to 203M; the other benchmarks' counts are unchanged. The builds of tests/tests now share countBuild and editModule with the generated projects. Signed-off-by: Jono Prest <jjprest@gmail.com>
1 parent 70da3d1 commit ea9680b

4 files changed

Lines changed: 347 additions & 31 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -455,7 +455,8 @@ jobs:
455455
# to 0.2%. The runtime and Belt are built there too, since the
456456
# replayed builds would otherwise write interfaces that don't
457457
# match those of the files they don't replay yet. Both build the
458-
# base's tests/tests, which the base's compiler builds already.
458+
# base's tests/tests, which the base's compiler builds already,
459+
# and the generated projects of this checkout's script.
459460
measure() {
460461
sudo mkdir -p /bench
461462
sudo mount --bind "$1" /bench
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- Count the words the build allocates on generated projects in the CI benchmarks: many modules using little of a few large dependencies, and a namespaced package of 1,000 modules.

0 commit comments

Comments
 (0)