bench: compare pnpm engines on Next.js

[zkochan]

Summary

Adds a pnpm-only benchmark fixture based on vercel/next.js at commit 31b78bb, the workload from pnpm/pnpm#13305.

The runner sparsely checks out the pinned workspace manifests and patches, migrates pnpm settings to pnpm-workspace.yaml, and compares the TypeScript and Rust pnpm engines across the existing install scenarios. Dynamic fixture paths are propagated through the benchmark recorder so the generated fixture is used by both engines.

A local cold frozen-install check reproduced the inversion: TypeScript pnpm completed in 11.30s and Rust pnpm in 13.84s.

Validation

• JavaScript syntax checks passed.
• Sparse fixture generation produced all 831 package manifests.
• Migrated workspace settings were verified.
• The generated fixture reproduced the reported cold-install performance inversion.

---

Written by an agent (Codex, GPT-5).

[bolt-new-by-stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
pnpm-io	Error		Jul 27, 2026 1:23pm

[coderabbitai]

Warning

Review limit reached

@zkochan, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 58 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5c9e4755-45d1-49ab-b1af-5e205b918eb6

📥 Commits

Reviewing files that changed from the base of the PR and between 91d0f3a and b53a999.

📒 Files selected for processing (4)

• benchmarks/benchmarkFixture.js
• benchmarks/checkoutNextFixture.js
• benchmarks/index.js
• benchmarks/recordBenchmark.js

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch benchmark-nextjs-pnpm-engines

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[qodo-code-review]

PR Summary by Qodo

bench: compare pnpm engines on Next.js

✨ Enhancement 🕐 40+ Minutes

AI Description

• Add a pnpm-only Next.js monorepo benchmark fixture pinned to a specific upstream commit.
• Generate the fixture via sparse git checkout and migrate pnpm settings into pnpm-workspace.yaml.
• Thread dynamic fixture paths through the benchmark runner/recorder to compare pnpm engines.

Diagram

graph TD
  A["benchmarks/index.js (runner)"] --> B["checkoutNextFixture.js"] --> C[("Next.js fixture dir")] --> D["recordBenchmark.js"] --> E["benchmarkFixture.js"]
  D --> H[("results/*.yaml")]
  E --> F["pnpm11 (TS engine)"]
  E --> G["pnpm12 (Rust engine)"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Check in a frozen fixture snapshot to the repo

• ➕ No network/git required to run benchmarks
• ➕ Fully reproducible fixture contents without relying on upstream availability
• ➖ Large repo bloat (hundreds of manifests/patches)
• ➖ Ongoing maintenance burden when updating the pinned upstream commit

2. Download a published fixture tarball (with checksum)

• ➕ Small repo footprint while avoiding git operations
• ➕ Can be cached aggressively in CI; easy integrity verification
• ➖ Requires hosting/release process for the artifact
• ➖ Extra tooling to build/publish the tarball when updating the fixture

3. Use a full shallow clone instead of sparse-checkout

• ➕ Simpler git logic; fewer sparse-checkout edge cases
• ➖ Much larger download and slower fixture generation
• ➖ Higher disk usage and more CI variability

Recommendation: The PR’s sparse-checkout approach is a good balance of reproducibility and size, since it pins to an exact commit while avoiding a full Next.js checkout. If benchmark runs are expected to be frequent/CI-driven, consider the “tarball + checksum” approach later to reduce dependence on git/network while keeping the repo small.

Files changed (4) +126 / -47

Enhancement (4) +126 / -47

benchmarkFixture.jsAllow benchmarks to copy from a dynamic fixture directory +1/-2

Allow benchmarks to copy from a dynamic fixture directory

• Adds support for an optional 'fixtureDir' override when copying fixtures into the per-scenario temp directory. This lets generated fixtures (like Next.js) flow through the existing benchmark execution logic without being checked into 'benchmarks/fixtures/'.

benchmarks/benchmarkFixture.js

checkoutNextFixture.jsAdd Next.js sparse-checkout fixture generator with pnpm settings migration +65/-0

Add Next.js sparse-checkout fixture generator with pnpm settings migration

• Introduces a helper that initializes a git repo, performs a sparse checkout of workspace manifests/patches at a pinned Next.js commit, removes git metadata, prunes empty directories, and migrates 'package.json.pnpm' settings into 'pnpm-workspace.yaml'. This produces a pnpm-focused benchmark fixture matching the referenced workload.

benchmarks/checkoutNextFixture.js

index.jsAdd pnpm-only Next.js fixture and integrate generated fixture into runs +59/-45

Add pnpm-only Next.js fixture and integrate generated fixture into runs

• Registers a new 'nextjs' fixture (pnpm-only) and generates it into a temp directory before running benchmarks. Adjusts the per-fixture PM matrix (pnpm-only vs full), threads the generated fixture path into benchmark runs, and skips the main multi-PM chart for pnpm-only fixtures while keeping the pnpm comparison output.

benchmarks/index.js

recordBenchmark.jsPropagate fixtureDir through the recorder into benchmark execution +1/-0

Propagate fixtureDir through the recorder into benchmark execution

• Threads an optional 'fixtureDir' option from the recording layer into the benchmark executor. This enables per-run fixture source selection without changing results persistence layout.

benchmarks/recordBenchmark.js