Repository navigation
chore(deps): upgrade Vitest to 5 and migrate benchmarks to the bench fixture - #8723
Merged
Merged
Conversation
…fixture Vitest 5 lets jsdom environments replace Node 25+'s native Web Storage globals (vitest-dev/vitest#10293), which Vitest 4 left in place on Node 26. - Bump vitest/@vitest/ui to ^5.0.4, @testing-library/jest-dom to ^7.0.1 and @codspeed/vitest-plugin to ^6.0.1 (Vitest 5 support). - Temporary jest-dom workaround until testing-library/jest-dom#742 ships: extend `expect` with the matchers directly and augment Vitest 5's `Matchers<R, T>`. `@testing-library/jest-dom/vitest` can bind a different Vitest copy under pnpm peer variants, which breaks `rejects.toThrow`. - Replace the removed `sequential` suite option and delete a fully commented-out typecheck file that Vitest 5 rejects as empty. - Rewrite all `bench()` files for the `bench` test fixture. CodSpeed suites turn the owning `describe` into a `test` with the same title, so every CodSpeed URI stays identical. Lifecycles move to per-bench hooks: suites that used suite hooks under CodSpeed keep fresh state per CodSpeed invocation, while Tinybench keeps one instance per warmup/run phase. - Bench functions are `async`: CodSpeed's Vitest 5 provider only awaits AsyncFunctions. - Run benchmarks with `--mode benchmark`, as Vitest 4 did, so vite-plugin-solid does not inject a jest-dom setup file, and give benchmark tests a 10 minute timeout for instrumented runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage |  |
Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 50 files. (223 skipped:… | Write docstrings for the functions missing them to satisfy the coverage threshold. |
-
Check name Status Explanation Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Title check The title clearly identifies the two main changes: upgrading Vitest to 5 and migrating benchmarks to the bench fixture. It is concise and specific. Description check The description is detailed and directly covers the motivation, dependency changes, benchmark migration, validation results, follow-ups, and AI authorship. It does not reproduce the template's exact C…
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 50 files. (223 skipped: 16 unsupported, 207 over the file limit.)
✨ Finishing Touches: 💡 2
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
🛠️ Fix failing CI checks: 💡
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
github-actions
Bot
added
package: react-router
package: router-core
package: valibot-adapter
package: zod-adapter
package: solid-router
package: arktype-adapter
package: solid-start-client
package: start-server-core
package: start-client-core
package: router-ssr-query-core
package: vue-router
labels
Contributor
|
View your CI Pipeline Execution ↗ for commit 7eead0c
☁️ Nx Cloud last updated this comment at |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Contributor
🚀 Changeset Version PreviewNo changeset entries found. Merging this PR will not cause a version bump for any packages. |
Contributor
Bundle Size BenchmarksThis pull request does not affect bundle size in any measured scenario. |
Merging this PR will degrade performance by 3.28%ℹ️ 3 benchmarks have no flame graph
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | mem server server-fn-churn (react) |
65.3 KB | 77.1 KB | -15.22% |
| ❌ | Simulation | ssr request loop (react) |
144.4 ms | 160.7 ms | -10.14% |
| ❌ | Memory | mem server error-paths redirect (react) |
10.8 KB | 12.1 KB | -10.11% |
| ❌ | Memory | mem client unique-location-churn (react) |
71.8 KB | 78.2 KB | -8.21% |
| ❌ | Simulation | ssr redirect (react) |
100.4 ms | 108.7 ms | -7.64% |
| ❌ | Simulation | ssr global-mw server-route 0 middleware (react) |
143.8 ms | 152.5 ms | -5.66% |
| ❌ | Simulation | ssr rewrite passthrough (react) |
154.9 ms | 163.9 ms | -5.5% |
| ❌ | Simulation | ssr server-route (react) |
109.5 ms | 115.8 ms | -5.48% |
| ❌ | Simulation | ssr global-mw server-route 1 middleware (react) |
156.8 ms | 165.5 ms | -5.25% |
| ❌ | Simulation | ssr global-mw server-route 3 middlewares (react) |
161.8 ms | 170.6 ms | -5.16% |
| ❌ | Simulation | ssr control-flow route headers (react) |
230.5 ms | 242.9 ms | -5.12% |
| ❌ | Simulation | ssr rewrite localized (react) |
162.3 ms | 171 ms | -5.09% |
| ❌ | Simulation | ssr global-mw server-route 2 middlewares (react) |
159.4 ms | 167.9 ms | -5.03% |
| ❌ | Simulation | ssr global-mw document (react) |
225.4 ms | 236.9 ms | -4.87% |
| ❌ | Simulation | ssr control-flow unmatched 404 (react) |
183.6 ms | 192.9 ms | -4.85% |
| ❌ | Simulation | ssr not-found (react) |
139.8 ms | 146.6 ms | -4.65% |
| ❌ | Simulation | ssr server-route middleware (react) |
102.6 ms | 107.6 ms | -4.65% |
| ❌ | Simulation | ssr server-fn during document ssr (react) |
170.4 ms | 178.7 ms | -4.63% |
| ❌ | Simulation | ssr control-flow error 500 (react) |
175.3 ms | 182.7 ms | -4.05% |
| ❌ | Simulation | ssr before-load chain (react) |
141.4 ms | 147.3 ms | -4.01% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing vitest-5 (7eead0c) with main (6777b8b)
Footnotes
-
3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Sheraff
requested changes
Oct 15, 2026
Vitest 5 already gives benchmark tests at least a 60 s timeout. In this PR's CodSpeed CI run, the slowest of 194 instrumented benchmark tests took 28.5 s (p95 17.3 s, median 7.7 s), and plain Tinybench runs stay around 10-15 s, so the default floor covers every suite and still catches a hung benchmark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sheraff
approved these changes
Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
On Node 25+/26, Vitest 4's jsdom environment keeps Node's native
localStorage/sessionStorageglobals whileStorageis jsdom's (vitest-dev/vitest#10867). That failsrouter-core'sscroll-restoration-lifecycle.test.tson Node 26 and forced a--no-experimental-webstorageworkaround in #8509. Vitest fixed this only in v5 (vitest-dev/vitest#10293). With this PR, the whole unit/type suite passes on Node 24.8.0 and 26.10.0 without that flag.Changes
Dependencies
vitest/@vitest/ui→^5.0.4(catalog and the 4 examples that pin it)@testing-library/jest-dom→^7.0.1. 7.x declaresvitestas a peer.@codspeed/vitest-plugin→^6.0.1, the first version with Vitest 5 support. This supersedes the plugin bump in bump node version to 26.10.0 #8509.Temporary jest-dom workaround: remove after testing-library/jest-dom#742 is merged and released
Each package that uses jest-dom gets a
jest-dom.setup.tsthat:expect.extend(matchers)from@testing-library/jest-dom/matchers.@testing-library/jest-dom/vitestimportsvitestfrom its own location, which under pnpm peer variants can be a differentvitest@5.0.4copy. Vitest 5 bundlesexpect, so extending that copy brokerejects.toThrowin react-router.Matchers<R, T>. jest-dom 7.0.1 still augments Vitest 4'sAssertion<T>, so every matcher fails typecheck.jest-domin its file name. vite-plugin-solid skips injecting@testing-library/jest-dom/vitestonly when a setup path contains it.Other test-side migration
{ sequential: true }→{ concurrent: false }(option removed in v5).solid-start-client/src/tests/createServerFn.test-d.tsx. It was fully commented out, and Vitest 5 fails empty suites.Benchmarks (bench API rewrite)
bench()files now use thebenchtest fixture.describethat owned the benches becomes atestwith the same title, so the URIs stayfile::<title>::<bench name>. I diffed all URIs from the client-nav (react/solid/vue), ssr, memory-client and memory-server React suites against the Vitest 4 runner: 94/94 identical.benchLifecycle()helper inbenchmarks/cpu-simulation.tskeepsmain's behavior. CodSpeed simulation/memory runs get a fresh app per invocation. Plain Tinybench runs keep one app per warmup/run phase (otherwise a single scenario took 268 s instead of ~11 s).memoryBenchHooks(), which yields the turn and then prepares state per invocation.beforeEach/afterEach.async. CodSpeed's v5 provider awaits onlyAsyncFunctions; a plain arrow returning a promise would be measured only up to its firstawait. SSR arrows and memory-serverrunfunctions are nowasync.--mode benchmarkon every bench command. Vitest 4 ran benches in that Vite mode. Under v5's defaulttestmode, vite-plugin-solid injects a jest-dom setup file that Solid benchmark packages cannot resolve.benchmark.includeto their ownspeed.bench.ts. Vitest 5's default glob otherwise collected every framework's benches.--outputJson→--reporter=json --outputFile, and lifecycle notes in the client-nav/memory READMEs andbenchmarks/AGENTS.md.Validation
test:unit+test:typesfor all 46 workspace projects (excluding examples/e2e): pass on Node 24.8.0 and Node 26.10.0 (Nx cache skipped)test:eslint: passvitest bench:-tfilter): passroute-tree-construction(--testNamePattern): passsearch-validator-adaptersexamples: tests and typecheck passCodSpeed numbers should be unchanged except for the
asyncwrapper on SSR and memory-server bench functions, which adds one promise adoption per invocation.Follow-ups
--no-experimental-webstorageworkaround once this lands.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Chores