You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 522eef7
Browse filesBrowse the repository at this point in the historyBrowse files
Measure the 2.6.0 performance pair at the last kernel change, the 2662nd (#2060)
The pair recorded on 2026-10-18 measured v2.5.0 beside the 2549th, which
was then the last change to the kernel; 113 commits came after it. Both
columns again, v2.5.0 (the 1957th) and 784f69d (the 2662nd), from one run
of benchmark_key_commits.sh on one machine with nothing else on it.
The v2.5.0 column reproduces the earlier run's to within 0.02% on every
gated row. Since the 2549th, SimplifyHard fell 10.4% more (-52.4% over the
release) and every other gated row moved less than 1%; Derivate is +5.65%
over the release, +0.1% of it after the 2549th.
Part of #746.
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Bytes allocated per call. `EvalEasy`, `RunEasy`, `RunMedium` and `RunHard` allocate nothing in both columns and are
221
-
left out.
220
+
left out.*Since the 2549th* compares with the 2549th column of the run of 2026-10-18.
222
221
223
222
-**The evaluation rows** are the numerical-evaluation work of the 2050th to the 2077th, recorded below.
224
223
-**`ParseHard`** is the 2203rd, the parser predicting in SLL first.
224
+
-**`SimplifyHard`** fell a further 10.4% after the 2549th. That is not attributed to any section of this file.
225
225
226
226
### The steps compose, and that was checked
227
227
@@ -231,12 +231,12 @@ left out.
231
231
- −41.4% and −20.2% at the 2054th to the 2075th;
232
232
- −42.1% at the 2077th.
233
233
234
-
Composed, they predict −94.9%, and the end-to-end measurement is −94.88%.
234
+
Composed, they predict −94.9%, and the end-to-end measurement is −94.89%.
235
235
236
-
`SimplifyHard`'s recorded steps predict about −44.5%, and the measurement is −46.88%. The 2.4 points between them
237
-
are not attributed to any section of this file.
236
+
`SimplifyHard`'s recorded steps predict about −44.5%. It measured −46.88% at the 2549th and −52.40% here, and neither
237
+
gap is attributed.
238
238
239
-
### `Derivate` went up 5.5%, in steps none of which the gate could see
239
+
### `Derivate` went up 5.65%, in steps none of which the gate could see
240
240
241
241
`Derivate` differentiates a parsed constant, so no parser change reaches it. Its gate baseline moved five times
242
242
across the release, each move within the gate's 3%:
@@ -249,7 +249,8 @@ across the release, each move within the gate's 3%:
249
249
| 2117th | 54,567 | +1.9% |
250
250
| 2130th | 55,095 | +1.0% |
251
251
| 2203rd | 55,783 | +1.2% |
252
-
| 2549th, measured here | 56,190 | +0.7% |
252
+
| 2549th, the run of 2026-10-18 | 56,190 | +0.7% |
253
+
| 2662nd, measured here | 56,247 | +0.1% |
253
254
254
255
A baseline is re-recorded by the pull request that moves it, so each step covers every commit since the step before it,
255
256
not one change. The largest step, +1,039 bytes, is the 2117th,
@@ -258,27 +259,28 @@ a cached list of their operands, and so makes each of those nodes larger. That i
258
259
**no step is bisected**, so the increase is recorded here as unattributed. This file has one other case of steps each free by the gate summing to a real cost, the
259
260
rule-set exchange's +13%. This is the same pattern at a smaller size, and it is why the pair is measured end to end.
260
261
261
-
`SolveEasyMedium` +1.67%, `ParseEasy` +0.88% and `SolveMedium` +0.71% are below the gate's resolution for a single step
262
-
and are not attributed either.
262
+
`SolveEasyMedium` +1.54%, `ParseEasy` +0.78%, `SolveMedium` +0.65% and `SimplifyEasy` +0.50% are below the gate's
263
+
resolution for a single step and are not attributed either.
263
264
264
265
### Timings
265
266
266
267
These moved well outside this machine's run-to-run spread and agree in sign with the allocation beside them:
267
268
268
269
| benchmark | change |
269
270
|---|--:|
270
-
|`EvalTranscendentalFresh`| −99.50% |
271
-
|`EvalPolynomialFresh`| −98.81% |
271
+
|`EvalTranscendentalFresh`| −99.51% |
272
+
|`EvalPolynomialFresh`| −98.83% |
273
+
|`ParseHard`| −98.54% |
272
274
|`EvalTrigPrecise`| −98.20% |
273
-
|`ParseHard`| −98.17% |
274
-
|`SolveEasy`| −95.81% |
275
-
|`EvalTrig`| −95.47% |
276
-
|`EvalPolynomialFresh15Digits`| −94.63% |
277
-
|`SimplifyHard`| −43.90% |
278
-
|`SolveHard`| −37.69% |
279
-
280
-
`Derivate` was +5.15% (13.7 → 14.5 µs), in line with its allocation. The smaller rows from the same run are not worth
281
-
reading and are not quoted.
275
+
|`SolveEasy`| −96.62% |
276
+
|`EvalTrig`| −95.53% |
277
+
|`EvalPolynomialFresh15Digits`| −94.85% |
278
+
|`SimplifyHard`| −56.99% |
279
+
|`SolveHard`| −39.96% |
280
+
281
+
`Derivate` was +1.66% (14.5 → 14.7 µs). `SimplifyEasy` was −14.9% and `ParseEasy` −25.9% with their allocation flat or
282
+
up. One run does not say whether that is the code or the machine, so they are not read as either. The smaller rows from the same run are not
283
+
worth reading and are not quoted.
282
284
283
285
## The 1845th and the 1954th, measured together on one machine — the pair for 2.5.0
0 commit comments