Skip to content

Commit 522eef7

Browse files
Measure the 2.6.0 performance pair at the last kernel change, the 2662nd (#2060)
The pair recorded on 2026-10-18 measured v2.5.0 beside the 2549th, which was then the last change to the kernel; 113 commits came after it. Both columns again, v2.5.0 (the 1957th) and 784f69d (the 2662nd), from one run of benchmark_key_commits.sh on one machine with nothing else on it. The v2.5.0 column reproduces the earlier run's to within 0.02% on every gated row. Since the 2549th, SimplifyHard fell 10.4% more (-52.4% over the release) and every other gated row moved less than 1%; Derivate is +5.65% over the release, +0.1% of it after the 2549th. Part of #746. Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent 784f69d commit 522eef7

1 file changed

Lines changed: 49 additions & 47 deletions

File tree

‎Sources/AngouriMath/Docs/WhatsNew/version_performance_control.md‎

Lines changed: 49 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -180,48 +180,48 @@ honest reason -- a compiled delegate over `Complex` has nothing to put on the he
180180

181181
---
182182

183-
## The 1957th and the 2549th, measured together on one machine — the pair for 2.6.0
183+
## The 1957th and the 2662nd, measured together on one machine — the pair for 2.6.0
184184

185185
The pair the release checklist owes: `v2.5.0`
186186
([`ab7a8bd6`](https://github.com/asc-community/AngouriMath/commit/ab7a8bd6b92d9ae99b3e23e51b10dca5c0e8fe11), the 1957th)
187187
re-measured beside
188-
[`001b1caa`](https://github.com/asc-community/AngouriMath/commit/001b1caa694f1dd8476c3a54d93d10d7a7fcfe4d), the
189-
2549th, which is the last change to the kernel before the release. Both columns come from one run of
190-
`Sources/Utils/benchmark_key_commits.sh`, on 2026-10-18, on one machine with nothing else on it. The period is the
191-
whole of 2.6.0, 592 commits.
188+
[`784f69d5`](https://github.com/asc-community/AngouriMath/commit/784f69d5b07d2f0c153c67791468b3754f2e3250), the 2662nd, which is the last change to
189+
the kernel before the release. Both columns come from one run of `Sources/Utils/benchmark_key_commits.sh`, on
190+
2026-10-22, on one machine with nothing else on it. The period is the whole of 2.6.0, 705 commits.
192191

193-
The `v2.5.0` column agrees with the 1954th column of the 2.5.0 pair, measured on 2026-09-09 three commits earlier.
194-
Every row the gate measures agrees to within 0.02%, and five agree to the byte, `SimplifyEasy` among them.
195-
`CompileEasy` and `CompileHard` differ by 0.30% and 0.19%. The gate leaves those two ungated because their
196-
allocation includes building a delegate.
192+
It replaces the pair measured on 2026-10-18 against the 2549th, which was then the last change to the kernel; 113
193+
commits came after it. The `v2.5.0` column agrees with that run's on every row the gate measures, to within 0.02%, and
194+
`Derivate` and `SimplifyEasy` agree to the byte. `CompileEasy` reads 11,003 here and 10,970 there, 0.30% apart. The
195+
gate leaves it and `CompileHard` ungated because their allocation includes building a delegate.
197196

198197
### Allocation
199198

200-
| benchmark | v2.5.0 | 2549th | change |
201-
|---|--:|--:|--:|
202-
| `CompileEasy` | 10,970 | 11,003 | +0.30% |
203-
| `CompileHard` | 20,434 | 20,434 | 0.00% |
204-
| `Derivate` | 53,239 | 56,190 | **+5.54%** |
205-
| `EvalPolynomialFresh` | 650,888 | 12,536 | **−98.07%** |
206-
| `EvalPolynomialFresh15Digits` | 265,816 | 12,064 | **−95.46%** |
207-
| `EvalTranscendentalFresh` | 5,668,395 | 34,760 | **−99.39%** |
208-
| `EvalTrig` | 1,341,457 | 69,944 | **−94.79%** |
209-
| `EvalTrigPrecise` | 12,742,285 | 336,552 | **−97.36%** |
210-
| `ParseEasy` | 18,261 | 18,421 | +0.88% |
211-
| `ParseHard` | 3,584,817 | 60,103 | **−98.32%** |
212-
| `SimplifyEasy` | 79,282 | 78,978 | −0.38% |
213-
| `SimplifyHard` | 331,710,136 | 176,205,784 | **−46.88%** |
214-
| `SolveEasy` | 8,865,872 | 454,021 | **−94.88%** |
215-
| `SolveEasyMedium` | 65,648 | 66,744 | **+1.67%** |
216-
| `SolveHard` | 11,884,272 | 10,001,832 | **−15.84%** |
217-
| `SolveMedium` | 455,893 | 459,134 | +0.71% |
218-
| `SolveMediumHard` | 1,449,394 | 1,411,524 | −2.61% |
199+
| benchmark | v2.5.0 | 2662nd | change | since the 2549th |
200+
|---|--:|--:|--:|--:|
201+
| `CompileEasy` | 11,003 | 11,003 | 0.00% | 0.00% |
202+
| `CompileHard` | 20,434 | 20,737 | +1.48% | +1.48% |
203+
| `Derivate` | 53,239 | 56,247 | **+5.65%** | +0.10% |
204+
| `EvalPolynomialFresh` | 650,888 | 12,536 | **−98.07%** | 0.00% |
205+
| `EvalPolynomialFresh15Digits` | 265,816 | 12,064 | **−95.46%** | 0.00% |
206+
| `EvalTranscendentalFresh` | 5,668,395 | 34,760 | **−99.39%** | 0.00% |
207+
| `EvalTrig` | 1,341,457 | 69,944 | **−94.79%** | 0.00% |
208+
| `EvalTrigPrecise` | 12,742,285 | 336,552 | **−97.36%** | 0.00% |
209+
| `ParseEasy` | 18,258 | 18,400 | +0.78% | −0.11% |
210+
| `ParseHard` | 3,584,841 | 60,023 | **−98.33%** | −0.13% |
211+
| `SimplifyEasy` | 79,282 | 79,682 | +0.50% | +0.89% |
212+
| `SimplifyHard` | 331,710,008 | 157,898,664 | **−52.40%** | **−10.39%** |
213+
| `SolveEasy` | 8,865,936 | 453,396 | **−94.89%** | −0.14% |
214+
| `SolveEasyMedium` | 65,650 | 66,664 | **+1.54%** | −0.12% |
215+
| `SolveHard` | 11,884,016 | 9,998,784 | **−15.86%** | −0.03% |
216+
| `SolveMedium` | 455,925 | 458,910 | +0.65% | −0.05% |
217+
| `SolveMediumHard` | 1,449,394 | 1,412,412 | −2.55% | +0.06% |
219218

220219
Bytes allocated per call. `EvalEasy`, `RunEasy`, `RunMedium` and `RunHard` allocate nothing in both columns and are
221-
left out.
220+
left out. *Since the 2549th* compares with the 2549th column of the run of 2026-10-18.
222221

223222
- **The evaluation rows** are the numerical-evaluation work of the 2050th to the 2077th, recorded below.
224223
- **`ParseHard`** is the 2203rd, the parser predicting in SLL first.
224+
- **`SimplifyHard`** fell a further 10.4% after the 2549th. That is not attributed to any section of this file.
225225

226226
### The steps compose, and that was checked
227227

@@ -231,12 +231,12 @@ left out.
231231
- −41.4% and −20.2% at the 2054th to the 2075th;
232232
- −42.1% at the 2077th.
233233

234-
Composed, they predict −94.9%, and the end-to-end measurement is −94.88%.
234+
Composed, they predict −94.9%, and the end-to-end measurement is −94.89%.
235235

236-
`SimplifyHard`'s recorded steps predict about −44.5%, and the measurement is −46.88%. The 2.4 points between them
237-
are not attributed to any section of this file.
236+
`SimplifyHard`'s recorded steps predict about −44.5%. It measured −46.88% at the 2549th and −52.40% here, and neither
237+
gap is attributed.
238238

239-
### `Derivate` went up 5.5%, in steps none of which the gate could see
239+
### `Derivate` went up 5.65%, in steps none of which the gate could see
240240

241241
`Derivate` differentiates a parsed constant, so no parser change reaches it. Its gate baseline moved five times
242242
across the release, each move within the gate's 3%:
@@ -249,7 +249,8 @@ across the release, each move within the gate's 3%:
249249
| 2117th | 54,567 | +1.9% |
250250
| 2130th | 55,095 | +1.0% |
251251
| 2203rd | 55,783 | +1.2% |
252-
| 2549th, measured here | 56,190 | +0.7% |
252+
| 2549th, the run of 2026-10-18 | 56,190 | +0.7% |
253+
| 2662nd, measured here | 56,247 | +0.1% |
253254

254255
A baseline is re-recorded by the pull request that moves it, so each step covers every commit since the step before it,
255256
not one change. The largest step, +1,039 bytes, is the 2117th,
@@ -258,27 +259,28 @@ a cached list of their operands, and so makes each of those nodes larger. That i
258259
**no step is bisected**, so the increase is recorded here as unattributed. This file has one other case of steps each free by the gate summing to a real cost, the
259260
rule-set exchange's +13%. This is the same pattern at a smaller size, and it is why the pair is measured end to end.
260261

261-
`SolveEasyMedium` +1.67%, `ParseEasy` +0.88% and `SolveMedium` +0.71% are below the gate's resolution for a single step
262-
and are not attributed either.
262+
`SolveEasyMedium` +1.54%, `ParseEasy` +0.78%, `SolveMedium` +0.65% and `SimplifyEasy` +0.50% are below the gate's
263+
resolution for a single step and are not attributed either.
263264

264265
### Timings
265266

266267
These moved well outside this machine's run-to-run spread and agree in sign with the allocation beside them:
267268

268269
| benchmark | change |
269270
|---|--:|
270-
| `EvalTranscendentalFresh` | −99.50% |
271-
| `EvalPolynomialFresh` | −98.81% |
271+
| `EvalTranscendentalFresh` | −99.51% |
272+
| `EvalPolynomialFresh` | −98.83% |
273+
| `ParseHard` | −98.54% |
272274
| `EvalTrigPrecise` | −98.20% |
273-
| `ParseHard` | −98.17% |
274-
| `SolveEasy` | −95.81% |
275-
| `EvalTrig` | −95.47% |
276-
| `EvalPolynomialFresh15Digits` | −94.63% |
277-
| `SimplifyHard` | −43.90% |
278-
| `SolveHard` | −37.69% |
279-
280-
`Derivate` was +5.15% (13.7 → 14.5 µs), in line with its allocation. The smaller rows from the same run are not worth
281-
reading and are not quoted.
275+
| `SolveEasy` | −96.62% |
276+
| `EvalTrig` | −95.53% |
277+
| `EvalPolynomialFresh15Digits` | −94.85% |
278+
| `SimplifyHard` | −56.99% |
279+
| `SolveHard` | −39.96% |
280+
281+
`Derivate` was +1.66% (14.5 → 14.7 µs). `SimplifyEasy` was −14.9% and `ParseEasy` −25.9% with their allocation flat or
282+
up. One run does not say whether that is the code or the machine, so they are not read as either. The smaller rows from the same run are not
283+
worth reading and are not quoted.
282284

283285
## The 1845th and the 1954th, measured together on one machine — the pair for 2.5.0
284286

0 commit comments

Comments
 (0)