Skip to content

Commit 49de0a2

Browse files
jphilleojphilleo
committed
Simplify benchmark leaderboard columns
1 parent 30abedd commit 49de0a2

2 files changed

Lines changed: 42 additions & 42 deletions

File tree

‎docs/benchmark/CCERL_LEADERBOARD.md‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -26,27 +26,27 @@ Protocol:
2626
- anchor: `random = 0`
2727
- latest promoted training checkpoint included: `JumpStar_82`
2828

29-
| Rank | Engine | Elo | CI95 | Games | W-D-L | Score | vs JumpStar_60 | vs JumpStar_82 |
30-
|---:|---|---:|---:|---:|---:|---:|---:|---:|
31-
| 1 | `JumpStar_60` | 911 | 874 - 947 | 3,096 | 2241-371-484 | 0.784 | - | 52-48-72 / 0.442 |
32-
| 2 | `JumpStar_82` | 900 | 864 - 937 | 3,096 | 2129-539-428 | 0.775 | 72-48-52 / 0.558 | - |
33-
| 3 | `JumpStar_63` | 891 | 855 - 927 | 3,096 | 2134-478-484 | 0.766 | 51-55-66 / 0.456 | 48-75-49 / 0.497 |
34-
| 4 | `JumpStar_64` | 888 | 852 - 925 | 3,096 | 2161-409-526 | 0.764 | 94-19-59 / 0.602 | 69-27-76 / 0.480 |
35-
| 5 | `JumpStar_61` | 879 | 843 - 916 | 3,096 | 2119-442-535 | 0.756 | 69-26-77 / 0.477 | 45-78-49 / 0.488 |
36-
| 6 | `JumpStar_62` | 873 | 837 - 909 | 3,096 | 2075-491-530 | 0.750 | 49-44-79 / 0.413 | 79-44-49 / 0.587 |
37-
| 7 | `JumpStar_57` | 813 | 778 - 849 | 3,096 | 1981-324-791 | 0.692 | 41-22-109 / 0.302 | 34-24-114 / 0.267 |
38-
| 8 | `tt-pvs` | 737 | 701 - 772 | 3,096 | 1805-189-1102 | 0.614 | 38-13-121 / 0.259 | 32-16-124 / 0.233 |
39-
| 9 | `JumpStar_46` | 684 | 649 - 720 | 3,096 | 1562-333-1201 | 0.558 | 22-10-140 / 0.157 | 19-37-116 / 0.218 |
40-
| 10 | `JumpStar_29` | 654 | 619 - 689 | 3,096 | 1478-301-1317 | 0.526 | 13-2-157 / 0.081 | 8-15-149 / 0.090 |
41-
| 11 | `JumpStar_9` | 516 | 482 - 551 | 3,096 | 1039-285-1772 | 0.382 | 9-4-159 / 0.064 | 8-30-134 / 0.134 |
42-
| 12 | `converter` | 511 | 476 - 546 | 3,096 | 1120-88-1888 | 0.376 | 8-0-164 / 0.047 | 6-4-162 / 0.047 |
43-
| 13 | `JumpStar_5` | 484 | 449 - 518 | 3,096 | 961-237-1898 | 0.349 | 3-2-167 / 0.023 | 6-3-163 / 0.044 |
44-
| 14 | `greedy` | 481 | 446 - 516 | 3,096 | 1039-67-1990 | 0.346 | 5-3-164 / 0.038 | 5-2-165 / 0.035 |
45-
| 15 | `marblefish-ab` | 443 | 408 - 478 | 3,096 | 727-460-1909 | 0.309 | 4-7-161 / 0.044 | 7-15-150 / 0.084 |
46-
| 16 | `harryz-rule` | 379 | 344 - 414 | 3,096 | 5-1547-1544 | 0.251 | 0-99-73 / 0.288 | 0-99-73 / 0.288 |
47-
| 17 | `zedrichu-minimax` | 370 | 335 - 405 | 3,096 | 407-694-1995 | 0.244 | 5-7-160 / 0.049 | 9-7-156 / 0.073 |
48-
| 18 | `traffic-greedy` | 340 | 305 - 375 | 3,096 | 607-141-2348 | 0.219 | 1-3-168 / 0.015 | 1-9-162 / 0.032 |
49-
| 19 | `random` | 0 | 0 - 0 | 3,096 | 0-248-2848 | 0.040 | 0-7-165 / 0.020 | 0-6-166 / 0.017 |
29+
| Rank | Engine | Elo | CI95 | Games | W-D-L | Score |
30+
|---:|---|---:|---:|---:|---:|---:|
31+
| 1 | `JumpStar_60` | 911 | 874 - 947 | 3,096 | 2241-371-484 | 0.784 |
32+
| 2 | `JumpStar_82` | 900 | 864 - 937 | 3,096 | 2129-539-428 | 0.775 |
33+
| 3 | `JumpStar_63` | 891 | 855 - 927 | 3,096 | 2134-478-484 | 0.766 |
34+
| 4 | `JumpStar_64` | 888 | 852 - 925 | 3,096 | 2161-409-526 | 0.764 |
35+
| 5 | `JumpStar_61` | 879 | 843 - 916 | 3,096 | 2119-442-535 | 0.756 |
36+
| 6 | `JumpStar_62` | 873 | 837 - 909 | 3,096 | 2075-491-530 | 0.750 |
37+
| 7 | `JumpStar_57` | 813 | 778 - 849 | 3,096 | 1981-324-791 | 0.692 |
38+
| 8 | `tt-pvs` | 737 | 701 - 772 | 3,096 | 1805-189-1102 | 0.614 |
39+
| 9 | `JumpStar_46` | 684 | 649 - 720 | 3,096 | 1562-333-1201 | 0.558 |
40+
| 10 | `JumpStar_29` | 654 | 619 - 689 | 3,096 | 1478-301-1317 | 0.526 |
41+
| 11 | `JumpStar_9` | 516 | 482 - 551 | 3,096 | 1039-285-1772 | 0.382 |
42+
| 12 | `converter` | 511 | 476 - 546 | 3,096 | 1120-88-1888 | 0.376 |
43+
| 13 | `JumpStar_5` | 484 | 449 - 518 | 3,096 | 961-237-1898 | 0.349 |
44+
| 14 | `greedy` | 481 | 446 - 516 | 3,096 | 1039-67-1990 | 0.346 |
45+
| 15 | `marblefish-ab` | 443 | 408 - 478 | 3,096 | 727-460-1909 | 0.309 |
46+
| 16 | `harryz-rule` | 379 | 344 - 414 | 3,096 | 5-1547-1544 | 0.251 |
47+
| 17 | `zedrichu-minimax` | 370 | 335 - 405 | 3,096 | 407-694-1995 | 0.244 |
48+
| 18 | `traffic-greedy` | 340 | 305 - 375 | 3,096 | 607-141-2348 | 0.219 |
49+
| 19 | `random` | 0 | 0 - 0 | 3,096 | 0-248-2848 | 0.040 |
5050

5151
## Champion Direct Match
5252

‎docs/benchmark/PUBLIC_BENCHMARK.md‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -72,27 +72,27 @@ experiments/public_benchmark/ladder_v2_latest_champions_s64/baseline_ladder.json
7272

7373
It uses all 86 audited v2 positions, paired color swaps, all 171 displayed-engine pairs, 29,412 games total, 64-simulation neural search, the existing native/CCP logs plus latest-champion incremental runs, and `random = 0`. Open-source adapter rows are fitted in the same all-pairs graph as every other displayed engine.
7474

75-
| Rank | Baseline | Elo | Games | Score | vs JumpStar_60 | vs JumpStar_82 |
76-
|---:|---|---:|---:|---:|---:|---:|
77-
| 1 | `JumpStar_60` | 911 | 3,096 | 0.784 | - | 52-48-72 / 0.442 |
78-
| 2 | `JumpStar_82` | 900 | 3,096 | 0.775 | 72-48-52 / 0.558 | - |
79-
| 3 | `JumpStar_63` | 891 | 3,096 | 0.766 | 51-55-66 / 0.456 | 48-75-49 / 0.497 |
80-
| 4 | `JumpStar_64` | 888 | 3,096 | 0.764 | 94-19-59 / 0.602 | 69-27-76 / 0.480 |
81-
| 5 | `JumpStar_61` | 879 | 3,096 | 0.756 | 69-26-77 / 0.477 | 45-78-49 / 0.488 |
82-
| 6 | `JumpStar_62` | 873 | 3,096 | 0.750 | 49-44-79 / 0.413 | 79-44-49 / 0.587 |
83-
| 7 | `JumpStar_57` | 813 | 3,096 | 0.692 | 41-22-109 / 0.302 | 34-24-114 / 0.267 |
84-
| 8 | `tt-pvs` | 737 | 3,096 | 0.614 | 38-13-121 / 0.259 | 32-16-124 / 0.233 |
85-
| 9 | `JumpStar_46` | 684 | 3,096 | 0.558 | 22-10-140 / 0.157 | 19-37-116 / 0.218 |
86-
| 10 | `JumpStar_29` | 654 | 3,096 | 0.526 | 13-2-157 / 0.081 | 8-15-149 / 0.090 |
87-
| 11 | `JumpStar_9` | 516 | 3,096 | 0.382 | 9-4-159 / 0.064 | 8-30-134 / 0.134 |
88-
| 12 | `converter` | 511 | 3,096 | 0.376 | 8-0-164 / 0.047 | 6-4-162 / 0.047 |
89-
| 13 | `JumpStar_5` | 484 | 3,096 | 0.349 | 3-2-167 / 0.023 | 6-3-163 / 0.044 |
90-
| 14 | `greedy` | 481 | 3,096 | 0.346 | 5-3-164 / 0.038 | 5-2-165 / 0.035 |
91-
| 15 | `marblefish-ab` | 443 | 3,096 | 0.309 | 4-7-161 / 0.044 | 7-15-150 / 0.084 |
92-
| 16 | `harryz-rule` | 379 | 3,096 | 0.251 | 0-99-73 / 0.288 | 0-99-73 / 0.288 |
93-
| 17 | `zedrichu-minimax` | 370 | 3,096 | 0.244 | 5-7-160 / 0.049 | 9-7-156 / 0.073 |
94-
| 18 | `traffic-greedy` | 340 | 3,096 | 0.219 | 1-3-168 / 0.015 | 1-9-162 / 0.032 |
95-
| 19 | `random` | 0 | 3,096 | 0.040 | 0-7-165 / 0.020 | 0-6-166 / 0.017 |
75+
| Rank | Baseline | Elo | Games | Score |
76+
|---:|---|---:|---:|---:|
77+
| 1 | `JumpStar_60` | 911 | 3,096 | 0.784 |
78+
| 2 | `JumpStar_82` | 900 | 3,096 | 0.775 |
79+
| 3 | `JumpStar_63` | 891 | 3,096 | 0.766 |
80+
| 4 | `JumpStar_64` | 888 | 3,096 | 0.764 |
81+
| 5 | `JumpStar_61` | 879 | 3,096 | 0.756 |
82+
| 6 | `JumpStar_62` | 873 | 3,096 | 0.750 |
83+
| 7 | `JumpStar_57` | 813 | 3,096 | 0.692 |
84+
| 8 | `tt-pvs` | 737 | 3,096 | 0.614 |
85+
| 9 | `JumpStar_46` | 684 | 3,096 | 0.558 |
86+
| 10 | `JumpStar_29` | 654 | 3,096 | 0.526 |
87+
| 11 | `JumpStar_9` | 516 | 3,096 | 0.382 |
88+
| 12 | `converter` | 511 | 3,096 | 0.376 |
89+
| 13 | `JumpStar_5` | 484 | 3,096 | 0.349 |
90+
| 14 | `greedy` | 481 | 3,096 | 0.346 |
91+
| 15 | `marblefish-ab` | 443 | 3,096 | 0.309 |
92+
| 16 | `harryz-rule` | 379 | 3,096 | 0.251 |
93+
| 17 | `zedrichu-minimax` | 370 | 3,096 | 0.244 |
94+
| 18 | `traffic-greedy` | 340 | 3,096 | 0.219 |
95+
| 19 | `random` | 0 | 3,096 | 0.040 |
9696

9797
Because top JumpStar checkpoints can be close in broad ladders, the champion
9898
call uses a promotion-grade direct match:

0 commit comments

Comments
 (0)