Skip to content

Commit a0ef01c

Browse files
petercorkeclaude
andauthored
chore(examples): rename benchmark scripts, add wiki pages (#692)
* chore(examples): rename ik_speed.py/rne_speed.py to benchmark_ik.py/benchmark_rne.py Clearer, more discoverable names for the two benchmark scripts, and update their cross-references in _cpu_info.py's shared-helper docstring and rne_compare.py. Companion wiki pages (Benchmark-IK, Benchmark-RNE) document where these live and how to run them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * chore(examples): size benchmark_ik.py to run in ~1-2 minutes nproblems=10_000/nproblems_slow=1_000 took the better part of an hour -- LM Sugihara converges far slower than the other methods on random targets (up to ~500x LM Chan's cost) and dominated the total runtime. Sized down so the script is actually practical to run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 95ac98e commit a0ef01c

4 files changed

Lines changed: 15 additions & 10 deletions

File tree

‎examples/_cpu_info.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env python
2-
"""Shared helper for the benchmark scripts in this directory (rne_speed.py,
3-
ik_speed.py) -- factored out once a second script needed the same thing.
2+
"""Shared helper for the benchmark scripts in this directory (benchmark_rne.py,
3+
benchmark_ik.py) -- factored out once a second script needed the same thing.
44
"""
55

66
import os
Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,18 @@
2323

2424
### Experiment parameters
2525
# Number of problems to solve for the C-accelerated columns (ik_XX and
26-
# ikine_XX with the C++ ETS)
27-
nproblems = 10_000
26+
# ikine_XX with the C++ ETS). Sized for a ~1-2 minute total run, not for
27+
# per-solver statistical precision: LM Sugihara converges far slower than
28+
# the other methods on random targets (tens of ms/problem even with the
29+
# C++ ETS, ~500x LM Chan's cost) and dominates this phase's total time, so
30+
# nproblems is capped by its cost rather than the cheaper solvers'.
31+
nproblems = 1_000
2832

2933
# The pure-Python ETS fallback is interpreted at every iteration, not just
30-
# the outer solver loop -- 10,000 problems would take far too long, so use
31-
# a much smaller sample for that column alone.
32-
nproblems_slow = 1_000
34+
# the outer solver loop -- full nproblems would take far too long (LM
35+
# Sugihara alone runs ~2s/problem here), so use a much smaller sample for
36+
# that column alone. This is a rough comparison, not a precise one.
37+
nproblems_slow = 15
3338

3439
# Cartesion DoF priority matrix
3540
mask = np.array([1.0, 1.0, 1.0, 1.0, 1.0, 1.0])
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
see rne.md plan step 7 and tests/test_fknm_fallback.py's
2020
TestRneTrajectoryVaryingRows for the correctness side of this).
2121
22-
Riffs off ik_speed.py's ANSITable reporting and rne_compare.py's timing
22+
Riffs off benchmark_ik.py's ANSITable reporting and rne_compare.py's timing
2323
harness.
2424
"""
2525

‎examples/rne_compare.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
and now asserts on the incompatible case instead of silently returning a
1414
wrong answer. So Robot.rne(puma, ...) below is *expected* to raise, and
1515
this script demonstrates that rather than crashing on it. For a 3-way
16-
comparison on a robot where all three genuinely apply, see rne_speed.py
16+
comparison on a robot where all three genuinely apply, see benchmark_rne.py
1717
(uses rtb.models.DH.Panda(), which is mdh=True).
1818
"""
1919

@@ -73,7 +73,7 @@ def bench(label, fn, repeat=5):
7373
print()
7474
print(f"speedup rne_python -> C: {t_py/t_c:6.1f}x")
7575
print("(Robot.rne skipped here -- Puma560 is standard DH, see note above."
76-
" For a full 3-way timing comparison, see rne_speed.py.)")
76+
" For a full 3-way timing comparison, see benchmark_rne.py.)")
7777

7878
print()
7979
print("Per-row C-call count check (does rne() batch the trajectory into one")

0 commit comments

Comments
 (0)