Skip to content

Commit e55e36a

Browse files
committed
BUG: Fix ASV harness against real ITK build + CLI reality
Changes surfaced during end-to-end smoke-test against a local ITK build with Module_PerformanceBenchmarking=ON: asv.conf.json - repo: '.' → 'itk-repo' (symlink the caller places into the harness pointing at the ITK source tree). Required so --set-commit-hash can validate SHAs against git. The consuming CI workflow and the local smoke-test recipe in README-asv.md both create this symlink. - branches: adjusted to 'origin/master' (the ITK branch whose tip we track for publish's time-series graph). - install_command: installs from relative 'python/' path rather than {build_dir}, since {build_dir} now refers to ITK checkouts. python/itk_perf_shim/registry.py - Executable names: itkCopyIterationBenchmark → CopyIterationBenchmark and the same for VectorIterationBenchmark. The add_executable() target name drops the 'itk' prefix present on the source file. - CLI: Core benchmarks take 'timingsFile iterations imageSize', not just 'timingsFile iterations'. Fixed imageSize to 128 (matching the in-tree CTest registrations) and iteration counts to 25 / 50. python/itk_perf_shim/runner.py - Input fixture: brainweb165a10f17.mha (the real ExternalData object) replaces the placeholder OAS1 filename I guessed pre-build-validation. .gitignore - Ignore the 'itk-repo' symlink. Smoke-test result (single-machine, same-binary, labeled as two different ITK SHAs to exercise asv compare): core.track_copy_iteration 10.5ms → 10.2ms ratio 0.98 core.track_vector_iteration 4.98ms → 4.61ms ratio 0.93 filtering.track_binary_add 5.39ms → 5.09ms ratio 0.94 filtering.track_gradient_magnitude 48.3ms → 47.3ms ratio 0.98 filtering.track_median 321ms → 312ms ratio 0.97 All within expected per-run jitter; exercise validates discovery, shim invocation, probe-JSON parsing, and asv compare output.
1 parent e33a735 commit e55e36a

4 files changed

Lines changed: 12 additions & 11 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,3 +47,4 @@ doc/_minted-ITKPerformanceBenchmarking/*
4747
*.egg-info/
4848
__pycache__/
4949
.asv/
50+
/itk-repo

asv.conf.json

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@
22
"version": 1,
33
"project": "ITK",
44
"project_url": "https://itk.org/",
5-
"repo": ".",
6-
"repo_subdir": "python",
7-
"branches": ["asv-native-harness", "master"],
5+
"_comment_repo": "'repo' points at the ITK source tree whose commits label the time series. The caller (CI workflow or local smoke test) must create a symlink named 'itk-repo' at the root of this harness pointing to the ITK clone before invoking asv.",
6+
"repo": "itk-repo",
7+
"branches": ["origin/master"],
88
"environment_type": "existing",
99
"show_commit_url": "https://github.com/InsightSoftwareConsortium/ITK/commit/",
1010
"benchmark_dir": "benchmarks",
@@ -13,7 +13,7 @@
1313
"html_dir": ".asv/html",
1414
"build_command": [],
1515
"install_command": [
16-
"python -m pip install --no-build-isolation -e {build_dir}"
16+
"python -m pip install --no-build-isolation -e python"
1717
],
1818
"uninstall_command": [
1919
"return-code=any python -m pip uninstall -y itk_perf_shim"

python/itk_perf_shim/registry.py

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@
1313

1414
BENCHMARKS = {
1515
"core.copy_iteration": {
16-
"exe": "itkCopyIterationBenchmark",
17-
"args": ["{timings_json}", "{iterations}"],
18-
"iterations": 5,
16+
"exe": "CopyIterationBenchmark",
17+
"args": ["{timings_json}", "{iterations}", "128"],
18+
"iterations": 25,
1919
},
2020
"core.vector_iteration": {
21-
"exe": "itkVectorIterationBenchmark",
22-
"args": ["{timings_json}", "{iterations}"],
23-
"iterations": 5,
21+
"exe": "VectorIterationBenchmark",
22+
"args": ["{timings_json}", "{iterations}", "128"],
23+
"iterations": 50,
2424
},
2525
"filtering.median": {
2626
"exe": "MedianBenchmark",

python/itk_perf_shim/runner.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ def run_benchmark(name: str) -> float:
8181
) as tf:
8282
timings_json = Path(tf.name)
8383

84-
brain_image = data_dir / "OAS1_0001_MR1_mpr-1_anon.nrrd"
84+
brain_image = data_dir / "brainweb165a10f17.mha"
8585
subs = {
8686
"timings_json": str(timings_json),
8787
"iterations": str(spec["iterations"]),

0 commit comments

Comments
 (0)