Skip to content

fix(hiptensor): Move benchmark tests off general ctests - #12202

Open
evedovelli wants to merge 2 commits into
developfrom
users/evedovel/hiptensor/ctest-without-bench
Open

evedovelli wants to merge 2 commits into
developfrom
users/evedovel/hiptensor/ctest-without-bench

Conversation

@evedovelli

@evedovelli evedovelli commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Motivation

Running a bare ctest from the installed component dir (/bin/hiptensor) picked up every registered test — including the bench performance runs. Benchmarks are very slow (5 hot runs + 1 cold run per config, plus CPU-reference
validation on every shape), so a plain ctest intended for a quick functional check ended up dominated by benchmark timing loops. The goal is to keep bench out of the default ctest run while still allowing it to be invoked explicitly,
and to stop benchmarks wasting time on validation they don't need.

ISSUE ID : ROCM-30711

Technical Details

  • Separate install fragment for bench. Added INSTALL_BENCH_TEST_FILE, emitted to its own bin/hiptensor/bench/CTestTestfile.cmake, mirroring the existing smoke/ and regression/ subdirs. Because the main
    bin/hiptensor/CTestTestfile.cmake does not descend into it, a bare ctest there no longer runs the bench entries. They are opt-in via ctest from /bin/hiptensor/bench (or --test-dir).
  • Path fixups for the nested fragment. From bin/hiptensor/bench/ the binary is two levels up (../../) and the shared config subtree is one level up (../configs/bench/...).
  • Disable validation on bench runs. The bench command now passes -v OFF. The CPU reference check is far slower than the kernel itself and would otherwise dominate the timing loop.
  • The other tiers (quick/standard/comprehensive/full, ffm-*) are unchanged and still run/filter from bin/hiptensor as before.
  • Docs (TESTING.md) updated to point bench commands at /bin/hiptensor/bench and note the -v OFF default.

Test Plan and Results

  • Configure + build + cmake --install --component tests.
  • From /bin/hiptensor: run ctest -N and confirm no * bench entries are listed; run ctest and confirm only regular/tier tests execute.
  • From /bin/hiptensor/bench: run ctest -N and ctest --print-labels, confirm the * bench entries appear and carry the bench label; run ctest -V and confirm each invokes ../../ -y ../configs/bench/... -v OFF --hot_runs 5
    --cold_runs 1 and that the config -y path resolves.

Submission Checklist

@therock-pr-bot

therock-pr-bot Bot commented Sep 16, 2026

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass
Forbidden Files ✅ Pass
🧪 Unit Test ✅ Pass PR does not contain code files — Unit Test auto-passed
🔎 pre-commit ✅ Pass
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled
🤖 therock-pr-bot ✅ Pass

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@Ryker0627 Ryker0627 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes are good. Worth notice that TESTING.md line 268 and test_categories.yaml line 91 need to update accordingly as well.

@Ryker0627 Ryker0627 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants