Conversation
3 tasks
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
jsandham
approved these changes
Sep 18, 2026
Code coverage builds pass -fprofile-instr-generate -fcoverage-mapping to HIP compilations, so clang instruments device code as well and emits a __profd_* device global for every instantiated kernel. When the profile is dumped at process exit the HIP runtime tries to register those globals and fails, so rocsparse-test dies after every test has already passed and the coverage target never produces a report. Pass -fno-profile-instr-generate -fno-coverage-mapping to the device compilation only, via -Xarch_device, on the library and unit-test targets. Host-side coverage is unchanged; no device globals are emitted, so the profile dump completes. AISPARSE-778 Co-authored-by: Cursor <cursoragent@cursor.com>
kliegeois
force-pushed
the
task/rocsparse-codecov-system-rocprim
branch
from
September 18, 2026 22:45
6306fd5 to
64f046b
Compare
kliegeois
changed the title
Restricting the coverage instrumentation to host compilation removed the __profd_* device globals, but the HIP runtime still aborted at profile dump, now on __llvm_profile_sections_<hash>. That symbol is the per-TU section bounds table contributed by the AMDGPU profile runtime, which the driver links into the device image whenever -fprofile-instr-generate is passed to a HIP link. Scope the link-time flag to the host with -Xarch_host, so the device image carries no profile runtime and the host still links libclang_rt.profile. AISPARSE-778 Co-authored-by: Cursor <cursoragent@cursor.com>
-Xarch_host did not keep device profiling out of the image: the driver decides from the original arguments, so a HIP link that mentions -fprofile-instr-generate still forwards it to clang-linker-wrapper as --device-compiler=amdgcn-amd-amdhsa=-fprofile-instr-generate. The device image then carries the AMDGPU profile runtime and its per-TU __llvm_profile_sections_<hash> table, which the HIP runtime aborts on while registering it at profile dump. Link libclang_rt.profile-x86_64.a directly, with -u__llvm_profile_runtime to pull in the at-exit writer, and stop passing -fprofile-instr-generate to the link at all. Confirmed locally that an instrumented shared library built this way still writes a .profraw when loaded by an uninstrumented executable. AISPARSE-778 Co-authored-by: Cursor <cursoragent@cursor.com>
This reverts commit c546901.
Reapplies the direct link of the host profile runtime, this time naming the archives it depends on. libclang_rt.profile is not self-contained: its ROCm platform object references __prof_rocm device-drain helpers and the symbol-interception layer, so linking it alone left librocsparse.so with undefined references and broke the gfortran-linked samples. Resolve clang_rt.profile, clang_rt.profile_rocm, clang_rt.interception and clang_rt.sanitizer_common at configure time with --print-file-name, taking whichever exist, and link them as a group since they reference each other. This keeps -fprofile-instr-generate off the HIP link, which is what makes the driver add --device-linker=...profile.bc and put the device profile runtime into the device image. AISPARSE-778 Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #12317 +/- ##
===========================================
+ Coverage 70.78% 70.80% +0.02%
===========================================
Files 2842 2842
Lines 465020 465021 +1
Branches 68499 68503 +4
===========================================
+ Hits 329152 329223 +71
+ Misses 112174 112093 -81
- Partials 23694 23705 +11
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every rocSPARSE codecov job has failed since Sep 16. All tests pass, then the binary dies while dumping the profile:
This is not specific to any PR.
developcodecov was last green on build #288 (Sep 15) withlibamdhip64.so.7.13.99004-3309; #289 onward all fail and runlibamdhip64.so.7.16.26370. hipBLASLt hits the identical abort (#12268), there on a kernel in its own client test code.Cause
ROCm 7.16 enables source-based coverage for HIP device code. Two things then land in the device image, and the runtime aborts in
hip_global.cpp:209when it tries to register either one:-fprofile-instr-generate -fcoverage-mappingon a HIP compile instruments device code and emits a__profd_*global per instantiated kernel.-fprofile-instr-generateon a HIP link pulls the AMDGPU profile runtime into the device image, which contributes a__llvm_profile_sections_<hash>section-bounds table per TU.Upstream, a failed lookup of these is a
PROF_WARNinInstrProfilingPlatformROCm.cpp; this CLR turns it into a fatalguarantee. Any such symbol triggers it regardless of which header it came from, so scoping includes asSYSTEMor de-instrumenting individual translation units does not help — it only changes which symbol dies first.Fix
Keep coverage on the host side only, on
rocsparseand the unit-test targets:-Xarch_device -fno-profile-instr-generate -fno-coverage-mapping, so no__profd_*globals are emitted;-Xarch_host -fprofile-instr-generate, so the device image gets no profile runtime while the host still linkslibclang_rt.profile.Verified against the driver that the
amdgcn-amd-amdhsajob receives neither flag while the x86_64 host job keeps-fprofile-instrument=clangand-fcoverage-mapping, and that a host.profrawis still written after the link change. TheSHELL:prefixes keep CMake from de-duplicating the repeated-Xarch_*and silently re-enabling device instrumentation.Host-side coverage is unchanged. Device kernel bodies drop out of the coverage mapping, which is the cost of unblocking the job until the runtime side is fixed.
Notes