Skip to content

fix(tensilelite): revert Stream-K launch limit changes - #12022

Merged
jayhawk-commits merged 1 commit into
developfrom
users/jayhawk-commits/revert-tensilelite-streamk-launch-limits-11518
Sep 14, 2026
Merged

jayhawk-commits merged 1 commit into
developfrom
users/jayhawk-commits/revert-tensilelite-streamk-launch-limits-11518

Conversation

@jayhawk-commits

Copy link
Copy Markdown
Collaborator

Motivation

Revert the TensileLite Stream-K launch limit changes from #11518 because they cause the TheRock Multi-Arch CI Test tensilelite job to fail on both gfx94X and gfx950.

This is blocking the rocm-libraries submodule bump in ROCm/TheRock#8183.

Triage

The failing control is ROCm/TheRock#8183, which bumps rocm-libraries from d316419 to a70d405:

The failing jobs are:

Linux::release / Test gfx94X-dcgpu / Test tensilelite / Test tensilelite (shard 1/1)
Linux::release / Test gfx950-dcgpu / Test tensilelite / Test tensilelite (shard 1/1)

The common failure signatures are CuCount_test.cpp assertions in the StreamK launch-limit tests:

StreamKForceDPOnlyTest.FixedGridOverridesForceDPOnlyGrid
Expected solution.getSKGrid(...) == device.skFixedGrid
Actual: 64 vs 17
SKLaunchGridLimitsTest.StillUsesDpFallbackBelowTileThreshold
Expected: tiles < maxTilesBeforeCap
Actual: 16781312 vs 16777216

gfx94X also reports:

Sk3Sk5OffPartition512Test.NativeSk3MatchesSk5OffHostPack
sk3Pack.grid: 304 vs sk5OffPack.grid: 256
sk3Pack.skItersPerWG: 6 vs sk5OffPack.skItersPerWG: 8

Validation

Scoped TheRock Multi-Arch CI was run on top of the ROCm/TheRock#8183 pin with only the three files changed by #11518 restored:

  • rocm-libraries proof branch: triage/pr8183-revert-tensilelite-11518 / a597b3c68978
  • TheRock proof branch: users/jayhawk-commits/pr8183-tensilelite-revert-11518 / 0772634db4

Scope:

linux_test_labels=test:tensilelite,test_filter:standard
windows_amdgpu_families=<empty>
Arch Run/job Result
gfx94X https://github.com/ROCm/TheRock/actions/runs/34799220624/job/103850008205 PASS
gfx950 https://github.com/ROCm/TheRock/actions/runs/34799074697/job/103850007794 PASS

This PR applies the same three-file revert on current develop. No commits after #11518 touched these three files before this revert.

Fixes #12021

Revert the Stream-K launch limit changes from #11518.

TheRock Multi-Arch CI showed that the rocm-libraries #8183 bump failed the
standard TensileLite test job on both gfx94X and gfx950. Restoring the three
files changed by #11518 on top of that failing pin made both targeted jobs pass.

Fixes #12021
@jayhawk-commits
jayhawk-commits requested a review from a team as a code owner September 14, 2026 04:52
@jayhawk-commits
jayhawk-commits merged commit a218bdd into develop Sep 14, 2026
29 checks passed
@jayhawk-commits
jayhawk-commits deleted the users/jayhawk-commits/revert-tensilelite-streamk-launch-limits-11518 branch September 14, 2026 04:54
@therock-pr-bot

therock-pr-bot Bot commented Sep 14, 2026

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass
Forbidden Files ✅ Pass
🧪 Unit Test ✅ Pass
🔎 pre-commit ✅ Pass
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled
🤖 therock-pr-bot ✅ Pass

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #12022   +/-   ##
========================================
  Coverage    70.23%   70.23%           
========================================
  Files         2810     2810           
  Lines       462535   462535           
  Branches     68105    68105           
========================================
+ Hits        324821   324828    +7     
+ Misses      114191   114187    -4     
+ Partials     23523    23520    -3     
Flag Coverage Δ *Carryforward flag
TensileLite-CPP 46.40% <ø> (ø) Carriedforward from a70d405
TensileLite-Unit 76.07% <ø> (ø) Carriedforward from a70d405
hipBLAS 90.62% <ø> (ø) Carriedforward from a70d405
hipBLASLt 35.27% <ø> (+0.03%) ⬆️
hipCUB 82.68% <ø> (ø) Carriedforward from a70d405
hipDNN 87.02% <ø> (ø) Carriedforward from a70d405
hipFFT 43.07% <ø> (ø) Carriedforward from a70d405
hipRAND 76.12% <ø> (ø) Carriedforward from a70d405
hipSOLVER 68.92% <ø> (ø) Carriedforward from a70d405
hipSPARSE 86.99% <ø> (ø) Carriedforward from a70d405
rocBLAS 48.31% <ø> (ø) Carriedforward from a70d405
rocFFT 47.16% <ø> (ø) Carriedforward from a70d405
rocRAND 56.90% <ø> (ø) Carriedforward from a70d405
rocSOLVER 76.83% <ø> (ø) Carriedforward from a70d405
rocSPARSE 74.61% <ø> (ø) Carriedforward from a70d405
rocThrust 91.60% <ø> (ø) Carriedforward from a70d405

*This pull request uses carry forward flags. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@assistant-librarian

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

tony-davis pushed a commit to ROCm/TheRock that referenced this pull request Sep 15, 2026
## Summary

Test selection returns **no tests** for a change confined to
TensileLite, because the nested subtree path
`projects/hipblaslt/tensilelite` has no entry in
`_EXTERNAL_SUBTREE_ALIASES`. This adds the alias.

## Motivation

ROCm/rocm-libraries#11785 taught rocm-libraries' change detection to
report `projects/hipblaslt/tensilelite` (longest-prefix match) instead
of collapsing TensileLite files into `projects/hipblaslt`.
`determine_rocm_test_dependencies.py` has no alias for that string, so
it falls through to the `projects/` strip, produces the non-existent
graph key `hipblaslt/tensilelite`, warns, and selects nothing — while
exiting `0`.

Verified against `df3d451` (the commit rocm-libraries pins today):

```console
$ python test_tools/determine_rocm_test_dependencies.py \
    --changed-projects projects/hipblaslt/tensilelite
Warning: unrecognized project(s) ['hipblaslt/tensilelite']; no tests will be selected for them
["hipblaslt/tensilelite"]
$ echo $?
0
```

The consequence is visible in rocm-libraries. TensileLite-only PRs ran a
sanity check and nothing else:

| PR | files | multi-arch test components |
|---|---|---|
| ROCm/rocm-libraries#11518 | `tensilelite/src` + 2 new
`tensilelite/tests/*.cpp` | no `Test tensilelite` |
| ROCm/rocm-libraries#11896 | 7, all under
`projects/hipblaslt/tensilelite/**` | sanity only |
| ROCm/rocm-libraries#11877 | 6, all under
`projects/hipblaslt/tensilelite/**` | sanity only |

ROCm/rocm-libraries#11518 merged that way and was reverted by
ROCm/rocm-libraries#12022 after failing `Test tensilelite` on gfx94X and
gfx950 in this repo's CI — after merge, blocking the submodule bump in
#8183.

## Technical Details

One alias entry. `tensilelite` is already a synthetic node at level 3
(`test_policies.toml`), so selection also reaches `hipblaslt`, `rocblas`
and `hipblas` transitively — the same set `shared/stinkytofu` already
gets.

After the change, on the real committed graph:

```console
$ python test_tools/determine_rocm_test_dependencies.py \
    --changed-projects projects/hipblaslt/tensilelite
["hipblas", "hipblaslt", "hipblasltprovider", "hipdnn-samples", "hipsolver",
 "hipsparse", "hipsparselt", "miopen", "miopenprovider", "rocalution",
 "rocblas", "rocsolver", "rocsparse", "rocwmma", "tensilelite"]
```

## Risk Assessment

Risk 2. Test-selection only; no product or device behaviour changes. It
strictly widens selection for one input that currently selects nothing,
so the failure mode is longer CI on TensileLite PRs, not reduced
coverage.

## Related

- Work tracking: ROCm/rocm-libraries#11784
- Introduced by: ROCm/rocm-libraries#11785 (rocm-libraries half of the
nested-subtree fix)
- Companion PRs: #7998 (TensileLite as a synthetic node), #7999
(longest-prefix match)
- Fallout: ROCm/rocm-libraries#11518, reverted by
ROCm/rocm-libraries#12022, blocking #8183

## Testing Summary

- Unit: new regression test in
`determine_rocm_test_dependencies_test.py`, plus the existing suite.
- Manual: CLI run against the real committed graph, before and after.

## Testing Checklist

- [x] New regression test — `python -m unittest
test_tools.tests.determine_rocm_test_dependencies_test -k
nested_tensilelite` — Status: Passed
- [x] Verified it fails without the fix (stashed the alias, test fails)
— Status: Passed
- [x] Full selector suite — `python -m unittest
test_tools.tests.determine_rocm_test_dependencies_test` — 70 tests —
Status: Passed
- [ ] PR CI — Status: Pending

The new test declares `[synthetic.tensilelite]` in its fixture,
mirroring the real `test_policies.toml`, so it exercises the walk past
the alias rather than stubbing it. Clean `stderr` is asserted
deliberately: the regression was a warning plus an empty selection, not
a non-zero exit.

## Device / Architecture Coverage

None. Change-detection logic only; no compiled code and no device
behaviour. PR CI is sufficient.

## Adjacent Tests Considered

`shared/stinkytofu`, `shared/origami`, `shared/mxdatagenerator` and
`shared/tensile` all resolve through the same alias table and are
covered by `test_shared_blas_prefixes_mapped`; all still pass.

## Follow-ups not in this PR

1. **`projects/hipblaslt` still does not select `tensilelite`.** A
non-TensileLite hipBLASLt change selects `[hipblas, hipblaslt,
hipblasltprovider, hipsparselt, miopen, rocblas]`. rocm-libraries'
`therock_matrix.py` maps `projects/hipblaslt` → `blas` → `[hipblaslt,
rocblas, hipblas, tensilelite]`. Closing that gap means adding
`tensilelite` as a consumer of `hipblaslt` in the consumer graph, which
is a larger change than an alias and wants its own review.
2. **Unmapped `projects/` paths fail open.** `_EXTERNAL_ONLY_NAMESPACES`
makes an unmapped `shared/` path raise, but an unmapped `projects/` path
warns and exits `0`, yielding a green run with no tests. Extending the
fail-closed behaviour would have surfaced this on the day
ROCm/rocm-libraries#11785 merged rather than via a revert a week later.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@amd-chiranjeevi amd-chiranjeevi added the bump-pr-revert reverted pr which were created to unblock bump pr in throck label Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bump-pr-revert reverted pr which were created to unblock bump pr in throck ci:hipsparselt-fast project: hipblaslt project: hipsparselt

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Issue] Linux gfx94X/gfx950: TensileLite StreamK launch-limit tests fail in TheRock multi-arch CI

2 participants