Repository navigation
fix: after a driver reset NVML is started anew before anything reads from it - #25
Merged
Merged
Conversation
…from it On driver 617.42 the clock, temperature and power readings come from memory the driver shares with the process. A driver reset takes that memory away, and every such reading is an access violation from then on; the one that was caught leaves nvmlShutdown waiting for ever. The power limit and the fan speed answer a reset with a status instead, so they are asked first. When one of them says the card is gone, nothing reads the shared memory, and the one NVML session of the process is stopped and started again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… one NVML session Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 of 6 tasks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #24 (which builds on #23 and #22) and contains their commits.
What went wrong
On driver 617.42 a run that reached the driver reset at the memory edge ended at stock with
the driver did not come back after a reset; v0.4.0 froze at the same point. #22 took the freeze away, not the cause.The cause, measured with a standalone program that only calls NVML (hardware check 84): the clock, temperature and power readings come from memory the driver shares with the process. A driver reset takes that memory away and the library goes on reading it. From then on every one of those calls is an access violation, until the process ends. One that was caught leaves
nvmlShutdownwaiting for ever.What this changes
nvmlInitis counted by the library, so the window'sNvmlobject and the run's now share one session. With one each, the window's kept the stale one alive under the run's reconnect.All of it is in
src/hw/nvml.cpp; no caller changed.Verification
cmake --workflow --preset ci:core_testsandapp_testspass. The new test fails when the check is taken outthe driver was reset; reconnecting, then on to its result. This is where the run of 2026-10-10 endedNot verified
🤖 Generated with Claude Code