Skip to content

Commit 1a986c9

Browse files
authored
Document cuDNN fallback for deterministic training (#7559)
# Description Document `TORCH_CUDNN_V8_API_DISABLED=1` as an optional setting that may improve run-to-run determinism for convolutional training workloads. The note explains that the setting selects PyTorch's cuDNN v7 API instead of cuDNN v8 execution plans. No new dependencies are required. ## Type of change - Documentation update ## Release backport - [x] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop` ## Screenshots Not applicable. ## Checklist Docker and GPU tests run on demand. Push the commits you want tested, then comment `run-ci` on the pull request. - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `uv run isaaclab -f` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works (not applicable to this documentation-only change) - [x] I have added a changelog fragment under `source/<pkg>/changelog.d/` for every touched package (no source package is touched) - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there ## Validation - `uv run --isolated --extra dev --extra ov -- make -C docs current-docs` - `uv run isaaclab -f`
1 parent c58715b commit 1a986c9

1 file changed

Lines changed: 6 additions & 0 deletions

File tree

docs/source/features/reproducibility.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,12 @@ optional PyTorch deterministic algorithms. Whether the **rendering** half of the
3737
on the workload: **physics-only** simulation does not render at all; **RTX** rendering (non-minimal
3838
mode) needs it for reproducible imagery; **Newton** rendering is already deterministic.
3939

40+
.. note::
41+
42+
For convolutional workloads, setting ``TORCH_CUDNN_V8_API_DISABLED=1`` before launching training
43+
may improve run-to-run determinism by making PyTorch use the cuDNN v7 API instead of cuDNN v8
44+
execution plans.
45+
4046
**Physics determinism** comes from the same flag in the Isaac Lab RL training entrypoints, which
4147
set :attr:`~isaaclab.physics.PhysicsCfg.deterministic` on the backend resolved by presets. That
4248
field is the backend-agnostic request: each physics manager translates it into its own settings

0 commit comments

Comments
 (0)