[Docs] Merge and clean up RL docs

[StafaH]

Description

Merge the reinforcement learning overview, framework comparison, existing-script reference, and training guide into one task-oriented page updated for the Isaac Lab 3.0 CLI and preset conventions.

The consolidated guide recommends RSL-RL as the default and explains the distinct reasons to choose RL-Games, SKRL, or Stable-Baselines3. It also adds a reproducible OVRTX-rendered ANYmal-D learning-progression GIF and a media-generation workflow modeled after the quickstart scripts.

No additional dependencies are required.

Type of change

• Documentation update

Release backport

• Backport this pull request to the active release branch after it merges into develop

Screenshots

The merged RL page includes the generated ANYmal-D learning-progression GIF, recorded with OVRTX at iterations 0, 100, and 299.

Validation

• uv run --isolated --extra test -- make -C docs current-docs (passed with no warnings)
• uv run --no-project python tools/skills/cli.py check (validated 21 skills)
• uvx ruff check tools/docs/media/capture_reinforcement_learning.py
• uvx ruff format --check tools/docs/media/capture_reinforcement_learning.py
• bash -n tools/docs/media/generate_reinforcement_learning.sh
• git diff --check
• uv run isaaclab -f (all applicable formatting, RST, codespell, executable, large-file, and LFS checks passed; the repository-wide changelog hook reports unrelated baseline fragment divergence in source packages not touched by this documentation-only PR)
• Ran tools/docs/media/generate_reinforcement_learning.sh end-to-end with OVRTX and inspected the generated GIF.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with uv run isaaclab -f
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works (documentation-only change; validated with a warning-as-error Sphinx build)
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (no source packages touched)
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

The PR consolidates the reinforcement-learning documentation around the Isaac Lab 3.0 CLI and adds a reproducible ANYmal-D learning-progression media workflow.

• Replaces three RL documentation pages with a task-oriented consolidated guide.
• Updates documentation and skill references to the consolidated page.
• Adds OVRTX playback configuration and a shell pipeline for training, recording, and assembling the progression GIF.

Confidence Score: 3/5

The PR should not merge until the media generator selects an emitted final checkpoint and the removed documentation URLs are preserved or redirected.

A fresh media-generation run stops when it looks for model_299.pt after a 300-iteration training run, and deleting the legacy source pages leaves existing repository links pointing to missing generated pages.

Files Needing Attention: tools/docs/media/generate_reinforcement_learning.sh; docs/source/overview/reinforcement-learning/rl_existing_scripts.rst; docs/source/overview/reinforcement-learning/rl_frameworks.rst

Important Files Changed

Filename	Overview
docs/source/overview/reinforcement-learning/index.rst	Consolidates library selection, training, monitoring, checkpoint, and troubleshooting guidance into one current-CLI page.
tools/docs/media/generate_reinforcement_learning.sh	Adds the end-to-end media pipeline, but its model_299.pt expectation does not match the configured run's final checkpoint.
tools/docs/media/capture_reinforcement_learning.py	Registers a fixed ANYmal-D capture configuration and preserves the custom visualizer-backed recorder through playback setup.
docs/source/overview/reinforcement-learning/rl_existing_scripts.rst	Removes a published page that still has inbound repository links, without a redirect stub.
docs/source/overview/reinforcement-learning/rl_frameworks.rst	Removes a published page still referenced by maintained source and release notes, without preserving its URL.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Train[Train ANYmal-D for 300 iterations] --> Saves[RSL-RL saves checkpoints]
  Saves --> Select[Select iterations 0, 100, and final]
  Select --> Play[Play each checkpoint with OVRTX]
  Play --> Clips[Write three MP4 clips]
  Clips --> FFmpeg[Stack and encode clips]
  FFmpeg --> GIF[Documentation GIF]

Loading

Comments Outside Diff (1)

1. docs/source/overview/reinforcement-learning/rl_existing_scripts.rst

   Deleted pages break inbound links

   Deleting the legacy RL pages without compatibility stubs leaves unchanged links in README.md, maintained agent configurations, and release notes pointing to missing generated pages instead of the consolidated guide.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "Merge and clean up RL documentation" | Re-trigger Greptile}

[isaaclab-review-bot]

Isaac Lab Review Bot

The RL documentation consolidation is coherent and references to the removed pages are retargeted, but two actionable issues remain: the media generator uses a physics selector inconsistent with its installed backend extra, and the merged source-of-truth page drops the maintenance link to its synchronized RL skills.

• Design and architecture: The task-oriented consolidated page is a suitable replacement for the deleted RL overview, comparison, workflow, and training-guide pages. However, it should preserve the former training guide’s explicit instruction to synchronize the train-rl-agents and debug-rl-training skills, since those skills now identify this merged page as their maintained documentation source.
• API: The supplied documentation targets and inbound references are consistently migrated: _rl-frameworks and _pretrained-checkpoints remain available, _choose-an-rl-library supports the new quickstart reference, and references in the changed documentation and skills point to the consolidated page.
• Implementation: The capture helper and generator follow the existing documentation-media workflow, but the generator installs the ovphysx extra while passing physics=physx during both training and playback. Those commands should use the documented physics=ovphysx selector to match the selected integration and make the workflow reproducible.

Minor fixes needed. Posted 2 actionable findings inline.

Automated review; human maintainers own approval decisions.

[StafaH]

Applied the review follow-up in 8f9411331:

• restored the RL skill synchronization note
• added orphaned compatibility stubs for all three legacy RL page URLs
• updated the README to link directly to the consolidated guide
• documented the verified zero-based RSL-RL checkpoint numbering and automatic PhysX selector behavior

Validation: bash -n passed, all 21 repository skills validated, and the full current-docs Sphinx build completed successfully without warnings.

[StafaH]

run-ci

[kellyguo11]

we try to keep gif files uploaded on S3 instead so that we can avoid bloating the repo size. generally, we only keep .jpg media files in the repo.

[StafaH]

Done

[kellyguo11]

maybe we don't use kit here since it requires isaac sim?

[StafaH]

Done

[kellyguo11]

what is the camera/ovrtx setup used for this? the environment doesn't come with a camera by default right?

[StafaH]

Fixed, this section was reworked so it no longer discuesses ovrtx

[kellyguo11]

maybe also a different visualizer here?

[StafaH]

Fixed (ended up removing this part of the docs)

[StafaH]

run-ci

[hujc7]

[P2] Agent found bug: this figure URL returns 404, so the page's only image will not render once published.

Measured against e7a284d0, with two same-host controls to rule out the CDN blocking range requests:

Object	GET
.../isaaclab/images/rl_progression_learning_anymald.gif	404 text/html
.../isaaclab/images/cosmos_inputs.gif	206 image/gif
.../isaaclab/images/record_video_example_sensor.gif	206 image/gif

The host answers and its other images serve fine — this object is simply not there, so it looks like the GIF was never uploaded when the reference moved off the in-repo copy.

Important

CI structurally cannot catch this. .github/workflows/check-links.yml passes --exclude 'download\.isaacsim\.omniverse\.nvidia\.com/isaaclab/images', so the link checker skips this host for every PR.

• Upload the GIF and confirm the URL returns image/gif

[hujc7]

[P2] Agent found bug: the 4.3 MB LFS blob is still committed here, and at head nothing references it.

Following up on the "keep GIFs on S3" thread above — the reference moved to S3 (docs/source/concepts/reinforcement_learning.rst:36), but the file stayed. At e7a284d0 the pointer still reads size 4335440, and git grep anymal-d-learning-progression matches only tools/docs/media/generate_reinforcement_learning.sh; no .rst uses it. The repo pays 4.3 MB of permanent LFS storage for a file the docs never load.

• git rm docs/source/_static/reinforcement-learning/anymal-d-learning-progression.gif

[hujc7]

[P2] Agent suggestion: the generator writes into the docs tree, so re-running the documented workflow re-creates the file that should not be committed.

Now that the page consumes the S3 copy, this OUTPUT_DIR produces an artifact nothing reads, and the next person who follows tools/docs/media/README.md silently re-adds the LFS blob. generate_quickstart.sh writes under _static/ legitimately, because its GIFs are in-repo; this one's is not, so copying that line carries the wrong assumption.

Suggest defaulting outside the checkout and naming the upload step — note WORK_DIR is rm -rf'd by the EXIT trap, so it cannot be the default:

-OUTPUT_DIR="${REPO_ROOT}/docs/source/_static/reinforcement-learning"
+# The page loads the published S3 copy, so the GIF is not committed; upload the result manually.
+OUTPUT_DIR="${RL_PROGRESS_OUTPUT_DIR:-$(mktemp -d)}"

and a line in tools/docs/media/README.md saying the output is uploaded to download.isaacsim.omniverse.nvidia.com/isaaclab/images/ rather than committed.

[hujc7]

[P2] Agent found bug: this bullet now points at a repo path while its four siblings are absolute docs URLs.

The lead-in is "Our [documentation page] provides everything ... Follow these links", and Installation / Tutorials / Available environments all use https://isaac-sim.github.io/IsaacLab/develop/.... A repo-relative .rst resolves only when the README is browsed on GitHub — it is dead on PyPI and any other README renderer, and even on GitHub it hands the reader raw reStructuredText instead of the rendered page.

Suggested change

	- [Reinforcement learning](docs/source/concepts/reinforcement_learning.rst)
	- [Reinforcement learning](https://isaac-sim.github.io/IsaacLab/develop/source/concepts/reinforcement_learning.html)

[hujc7]

[P2] Agent found bug: the notice now points at a page that has no "Training Performance" table.

This PR deleted that table along with rl_frameworks.rst, and the consolidated page deliberately omits throughput numbers ("Choose based on the features your experiment needs, not on a single throughput result"). So retargeting the URL leaves four files promising a table that is not at the destination — same edit in rl_games_manager_ppo_cfg.yaml:9, sb3_manager_ppo_cfg.yaml:9 and skrl_manager_ppo_cfg.yaml:9.

Either restore the comparison table on the new page, or reword the four notices to say these configs are kept in sync for cross-library comparison, without pointing at a table.

[hujc7]

[P2] Agent suggestion: these five hyperlink targets are never referenced anywhere on the page.

RSL-RL, skrl, RL-Games, Stable-Baselines3 and RLinf appear only as bold text in the comparison table, never as RSL-RL_-style references — the targets came across from rl_frameworks.rst, which did use them. Docutils reports unreferenced targets at INFO level, so the build stays clean and they simply sit here as dead markup.

Either drop lines 520-524, or turn the library names in the "Choose an RL library" table into references so a reader can click through to each project.

[StafaH]

run-ci

[StafaH]

run-ci

[isaaclab-bot]

Backported to release/3.0.0 as 4afadc7.