Skip to content

Move sensor documentation to Concepts - #7372

Open
AntoineRichard wants to merge 26 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/docs-sensor-concepts
Open

Move sensor documentation to Concepts#7372
AntoineRichard wants to merge 26 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/docs-sensor-concepts

Conversation

@AntoineRichard

@AntoineRichard AntoineRichard commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Description

Moves the camera, contact, frame transformer, IMU, PVA, and joint-wrench sensor pages into the top-level Concepts section and rewrites them around configuration, data contracts, backend behavior, and runnable examples.

Keeps ray casting with the backend architecture documentation, updates navigation and cross-references, updates the sensor/actuator agent skill, and preserves the previous public URLs with redirect pages.

Adds a same-scene renderer output gallery to the renderer concept page. Newton Warp, OVRTX, and Isaac RTX each show animated RGB plus albedo, depth, normals, semantic segmentation, and instance segmentation. OVRTX and Isaac RTX also show motion vectors and all three distinct simple-shading modes. The scene uses mirror-like, transparent, semi-transparent, matte, glossy, and emissive falling spheres.

Adds the editable USD stage, capture script, generator, and focused tests so every gallery asset can be reproduced. No new dependencies.

Type of change

  • Documentation update

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Screenshots

Newton Warp OVRTX Isaac RTX
Newton Warp renderer comparison OVRTX renderer comparison Isaac RTX renderer comparison

The renderer Concepts page contains the complete tabbed still-output gallery.

Validation

  • uv run isaaclab -f
  • uv run --isolated --extra test -- python -m pytest --confcutdir=tools/test -q tools/test/test_capture_renderer_gallery.py (15 passed)
  • uv run --no-project python tools/skills/cli.py check (21 skills validated)
  • uv run --isolated --extra test -- make -C docs current-docs (warning-free)
  • bash -n tools/docs/media/generate_renderer_gallery.sh
  • git lfs fsck
  • Media audit: 26 assets at 640x360; three decoded animated WebPs

Checklist

  • I have read and understood the contribution guidelines
  • I have run the repository formatting checks with uv run isaaclab -f
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added focused tests for the gallery contract and capture pipeline
  • No changelog fragment is required because no source package changed
  • My name already exists in CONTRIBUTORS.md

@AntoineRichard
AntoineRichard requested a review from a team August 26, 2026 22:12
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 26, 2026
@greptile-apps

greptile-apps Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR relocates and substantially rewrites the camera, contact, frame-transformer, IMU, PVA, and joint-wrench documentation under the top-level Concepts section.

  • Preserves prior documentation URLs through redirect stubs and updates navigation and cross-references.
  • Adds renderer comparison media and expands camera renderer, output, and ISP guidance.
  • Updates backend support pages and the sensor/actuator skill.
  • The joint-wrench page currently presents a backend-specific body-cardinality contract as universal.

Confidence Score: 4/5

The documentation should be corrected before merging because its joint-wrench cardinality guidance can misalign PhysX body indices.

PhysX exposes a root-link wrench entry while the rewritten page promises only non-root entries, making the documented cross-backend shape and ordering contract inaccurate.

Files Needing Attention: docs/source/concepts/sensors/joint_wrench_sensor.rst

Important Files Changed

Filename Overview
docs/source/concepts/sensors/joint_wrench_sensor.rst Rewrites the joint-wrench contract but incorrectly says all backends exclude the root body; PhysX includes it.
docs/source/concepts/sensors/camera.rst Adds comprehensive camera configuration, output-contract, renderer-support, ISP, and performance documentation consistent with the inspected implementations.
docs/source/concepts/sensors/index.rst Introduces the top-level sensor concept index and links the retained backend-oriented ray-caster page.
docs/source/overview/core-concepts/renderers.rst Adds a qualitative same-scene visual comparison for the three renderer backends.
docs/source/overview/core-concepts/sensors/index.rst Replaces the old sensor index with a redirect whose relative target resolves to the new Concepts index.
skills/user/use-sensors-actuators/SKILL.md Updates sensor-documentation references to the relocated Concepts pages.

Reviews (1): Last reviewed commit: "Clarify contact sensor backend behavior" | Re-trigger Greptile

Comment on lines +23 to +24
Set :attr:`~sensors.JointWrenchSensorCfg.prim_path` to the articulation root. The sensor reports one
entry for each non-root articulation body:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Root wrench entry omitted

When users rely on the documented body count or positional ordering with PhysX, the sensor includes the articulation root in body_names, force, and torque, contrary to the stated non-root-only contract. This adds an entry and can misalign shape assertions or index-based wrench selection.

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The sensor documentation relocation is structurally coherent, but the camera rewrite drops an important operational caveat about ISP behavior on RTX renderers.

  • Design and architecture: Moving the sensor concepts while retaining ray casting under backend architecture is consistent with the updated toctrees and redirect stubs. The old public reference labels are preserved on the new pages.
  • API: No source API changes are introduced, and existing sensor documentation labels remain available. However, the documented ISP contract is incomplete because a backend-specific interaction with RTX exposure and tonemapping was removed.
  • Implementation: The new camera ISP section omits the prior warning that enabling isp_cfg on Isaac RTX or OVRTX disables RTX-side tonemapping, authors a neutral exposure schema, and is incompatible with RTX-side exposure authoring. This material side effect should be restored as a concise caveat.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

PPISP is composed by Isaac RTX, OVRTX, and Newton Warp. It requires ``rgb`` or ``rgba`` output. A
static configuration is shared by all cloned views in one camera batch; controller weights may
predict per-view exposure and color parameters, while the remaining coefficients stay shared. ISP
configuration and discovered USD attributes are fixed for the camera lifetime.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Suggestion · Implementation — Rewrite drops documented RTX ISP side effects

The deleted camera page documented that enabling isp_cfg on Isaac RTX/OVRTX force-disables RTX tonemapping and authors a neutral OmniRtxCameraExposureAPI_1 schema on each camera prim, so mixing it with RTX-side exposure authoring is unsupported. That stage-mutating behavior is not restated here or on any other page, and users cannot infer it from the new text. Re-add a short caveat in this section.

@matthewtrepte matthewtrepte Aug 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we can frame the marbles a bit lower in these images. even though they are dropped from the video, i think these images would benefit. perhaps capture these at a later timestep?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for depth map visuals, i like to use the "turbo" color map which has some greens / blues. also normalizing the range helps, so that the main foreground colors don't get squished.

Image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm should motion vectors show arrow / motion data?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!

Comment thread docs/source/concepts/renderers.rst Outdated
Renderers
=========

Isaac Lab uses a pluggable renderer architecture to support different rendering backends for camera sensors.

@matthewtrepte matthewtrepte Aug 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the Visualization intro, I quickly compare the visualizers against renderers, since they are similar sounding.

Isaac Lab provides 5 visualizers for real-time simulation inspection and debugging. Where renderers produce sensor data for training, visualizers give fast, lightweight feedback for human monitoring and recording. Visualizers can also stream that same sensor data through the streaming camera panel.

perhaps it'd be good to do similar here

@matthewtrepte

Copy link
Copy Markdown
Contributor

in the renderer tab view, it'd be good to include a bulleted list of per-renderer information. like for newton warp, a note why there are fewer GTs available. and just generally any useful per-renderer data

@matthewtrepte

Copy link
Copy Markdown
Contributor

perhaps we can add an automated script for capturing the marbles clip + images
tools/docs/media/renderers

there are example scripts there

@StafaH StafaH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Antoine, Can we move the images to S3, and remove the orphaned pages? For now just doing a macro review, we can do a finer pass later

@@ -1,104 +1,10 @@
.. _overview_sensors_contact:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's going on with these files? We're not able to remove?

@AntoineRichard

Copy link
Copy Markdown
Collaborator Author

perhaps we can add an automated script for capturing the marbles clip + images tools/docs/media/renderers

there are example scripts there

Yeah we can add the code, we'd also need to send the asset to nucleus. I wanted to avoid it. But it would be useful for sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation infrastructure

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

4 participants