Skip to content

OMPE-103001: Render and animate Newton cables on all three renderers - #6981

Closed
jmart-nv wants to merge 1 commit into
isaac-sim:developfrom
jmart-nv:jmart/cable-render-binding
Closed

OMPE-103001: Render and animate Newton cables on all three renderers#6981
jmart-nv wants to merge 1 commit into
isaac-sim:developfrom
jmart-nv:jmart/cable-render-binding

Conversation

@jmart-nv

@jmart-nv jmart-nv commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

  • Newton cables (UsdGeom.BasisCurves) now render and animate on newton_warp, ovrtx and isaac_rtx. Previously they drew at their spawn pose and never moved.
  • Cable curve points are computed on device from the Newton segment bodies and written GPU/ASYNC, zero-copy, once per render frame.
  • Drops the CPU mirror in sync_cables_to_usd — it copied the whole model's body_q device→host on every dirty render frame, for a cable that reads only its own segments.
  • Adds the first rendering coverage for cables: 9 cells across the three renderers, gated on motion, not just presence.
  • Fixes three defects found alongside: view_count/camera-prim mismatch, the Isaac RTX sensor pump refreshing transforms only, and the Fabric stage being resolved exactly once at startup.

Same scene, same seed, three renderers: cables fall and settle. A cable that isn't driven renders frozen at its spawn pose — it does not disappear — which is why the tests gate on centroid displacement.

Requires:

Cable rendering depends on renderer-side fixes that are not in the pinned Kit/OVRTX:

needed
Kit !47946, !48359, !48511
OVRTX >= 0.4.1

The rendering cells skip below those floors rather than fail, so a green run on stock deps proves nothing. Check for 0 skipped.

Changes:

package what
isaaclab CameraRenderSpec rejects a view_count that disagrees with camera_prim_paths; Camera sizes it from the prims it found. Shared rendering-test harness in isaaclab.test.utils.cable_rendering.
isaaclab_newton sync_cables_to_usd runs on device and invalidates each curve's GPU data afterwards. New collect_cable_segment_shapes() for kit-less renderers. Lazy Fabric-stage recovery.
isaaclab_ov New OVRTX cable point binding (legacy zero-copy + ovstage).
isaaclab_physx Sensor pump uses pre_render() instead of forward(); empty-annotator-buffer guard.
scripts/demos/cables.py --camera, --seed, --frames_dir. Camera pose baked into the cfg; cables moved under /World/envs/env_0/.

Testing:

Three scenarios, each run on all three renderers — 9 cells:

test asserts
test_cable_renders a cable produces geometry at all
test_cable_render_tracks_simulation the rendered image follows the cable as it falls
test_cable_renders_across_environments both of the above, replicated across 4 envs
export OMNI_KIT_ACCEPT_EULA=yes
python -m pytest source/isaaclab_ov/test/ -q
python -m pytest source/isaaclab/test/renderers/test_cable_rendering.py -q -ra
python -m pytest source/isaaclab_physx/test/renderers/test_cable_rendering_isaac_rtx.py -q -ra
python scripts/demos/cables.py --visualizer none --camera isaac_rtx --seed 7 --max_steps 60
check expected
isaaclab_ov contract 112 passed
kit-less cells 6 passed, 0 skipped
isaac_rtx cells 3 passed
demo exit 0, 60 frames

Confirmed on four machines (1 Windows/RTX, 3 Linux/L40).

The isaac_rtx cells live in their own module because Kit must start before anything imports USD — do not merge them into the kit-less module, and do not run both in one pytest process.

Handoff notes for reviewers:

  • isaac_rtx_renderer.py empty-buffer guard: never fires in any cell, ships without a test, and its getattr(buf, "size", None) == 0 check cannot fire for a torch tensor (.size is a method there). The condition it was written for is now rejected earlier by CameraRenderSpec.__post_init__. Suggest dropping it from this PR and filing separately.
  • ovstage cable path hands over via host copy: ovstage 0.1.0 accepts the points lanes=3 dtype override only on numpy, not on DLPack producers. The legacy path — the GA one — stays zero-copy.
  • collect_cable_segment_shapes duplicates the discovery half of _initialize_fabric_cable_prims. Deliberate: the Kit path raises on a malformed cable, the kit-less path skips it. Merging them would change one of those behaviours.
  • The demo's cable prims moved from /World/Env_0/ to /World/envs/env_0/ — required for scene partitions, but it is a user-visible path change.

Appendix - required Kit MRs:

All three are merged to release/ovrtx-0.4-ovstage-0.1 and labelled ovrtx-0.4.1 — which is where the OVRTX ≥ 0.4.1 floor comes from.

  • !47946Allow Points and Curves to be created from GPU interop points data. Without it a GPU-authored curve resolves zero points and logs Not enough points, N is required but 0 was found. Merged 2026-08-01.
  • !48359Fix for curves empty bounding boxes in scenePartition. A BasisCurves in a partition published an inverted empty bbox, blanking every prim in that partition. Merged 2026-08-03.
  • !48511Update geometry group BB when ProceduralGeometry is updated. Animated point writes kept the creation-time bbox, so a falling cable was culled once it moved about its own thickness. Merged 2026-08-06.

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

cables_all_renderers

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump extension.toml — CI handles that)
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Cable curve points are now computed on device from the Newton segment bodies and written GPU/ASYNC each frame, so cables follow their simulated pose instead of drawing at their spawn pose and never moving. Drops the CPU mirror in sync_cables_to_usd, which copied the whole model's body_q device-to-host on every dirty render frame.

Adds the first rendering coverage for cables, gated on motion rather than presence, and fixes three defects found alongside: the view_count / camera-prim mismatch, the Isaac RTX sensor pump refreshing transforms only, and the Fabric stage being resolved exactly once at startup.

Requires Kit !47946, !48359 and !48511, and OVRTX >= 0.4.1. The rendering cells skip below those floors rather than fail.
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 7, 2026
@huidongc

Copy link
Copy Markdown
Collaborator

The work is being continued on #7016

@jmart-nv

Copy link
Copy Markdown
Contributor Author

Closing - superseded by #7016

@jmart-nv jmart-nv closed this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants