Add shared kinematic rigid-object renderer contract

[ooctipus]

Description

Revives #6308 on current develop and supersedes #3728 with a single backend-neutral contract for kinematic rigid-object rendering.

Architecture

source/isaaclab/test/renderers/rigid_object_rendering_contract.py is the composition root. It owns the cloned scene, kinematic pose sequence, depth measurements, and assertions. Package-local adapters own only availability checks, simulation/renderer selection, and backend cleanup:

• Isaac RTX + PhysX on CPU and CUDA, with and without a coexisting articulation;
• Newton Warp + PhysX on CUDA;
• OVRTX + OVPhysX on CUDA through both legacy and OVStage scene ownership (OVStage runs when installed).

The dependency direction is adapter -> shared test contract -> public Isaac Lab APIs. An AST architecture gate rejects backend imports in the shared contract and rejects scene, asset, sensor, or class ownership in adapters.

The contract creates two cloned instanceable DexCubes with root-level nonuniform scale, verifies their depth silhouettes, moves both kinematic bodies through the public rigid-object tensor API, verifies the physics poses, and requires opposite rendered centroid displacement.

Current-develop audit

Most production changes in the old PR have since landed through newer ownership boundaries: Isaac RTX render-product lifetime in #6729, Newton shadow-state copying in #6773, OVRTX scale-aware transform writes in #7010, and Newton Fabric scale preservation in #7481. This revival removes those stale patches rather than carrying duplicate implementations.

The revived contract exposed one remaining OVRTX bug: composed scale was captured only for clone-plan source paths, while OVRTX creates non-source destinations after exporting the host USD stage. Those destinations therefore defaulted to unit scale. As a deliberately temporary bridge, this PR projects only captured non-unit scales through the existing isaaclab.cloner.query.path_env_ids and path_to_clone boundary using the already-validated ClonePlan; real destination scales take precedence. It adds no plan fields, query APIs, renderer configuration, or per-body fallback, and the bridge can be deleted as one unit when SDP supplies composed scale aligned with canonical rigid-body paths.

Historical context: Isaac Sim forum report.

Type of change

• Bug fix
• Shared regression coverage

Testing

• Isaac RTX contract: 4 passed (CUDA/CPU x articulation absent/present).
• Newton Warp contract: 1 passed.
• OVRTX contract: 2 passed (legacy and OVStage).
• OVRTX renderer unit surface: 167 passed.
• Core architecture and Newton visualization suites: 25 passed.
• OVRTX clone-plan suite: 17 passed.
• Cloner query and rendering-contract architecture suites: 87 passed.
• Controlled OVRTX regression: failed before the production fix with clone silhouettes of 264 vs. 36 pixels; passed after the fix.
• Incoming Route clone backends through ClonePlan #7462 NumPy-backed ClonePlan query-boundary smoke check: passed unchanged, including non-dense environment ids.
• uv run isaaclab -f: all hooks passed against the exact upstream develop base, including changelog validation.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks
• Documentation changes are not applicable
• I have added unit and integration regression coverage
• I have added changelog fragments for every touched package
• My name is already present in CONTRIBUTORS.md

[greptile-apps]

Greptile Summary

This PR adds a regression test for the PhysX/Fabric-to-RTX synchronization bug affecting kinematic rigid bodies (Isaac Sim 5.0), verifying that both non-uniform root scale and public pose writes are correctly reflected in the rendered depth output.

• Adds test_isaac_rtx_renderer_kinematic_rigid_object.py under the renderer integration suite, parametrized over cuda:0/cpu × articulation absent/present (4 variants), using regex-cloned instanceable DexCubes with root-level Z-scale=8 to confirm scale preservation and centroid displacement after pose writes.
• Adds an empty .skip changelog fragment, intentionally suppressing a changelog entry for this test-only change.

Confidence Score: 4/5

This is a test-only addition with no runtime or API changes; it is safe to merge.

The test correctly uses the public write_root_pose_to_sim_index / root_link_pose_w APIs, follows the established single-frame RTX capture pattern used throughout the renderer test suite, and properly cleans up RTX resources in a finally block. The two findings are purely stylistic: a missing type hint on the sim parameter of _write_pose_and_render, and a bare assert inside a non-test helper where pytest's assertion rewriting may not produce the richest failure output.

No files require special attention — both changed files are straightforward.

Important Files Changed

Filename	Overview
source/isaaclab_physx/test/renderers/test_isaac_rtx_renderer_kinematic_rigid_object.py	New regression test for PhysX/Fabric-to-RTX synchronization with kinematic rigid objects; well-structured, uses correct public APIs, with two minor style issues (missing type hint for sim, bare assert in a non-test helper)
source/isaaclab_physx/changelog.d/kinematic-rigid-object-rendering-test.skip	Empty .skip changelog fragment — intentionally suppresses a changelog entry for this test-only PR

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant T as Test
    participant RO as RigidObject (PhysX)
    participant Sim as SimulationContext
    participant Cam as Camera (RTX)

    T->>Sim: build_simulation_context(device)
    T->>RO: spawn kinematic cubes (scale 1,1,8)
    T->>Cam: spawn pinhole camera
    T->>Sim: sim.reset()
    T->>Cam: set_world_poses_from_view(eyes, targets)

    Note over T,Cam: Scale check
    T->>RO: write_root_pose_to_sim_index(center_poses)
    T->>Sim: sim.step()
    T->>RO: rigid_object.update(dt)
    T->>Cam: camera.update(dt)
    T->>RO: assert root_link_pose_w ≈ center_poses
    Cam-->>T: center_depth
    T->>T: "assert height > 3x width (scale preserved)"

    Note over T,Cam: Centroid-shift check
    T->>RO: write_root_pose_to_sim_index(negative_poses)
    T->>Sim: sim.step()
    Cam-->>T: negative_depth → negative_centroids
    T->>RO: write_root_pose_to_sim_index(positive_poses)
    T->>Sim: sim.step()
    Cam-->>T: positive_depth → positive_centroids
    T->>T: "assert shift > 10px and opposite directions"

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant T as Test
    participant RO as RigidObject (PhysX)
    participant Sim as SimulationContext
    participant Cam as Camera (RTX)

    T->>Sim: build_simulation_context(device)
    T->>RO: spawn kinematic cubes (scale 1,1,8)
    T->>Cam: spawn pinhole camera
    T->>Sim: sim.reset()
    T->>Cam: set_world_poses_from_view(eyes, targets)

    Note over T,Cam: Scale check
    T->>RO: write_root_pose_to_sim_index(center_poses)
    T->>Sim: sim.step()
    T->>RO: rigid_object.update(dt)
    T->>Cam: camera.update(dt)
    T->>RO: assert root_link_pose_w ≈ center_poses
    Cam-->>T: center_depth
    T->>T: "assert height > 3x width (scale preserved)"

    Note over T,Cam: Centroid-shift check
    T->>RO: write_root_pose_to_sim_index(negative_poses)
    T->>Sim: sim.step()
    Cam-->>T: negative_depth → negative_centroids
    T->>RO: write_root_pose_to_sim_index(positive_poses)
    T->>Sim: sim.step()
    Cam-->>T: positive_depth → positive_centroids
    T->>T: "assert shift > 10px and opposite directions"

Loading

_{Reviews (1): Last reviewed commit: "Add kinematic rigid object rendering reg..." | Re-trigger Greptile}

[greptile-apps]

Missing type annotation for sim parameter

The _write_pose_and_render helper is missing a type hint for sim, while rigid_object, camera, and root_poses are all annotated. Downstream tools (e.g. mypy, pyright) and any future callers have no signal about what concrete type is expected — a stale usage passing the wrong context won't be caught statically.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Bare assert in non-test helper function may lose context on failure

_measure_depth_mask is a plain function (not a test), so pytest's assertion introspection / rewriting doesn't apply when it is called from within _write_pose_and_render (which is itself also a plain helper). If pixel_counts fails the minimum-pixel guard, the only output is the f-string message — there's no captured locals, diff view, or surrounding context that pytest normally provides for assertions in test bodies. Consider using pytest.fail or raising a descriptive AssertionError explicitly, or annotating the function with inline comments explaining that assertion rewriting applies only to direct test-function bodies.

[ooctipus]

run-ci

[ooctipus]

I would make #6308 a deliberately temporary, deletable bridge—using today’s ClonePlan without adding any new plan fields or APIs.

Use the existing query boundary and pass the already-required plan explicitly:

from isaaclab.cloner import ClonePlan
from isaaclab.cloner import query as clone_query

def _capture_object_scales(self, stage: Any, plan: ClonePlan) -> None:
...
for source_path, scale in tuple(self._object_scales_by_path.items()):
for env_id in clone_query.path_env_ids(plan, source_path):
clone_path = clone_query.path_to_clone(plan, source_path, env_id)
assert clone_path is not None
self._object_scales_by_path.setdefault(clone_path, scale)

Call it only after prepare_stage() has validated the plan:

self._capture_object_scales(stage, self._clone_plan)

Why this transition is clean:

• It works with the current Torch-backed plan and the incoming NumPy plan unchanged; /home/zhengyuz/Projects/IsaacLab.wt/northstar-pr3a/source/isaaclab/isaaclab/cloner/query.py:120 hide that representation.
• Nested, heterogeneous, partial-environment, and non-dense layouts retain the cloner’s nearest-owner semantics.
• setdefault() preserves a destination scale already captured from the real stage.
• There is no silent missing-plan fallback.
• It adds no ClonePlan.scales, ScaleMap, new query function, or renderer configuration.
• The extra work remains sparse: scenes without non-unit scales perform no projection.

I would not resolve every object path through path_to_source() inside _create_object_scale_array(). Although visually flatter, it queries every rigid body in every environment. A host-only 4,096×48-body check cost roughly one second, including completely unscaled bodies. Source-side
projection only touches captured non-unit scales.

For testing, keep the cross-backend rendered contract. Change the focused OVRTX unit test to assert the resulting scale array for source and destination bodies, rather than asserting the private dictionary’s exact contents. That lets the temporary cache disappear later without
preserving its implementation.

The later SDP cutover should replace the whole bridge:

Plan completion: exact rigid-body paths + composed scales
Physics: native pose pointer + dirty
OVRTX: requests TransposedMatrix44d
SDP: pose + composed scale → matrix in one conversion
OVRTX: writes that pointer directly

Importantly, future FrameLayout.scale currently records only local xformOp:scale; the final conversion needs composed scale aligned with the canonical rigid-body paths. Once that exists, delete _capture_object_scales, _object_scales_by_path, and the destination projection together.

That version can merge before or after #7462 and should transition without another mechanical rewrite.

[ooctipus]

run-ci

[ooctipus]

run-ci