[Backport release/3.0.0] Newton startup, cloner, and rendering fixes (#7299)

## Summary

Backports the following merged changes to `release/3.0.0` as separate
provenance-preserving cherry-picks:

- #7292 — scope Newton global imports with clone plans
- #7285 — stabilize the sensor/PhysX video recording test
- #7269 — streamline Newton contact and raycast sensor startup
- #7119 — normalize non-finite depth display values safely
- #7295 — avoid repeated Newton model and articulation startup work

Each source squash commit was cherry-picked with `-x` and applied
without conflicts.

## Validation

- Stable patch IDs match all five source squash commits exactly.
- File-by-file manifests match each source squash commit.
- `git diff --check upstream/release/3.0.0..HEAD`
- `uv run --frozen python tools/changelog/cli.py check
backport-7285-7292-base`
- `SKIP=check-changelog-fragments uv run --frozen isaaclab -f`
- Cloner/Newton focused tests: 98 passed
- Scene global-ownership tests: 2 passed
- Simulator clone-plan tests: 4 passed
- Video recording regression test: 1 passed
- Newton BVH lifecycle tests: 2 passed
- Newton contact-selector tests: 7 passed
- Newton raycast BVH test: 4 passed
- Non-finite depth display tests: 4 passed
- #7295 physics lifecycle, cloner, manager, and coupling tests: 248
passed
- #7295 Newton joint-wrench sensor tests: 11 passed
- #7295 PhysX joint-wrench sensor tests: 16 passed

PR #7121 remains open and is intentionally excluded; it will be
backported from its final merge commit after merging.

---------

Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
Co-authored-by: matthewtrepte <mtrepte@nvidia.com>
Co-authored-by: camevor <camevor@nvidia.com>
Co-authored-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
Co-authored-by: Antoine RICHARD <antoiner@nvidia.com>

Author: 5 people

docs/source/how-to/cloning.rst

@@ -130,6 +130,8 @@ this page:
      - Long tensor of target env ids.
    * - ``positions``
      - Optional per-env world positions [m], shape ``[num_envs, 3]``.
+   * - ``global_paths``
+     - Unique prim paths for scene assets shared by every env and therefore not replicated.
 
 The plan is stage-agnostic by design — the same instance can be replayed against a
 different stage, inspected by tooling, or serialized.
@@ -141,6 +143,7 @@ When every env is a copy of env_0:
     sources      = ("/World/envs/env_0",)
     destinations = ("/World/envs/env_{}",)
     clone_mask   = [[True, True, ..., True]]
+    global_paths = ("/World/Ground", "/World/Light")
 
 When envs differ — say a cartpole in every env plus a 2-variant obstacle (box into
 envs 0/1, sphere into envs 2/3):
@@ -215,8 +218,7 @@ and exiting the block drains every registration against the plan:
 
 .. code-block:: python
 
-    with cloner.ReplicateSession(cfgs, num_clones=N, env_spacing=2.0,
-                                 device=device, stage=stage):
+    with cloner.ReplicateSession(cfgs, num_clones=N, env_spacing=2.0, device=device, stage=stage):
         for cfg in cfgs:
             cfg.class_type(cfg)
 
@@ -264,7 +266,7 @@ Shortcut for the case where every env is just a copy of env_0.
 one line by pointing at the prototype, and :func:`~isaaclab.cloner.replicate`
 finishes the setup. This is the pattern most :class:`~isaaclab.envs.DirectRLEnv`
 subclasses use — they author the env-0 prototype prim by prim in
-``_setup_scene`` and end the method with these four lines:
+``_setup_scene`` and end the method with this sequence:
 
 .. code-block:: python
 
@@ -275,11 +277,13 @@ subclasses use — they author the env-0 prototype prim by prim in
 
         src, dest = "/World/envs/env_0", "/World/envs/env_{}"
         pos = cloner.grid_transforms(self.scene.num_envs, self.scene.cfg.env_spacing, device=self.device)[0]
-        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos)
+        global_paths = ("/World/ground",)
+        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos, global_paths=global_paths)
         cloner.replicate(plan, stage=self.scene.stage)
 
 Every env receives the same prototype. When envs need to differ, use one of the
-other two.
+other two. Hand-built scenes must pass every shared asset root in ``global_paths``;
+use ``()`` when there are none.
 
 
 Under the Hood

docs/source/migration/migrating_from_isaacgymenvs.rst

@@ -231,7 +231,7 @@ adding any other optional objects into the scene, such as lights.
 |                                                                              |     spawn_ground_plane(                                                |
 |                                                                              |         prim_path="/World/ground", cfg=GroundPlaneCfg())               |
 |     self.sim = super().create_sim(self.device_id, self.graphics_device_id,   |     # create and apply a clone plan                                    |
-|                                     self.physics_engine, self.sim_params)    |     plan = cloner.clone_plan_from_env_0(...)                           |
+|                                     self.physics_engine, self.sim_params)    |     plan = cloner.clone_plan_from_env_0(..., global_paths=...)         |
 |     self._create_ground_plane()                                              |     cloner.replicate(plan, stage=self.scene.stage)                     |
 |     self._create_envs(self.num_envs, self.cfg["env"]['envSpacing'],          |     # add articulation to scene                                        |
 |                         int(np.sqrt(self.num_envs)))                         |     self.scene.articulations["cartpole"] = self.cartpole               |
@@ -680,8 +680,10 @@ the need to set simulation parameters for actors in the task implementation.
 |         self.sim_params)                                               |     positions = cloner.grid_transforms(                             |
 |     self._create_ground_plane()                                        |         self.scene.num_envs, self.scene.cfg.env_spacing,            |
 |     self._create_envs(self.num_envs,                                   |         device=self.device)[0]                                      |
-|         self.cfg["env"]['envSpacing'],                                 |     plan = cloner.clone_plan_from_env_0(                            |
-|         int(np.sqrt(self.num_envs)))                                   |         src, dest, self.scene.num_envs, self.device, positions)     |
+|         self.cfg["env"]['envSpacing'],                                 |     global_paths = ("/World/ground",)                               |
+|         int(np.sqrt(self.num_envs)))                                   |     plan = cloner.clone_plan_from_env_0(                            |
+|                                                                        |         src, dest, self.scene.num_envs, self.device, positions,     |
+|                                                                        |         global_paths=global_paths)                                  |
 |                                                                        |     cloner.replicate(plan, stage=self.scene.stage)                  |
 | def _create_ground_plane(self):                                        |     if "physx" in self.scene.physics_backend:                       |
 |     plane_params = gymapi.PlaneParams()                                |         self.scene.filter_collisions(global_prim_paths=[])          |

scripts/demos/pick_and_place.py

@@ -228,7 +228,8 @@ def _setup_scene(self):
         spawn_ground_plane(prim_path="/World/ground", cfg=GroundPlaneCfg())
         src, dest = "/World/envs/env_0", "/World/envs/env_{}"
         pos = cloner.grid_transforms(self.scene.num_envs, self.scene.cfg.env_spacing, device=self.device)[0]
-        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos)
+        global_paths = ("/World/ground",)
+        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos, global_paths=global_paths)
         cloner.replicate(plan, stage=self.scene.stage)
         # PhysX replication requires explicit collision filtering between environments.
         if "physx" in self.scene.physics_backend:

scripts/tutorials/06_deploy/anymal_c_env.py

@@ -66,7 +66,8 @@ def _setup_scene(self):
         self._terrain = self.cfg.terrain.class_type(self.cfg.terrain)
         src, dest = "/World/envs/env_0", "/World/envs/env_{}"
         pos = cloner.grid_transforms(self.scene.num_envs, self.scene.cfg.env_spacing, device=self.device)[0]
-        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos)
+        global_paths = (self.cfg.terrain.prim_path,)
+        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos, global_paths=global_paths)
         cloner.replicate(plan, stage=self.scene.stage)
         # PhysX replication requires explicit collision filtering between environments.
         if "physx" in self.scene.physics_backend:

source/isaaclab/changelog.d/contact-sensor-regex.skip

@@ -0,0 +1 @@
+Docstring-only clarification of the shape-expression convention; behaviour change lives in isaaclab_newton.

source/isaaclab/changelog.d/explicit-global-clone-plan.major.rst

@@ -0,0 +1,11 @@
+Added
+^^^^^
+
+* Added :attr:`~isaaclab.cloner.ClonePlan.global_paths` to identify scene assets shared by every environment
+  without representing them as replication rows.
+
+Changed
+^^^^^^^
+
+* Changed :func:`~isaaclab.cloner.make_clone_plan`, :func:`~isaaclab.cloner.clone_plan_from_env_0`, and
+  :class:`~isaaclab.cloner.ReplicateSession` to accept explicit ``global_paths`` tuples.

source/isaaclab/changelog.d/ooctipus-newton-active-solver-attributes.skip

@@ -0,0 +1,5 @@
+Fixed
+^^^^^
+
+* Fixed camera depth display normalization producing ``NaN`` values and suppressing finite depth contrast when
+  no-hit pixels contain ``inf`` or ``NaN`` values.

source/isaaclab/changelog.d/sylvesterkaczmarek-fix-depth-display-nonfinite.rst

@@ -61,6 +61,9 @@ class ClonePlan:
     cfg_rows: dict[int, tuple[int, ...]] = field(default_factory=dict)
     """``id(cfg)`` to the row indices the cfg owns."""
 
+    global_paths: tuple[str, ...] = ()
+    """Unique prim paths for scene assets shared by every environment."""
+
 
 def grid_transforms(N: int, spacing: float = 1.0, up_axis: str = "z", device="cpu"):
     """Create a centered grid of transforms for ``N`` instances.
@@ -221,7 +224,7 @@ def make_clone_plan(
     num_clones: int,
     env_spacing: float,
     device: str,
-    *,
+    global_paths: tuple[str, ...] = (),
     clone_strategy: Callable = sequential,
     valid_set: torch.Tensor | None = None,
     env_template: str = DEFAULT_ENV_TEMPLATE,
@@ -234,15 +237,16 @@ def make_clone_plan(
     envs, and returns a self-contained :class:`ClonePlan` with ``cfg_rows`` populated.
 
     Each input cfg's ``spawn_path`` / ``spawn_paths`` is mutated so the subsequent
-    asset constructor spawns the prototype into its first active environment. Cfgs
-    whose ``prim_path`` is global (not under the env root ``/World/envs/``) or that
-    lack a spawn are skipped — they do not appear in the plan and are not replicated.
+    asset constructor spawns the prototype into its first active environment. Every cfg
+    is an env-scoped entity with a spawner. Shared assets are declared explicitly through
+    ``global_paths`` and are never replicated.
 
     Args:
-        cfgs: Asset cfgs with resolved ``prim_path`` (no ``{ENV_REGEX_NS}`` macros).
+        cfgs: Cloneable asset cfgs with resolved env-scoped ``prim_path`` and ``spawn``.
         num_clones: Number of target envs.
         env_spacing: Distance between neighboring grid env origins [m].
         device: Torch device for plan tensors.
+        global_paths: Complete shared-asset roots declared by the scene composition root. Defaults to none.
         clone_strategy: Function that assigns prototype combinations to envs. Defaults
             to :func:`~isaaclab.cloner.sequential`.
         valid_set: Optional ``[num_combos, num_groups]`` long tensor of valid prototype
@@ -251,8 +255,8 @@ def make_clone_plan(
 
     Returns:
         A :class:`ClonePlan` whose ``sources``/``destinations``/``clone_mask`` describe
-        the flat prototype-to-env mapping and whose ``cfg_rows`` maps each cfg to the
-        rows it owns.
+        the flat prototype-to-env mapping, whose ``cfg_rows`` maps each replicated cfg
+        to the rows it owns, and whose ``global_paths`` names shared scene assets.
     """
 
     def set_spawn_paths(spawn_cfg: Any, paths: list[str | None]) -> None:
@@ -270,16 +274,11 @@ def set_spawn_paths(spawn_cfg: Any, paths: list[str | None]) -> None:
     # 1) Build per-group records: (cfg, spawn_cfg, destination_template, num_variants).
     groups: list[tuple[Any, Any, str, int]] = []
     for cfg in cfgs:
-        if not hasattr(cfg, "prim_path") or not hasattr(cfg, "spawn") or cfg.spawn is None:
-            continue
-        prim_path = cfg.prim_path
-        if (matched := match(prim_path, env_template)) is None:
-            continue
+        matched = match(cfg.prim_path, env_template)
         count = num_spawn_variants(cfg.spawn)
         if count <= 0:
-            raise ValueError(f"Spawner at '{prim_path}' must have at least one variant.")
+            raise ValueError(f"Spawner at '{cfg.prim_path}' must have at least one variant.")
         groups.append((cfg, cfg.spawn, env_template + matched.suffix, count))
-
     env_ids = torch.arange(num_clones, dtype=torch.long, device=device)
     positions, _ = grid_transforms(num_clones, env_spacing, device=device)
 
@@ -293,6 +292,7 @@ def set_spawn_paths(spawn_cfg: Any, paths: list[str | None]) -> None:
             env_ids=env_ids,
             positions=positions,
             cfg_rows={},
+            global_paths=global_paths,
         )
 
     # 3) Homogeneous (every cfg is single-variant): emit the simpler env-root plan.
@@ -307,6 +307,7 @@ def set_spawn_paths(spawn_cfg: Any, paths: list[str | None]) -> None:
             env_ids=env_ids,
             positions=positions,
             cfg_rows=cfg_rows,
+            global_paths=global_paths,
         )
 
     # 4) Heterogeneous: enumerate prototype combos, build per-row mask, mutate spawn paths.
@@ -375,6 +376,7 @@ def validate_combo_tensor(combos: torch.Tensor, name: str, expected_rows: int |
         env_ids=env_ids,
         positions=positions,
         cfg_rows=cfg_rows,
+        global_paths=global_paths,
     )
 
 
@@ -384,35 +386,37 @@ def clone_plan_from_env_0(
     num_clones: int,
     device: str,
     positions: torch.Tensor | None = None,
+    global_paths: tuple[str, ...] = (),
 ) -> ClonePlan:
     """Build a single-source clone plan that targets every env from one source row.
 
     Auto-populates :attr:`ClonePlan.cfg_rows` from :data:`~isaaclab.cloner.REPLICATION_QUEUE`,
     including only cfgs whose ``prim_path`` falls under the env-root prefix of
-    ``destination``. Must be called *after* all asset constructors have run, so their cfgs
-    are already registered in the queue; otherwise those assets will be skipped by the
-    subsequent :func:`~isaaclab.cloner.replicate` call.
+    ``destination``. ``global_paths`` is the complete declaration of shared assets; it is
+    never inferred from the stage or replication queue. Must be called *after* all asset
+    constructors have run, so their cfgs are already registered in the queue; otherwise
+    those assets will be skipped by the subsequent :func:`~isaaclab.cloner.replicate` call.
 
     Args:
         source: Source prim path (typically ``/World/envs/env_0``).
         destination: Destination template with ``"{}"`` for the env id.
         num_clones: Number of target envs.
         device: Torch device for the mask and env id buffers.
         positions: Optional per-env world positions [m], shape ``[num_clones, 3]``.
+        global_paths: Complete shared-asset roots for the hand-built scene. Defaults to none.
 
     Returns:
         A :class:`ClonePlan` with a single source row covering every env.
     """
     from .replicate_session import REPLICATION_QUEUE  # noqa: PLC0415
 
-    cfg_rows: dict[int, tuple[int, ...]] = {
-        id(cfg): (0,) for cfg in REPLICATION_QUEUE if match(cfg.prim_path, destination) is not None
-    }
+    cfg_rows = {id(cfg): (0,) for cfg in REPLICATION_QUEUE if match(cfg.prim_path, destination) is not None}
     return ClonePlan(
         sources=(source,),
         destinations=(destination,),
         clone_mask=torch.ones((1, num_clones), dtype=torch.bool, device=device),
         env_ids=torch.arange(num_clones, dtype=torch.long, device=device),
         positions=positions,
         cfg_rows=cfg_rows,
+        global_paths=global_paths,
     )

source/isaaclab/isaaclab/cloner/clone_plan.py

@@ -58,7 +58,8 @@ def replicate(plan: ClonePlan, *, stage: Usd.Stage, replicate_physics: bool = Tr
 
     Cfgs absent from ``plan.cfg_rows`` are silently skipped. Backend contexts run in
     ascending ``replicate_priority`` order. The queue is cleared up front, so a backend
-    failure cannot leak stale entries into the next call.
+    failure cannot leak stale entries into the next call. Every context receives the plan's
+    explicitly declared shared assets when it is constructed.
 
     Args:
         plan: Replication layout to dispatch.
@@ -72,7 +73,7 @@ def replicate(plan: ClonePlan, *, stage: Usd.Stage, replicate_physics: bool = Tr
     REPLICATION_QUEUE.clear()
 
     backend_package = FactoryBase._get_package_name(FactoryBase._get_backend())
-    backend_physics_ctx = getattr(importlib.import_module(f"{backend_package}.cloner"), "PHYSICS_CONTEXT", None)
+    backend_physics_ctx = importlib.import_module(f"{backend_package}.cloner").PHYSICS_CONTEXT
 
     # Group queued cfgs by backend, taking the union of row indices each backend owns.
     # In the homogeneous plan every cfg maps to row 0, so multiple queue_replication
@@ -85,20 +86,20 @@ def replicate(plan: ClonePlan, *, stage: Usd.Stage, replicate_physics: bool = Tr
         if rows is None:
             continue
         if cfg.cloning_contexts is None:
-            contexts = [backend_physics_ctx] if backend_physics_ctx else []
+            contexts = [backend_physics_ctx]
         else:
             contexts = [string_to_callable(c) if isinstance(c, str) else c for c in cfg.cloning_contexts]
         if not replicate_physics:
             contexts = [c for c in contexts if c is UsdReplicateContext]
         ctx_set = dict.fromkeys(contexts)
-        if getattr(cfg, "spawn", None) is not None and kit_available:
+        if cfg.spawn is not None and kit_available:
             ctx_set.setdefault(UsdReplicateContext, None)
         for BackendCtxCls in ctx_set:
             backend_rows.setdefault(BackendCtxCls, set()).update(rows)
 
     backend_ctxs: dict[type, Any] = {}
     for BackendCtxCls, row_set in backend_rows.items():
-        ctx = BackendCtxCls(stage)
+        ctx = BackendCtxCls(stage, global_paths=plan.global_paths)
         backend_ctxs[BackendCtxCls] = ctx
         row_list = sorted(row_set)
         ctx.queue_mapping(
@@ -109,7 +110,7 @@ def replicate(plan: ClonePlan, *, stage: Usd.Stage, replicate_physics: bool = Tr
             positions=plan.positions,
         )
 
-    for ctx in sorted(backend_ctxs.values(), key=lambda c: getattr(c, "replicate_priority", 0)):
+    for ctx in sorted(backend_ctxs.values(), key=lambda ctx: ctx.replicate_priority):
         ctx.replicate()
 
     SimulationContext.instance().set_clone_plan(plan)
@@ -139,6 +140,7 @@ def __init__(
         device: str,
         *,
         stage: Usd.Stage,
+        global_paths: tuple[str, ...] = (),
         clone_strategy: Callable = sequential,
         valid_set: torch.Tensor | None = None,
         replicate_physics: bool = True,
@@ -152,6 +154,7 @@ def __init__(
             env_spacing: Grid spacing between env origins [m].
             device: Torch device for plan tensors.
             stage: USD stage to author replicated prim specs into.
+            global_paths: Complete shared-asset roots declared by the composition root. Defaults to none.
             clone_strategy: Prototype-to-env assignment function.
             valid_set: Optional ``[num_combos, num_groups]`` long tensor of valid
                 prototype combinations; ``None`` uses the full cartesian product.
@@ -166,6 +169,7 @@ def __init__(
             num_clones=num_clones,
             env_spacing=env_spacing,
             device=device,
+            global_paths=global_paths,
             clone_strategy=clone_strategy,
             valid_set=valid_set,
             env_template=env_template,