Merge branch 'develop' into jichuanh/pretrained-feature-extractor-ckpt

Author: hujc7

.github/workflows/build.yaml

@@ -346,6 +346,7 @@ jobs:
         isaacsim-base-image: ${{ needs.config.outputs.isaacsim_image_name }}
         isaacsim-version: ${{ needs.config.outputs.isaacsim_image_tag }}
         filter-pattern: "not isaaclab_"
+        extra-uv-packages: "opencv-python-headless==4.13.0.90"
         shard-index: "0"
         shard-count: "3"
         warp-cache: restore
@@ -370,6 +371,7 @@ jobs:
         isaacsim-base-image: ${{ needs.config.outputs.isaacsim_image_name }}
         isaacsim-version: ${{ needs.config.outputs.isaacsim_image_tag }}
         filter-pattern: "not isaaclab_"
+        extra-uv-packages: "opencv-python-headless==4.13.0.90"
         shard-index: "1"
         shard-count: "3"
         warp-cache: restore
@@ -394,6 +396,7 @@ jobs:
         isaacsim-base-image: ${{ needs.config.outputs.isaacsim_image_name }}
         isaacsim-version: ${{ needs.config.outputs.isaacsim_image_tag }}
         filter-pattern: "not isaaclab_"
+        extra-uv-packages: "opencv-python-headless==4.13.0.90"
         shard-index: "2"
         shard-count: "3"
         warp-cache: restore

.github/workflows/config.yaml

@@ -10,6 +10,6 @@
 # which the CI credential can reach.
 isaacsim_image_name: nvcr.io/0947644777160149/internal/isaac-sim
 # Isaac Sim 6.1.0-alpha.50 (b86cf6ce) includes Kit 110.3.0-360924's fix for NVBug 6566677.
-isaacsim_image_tag: latest-develop@sha256:50d2491d9e04da52bb39f0c8a993fba991d2d13939e88f2008577e00a17c4b03
+isaacsim_image_tag: latest-develop@sha256:5633a399fc6a69f3b80a8b497ebbfce209e75877afcdf7f1f62e95f51f5e5288
 isaaclab_image_name: nvcr.io/0947644777160149/internal/isaac-lab
 ovphysx_wheelhouse_image: ""

docker/test/test_dockerfile_nonroot.py

@@ -169,6 +169,15 @@ def test_kitless_dockerfile_installs_newton_rl_ov_and_visualizers_without_isaac_
         )
 
 
+def test_container_test_runner_only_links_an_actual_isaac_sim_runtime():
+    """Cache mount points under /isaac-sim must not masquerade as an Isaac Sim installation."""
+    runner_text = (REPO_ROOT / ".github/actions/run-tests/run_tests.sh").read_text(encoding="utf-8")
+    guarded_link = re.compile(r"if \[ -x /isaac-sim/python\.sh \]; then\s+ln -s /isaac-sim _isaac_sim;?\s+fi")
+
+    assert guarded_link.search(runner_text)
+    assert runner_text.count("ln -s /isaac-sim _isaac_sim") == 1
+
+
 # --------------------------------------------------------------------------- #
 # Volume mount-point writability
 #

docs/source/features/isaac_teleop.rst

@@ -1525,7 +1525,7 @@ Haptic glove (per-finger grip)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Use :class:`~isaaclab_teleop.GloveHapticFeedbackCfg`. Each hand's contact sensor is **filtered
-against the grasped object** (``filter_prim_paths_expr``), so ``force_matrix_w`` reports each
+against the grasped object** (``filter_prim_paths_expr``), so ``normal_force_matrix_w`` reports each
 finger's force on that object and nothing else. The driver orders the per-finger magnitudes
 Thumb..Pinky (matched from the sensor's body names via ``finger_order``) into a
 ``FingerPowerVector`` rendered by a cross-process glove device on ``collection_id``.

docs/source/features/population_based_training.rst

@@ -11,10 +11,9 @@ by hand would be hopelessly slow. Cloning is Isaac Lab's answer: you author a
 small representative scene under ``/World/envs/env_n`` and the cloner expands it
 across the rest of the env population for you, optionally with per-env variation.
 
-The expansion itself is performed by each physics backend's native replicator —
-USD, PhysX, or Newton — wrapped by Isaac Lab's core :mod:`isaaclab.cloner` module
-behind a single uniform surface so the same user code works regardless of which
-backend is active.
+The expansion itself is performed by USD and the active physics backend's native
+replicator, wrapped by Isaac Lab's core :mod:`isaaclab.cloner` module behind a
+single uniform surface.
 
 .. contents:: On this page
    :local:
@@ -24,24 +23,29 @@ backend is active.
 The Backend Layer
 -----------------
 
-At the bottom of the stack, each backend exposes a single function that takes a
-flat description of the world layout and materializes it on its runtime. The
-signatures are deliberately parallel so the layers above can target every backend
-through one interface:
+At the bottom of the stack, each backend exposes a raw function that takes a flat
+description of the world layout. These functions are useful for standalone tools
+and tests and deliberately have parallel signatures:
 
 .. code-block:: text
 
-    backend_replicate(stage, sources, destinations, env_ids, mask, positions=None, quaternions=None, ...)
+    backend_replicate(stage, sources, destinations, env_ids, selection, positions=None, quaternions=None, ...)
 
 The arguments are parallel arrays describing the layout:
 
 * ``sources`` — source prim paths already authored on the stage.
 * ``destinations`` — destination templates containing ``"{}"``, formatted with each env id.
-* ``env_ids`` — long tensor of target env indices.
-* ``mask`` — bool tensor of shape ``[len(sources), num_envs]``; ``mask[i, j]`` is
-  ``True`` when env ``j`` should be populated from source ``i``.
+* ``env_ids`` — NumPy integer array of target env indices.
+* ``selection`` — NumPy boolean array of shape ``[len(sources), num_envs]``;
+  ``selection[i, j]`` is ``True`` when env ``j`` should be populated from source ``i``.
+  The raw USD function names this argument ``mask``; physics functions name it ``mapping``.
 * ``positions`` / ``quaternions`` — optional per-env world transforms.
 
+Production scene construction stores those arrays once in a
+:class:`~isaaclab.cloner.ClonePlan`. Simulation-owned backend contexts consume the
+same value through ``context.replicate(plan)``; no backend rebuilds the mapping
+from a second queue of array arguments.
+
 
 Standalone Examples
 ~~~~~~~~~~~~~~~~~~~
@@ -54,7 +58,7 @@ control. Production code reaches for one of the ways in
 
 .. code-block:: python
 
-    import torch
+    import numpy as np
     import isaaclab.sim as sim_utils
     from isaaclab.cloner import usd_replicate
 
@@ -65,10 +69,10 @@ control. Production code reaches for one of the ways in
 
     usd_replicate(
         stage,
-        sources=["/World/envs/env_0/Cube"],
-        destinations=["/World/envs/env_{}/Cube"],
-        env_ids=torch.arange(num_envs, device="cuda:0"),
-        mask=torch.ones((1, num_envs), dtype=torch.bool, device="cuda:0"),
+        sources=("/World/envs/env_0/Cube",),
+        destinations=("/World/envs/env_{}/Cube",),
+        env_ids=np.arange(num_envs),
+        mask=np.ones((1, num_envs), dtype=np.bool_),
     )
 
 **PhysX** — call PhysX and USD on the same sources and destinations (either order):
@@ -77,16 +81,28 @@ control. Production code reaches for one of the ways in
 
     from isaaclab_physx.cloner import physx_replicate
 
-    physx_replicate(stage, sources, destinations, env_ids, mask)
-    usd_replicate(stage, sources, destinations, env_ids, mask)
+    sources = ("/World/envs/env_0/Cube",)
+    destinations = ("/World/envs/env_{}/Cube",)
+    env_ids = np.arange(num_envs)
+    mapping = np.ones((1, num_envs), dtype=np.bool_)
+    physx_replicate(stage, sources, destinations, env_ids, mapping=mapping)
+    usd_replicate(stage, sources, destinations, env_ids, mask=mapping)
 
 **Newton**:
 
 .. code-block:: python
 
     from isaaclab_newton.cloner import newton_physics_replicate
 
-    newton_physics_replicate(stage, sources, destinations, env_ids, mapping=mask)
+    newton_physics_replicate(stage, sources, destinations, env_ids, mapping=mapping)
+
+**OvPhysX**:
+
+.. code-block:: python
+
+    from isaaclab_ov.cloner import ovphysx_replicate
+
+    ovphysx_replicate(stage, sources, destinations, env_ids, mapping=mapping)
 
 
 Cloning in a Backend-Agnostic Way
@@ -106,13 +122,10 @@ code never branches on the backend.
 ClonePlan
 ~~~~~~~~~
 
-A plan holds the parallel arrays a backend replicate consumes — sources,
+A plan holds the parallel arrays used by production clone contexts — sources,
 destinations, mask, env ids — in one place. Conceptually it is a small table
 where each row describes one distinct prototype-to-destination mapping; the
-fields listed below are that table's columns. Every entry point in
-:mod:`isaaclab.cloner` either produces a plan, consumes a plan, or both, so a
-quick look at the fields is the fastest way to build intuition for the rest of
-this page:
+fields listed below are that table's columns:
 
 .. list-table::
    :header-rows: 1
@@ -125,16 +138,18 @@ this page:
    * - ``destinations``
      - Destination templates with ``"{}"`` for the env id, one per row.
    * - ``clone_mask``
-     - Bool tensor ``[len(sources), num_envs]``; ``True`` when env ``j`` comes from row ``i``.
+     - NumPy boolean array ``[len(sources), num_envs]``; ``True`` when env ``j`` comes from row ``i``.
    * - ``env_ids``
-     - Long tensor of target env ids.
+     - Optional NumPy integer array of target env ids; execution requires it.
    * - ``positions``
      - Optional per-env world positions [m], shape ``[num_envs, 3]``.
    * - ``global_paths``
      - Unique prim paths for scene assets shared by every env and therefore not replicated.
+   * - ``context_rows``
+     - Clone-context types mapped to the rows they consume.
 
-The plan is stage-agnostic by design — the same instance can be replayed against a
-different stage, inspected by tooling, or serialized.
+The plan does not own a stage. Simulation-owned contexts supply their own runtime
+when they consume it.
 
 When every env is a copy of env_0:
 
@@ -193,7 +208,7 @@ column ``j`` stands for ``env_ids[j]``, and the queries speak ids throughout.
 
 A plan is the *what*. Putting one together and handing it to the backends is
 the *how*, and Isaac Lab exposes three idiomatic ways to do that. All three end
-in the same ``cloner.replicate(plan, stage=...)`` call, so the choice between
+in the same ``cloner.replicate(plan)`` call, so the choice between
 them is purely about ergonomics:
 
 * The first wraps both phases in a context manager and is what
@@ -214,11 +229,11 @@ them is purely about ergonomics:
 :class:`~isaaclab.cloner.ReplicateSession` is a context manager that brackets the
 whole cloning lifecycle. Entering the block builds the plan, the body is where
 you construct your assets (each one registers itself as part of its constructor),
-and exiting the block drains every registration against the plan:
+and exiting the block clears those constructor registrations and dispatches the plan:
 
 .. code-block:: python
 
-    with cloner.ReplicateSession(cfgs, num_clones=N, env_spacing=2.0, device=device, stage=stage):
+    with cloner.ReplicateSession(cfgs, num_clones=N, env_spacing=2.0):
         for cfg in cfgs:
             cfg.class_type(cfg)
 
@@ -253,10 +268,10 @@ intervene before replication actually happens:
 
 .. code-block:: python
 
-    plan = cloner.make_clone_plan(cfgs, num_clones=N, env_spacing=2.0, device=device)
+    plan = cloner.make_clone_plan(cfgs, num_clones=N, env_spacing=2.0)
     for cfg in cfgs:
         cfg.class_type(cfg)
-    cloner.replicate(plan, stage=stage)
+    cloner.replicate(plan)
 
 ``clone_plan_from_env_0`` + ``replicate``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -276,10 +291,10 @@ subclasses use — they author the env-0 prototype prim by prim in
         # ... any other assets ...
 
         src, dest = "/World/envs/env_0", "/World/envs/env_{}"
-        pos = cloner.grid_transforms(self.scene.num_envs, self.scene.cfg.env_spacing, device=self.device)[0]
+        pos = cloner.grid_transforms(self.scene.num_envs, self.scene.cfg.env_spacing)[0]
         global_paths = ("/World/ground",)
-        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, self.device, pos, global_paths=global_paths)
-        cloner.replicate(plan, stage=self.scene.stage)
+        plan = cloner.clone_plan_from_env_0(src, dest, self.scene.num_envs, pos, global_paths=global_paths)
+        cloner.replicate(plan)
 
 Every env receives the same prototype. When envs need to differ, use one of the
 other two. Hand-built scenes must pass every shared asset root in ``global_paths``;
@@ -289,118 +304,37 @@ use ``()`` when there are none.
 Under the Hood
 --------------
 
-To see how the backend-agnostic surface works, follow one asset through the
-system. Suppose you write ``Articulation(cfg)`` for a PhysX articulation
-somewhere inside a :class:`~isaaclab.cloner.ReplicateSession`. The constructor
-does not actually clone anything yet — at that moment the plan describing how
-the full env population should be laid out may not even exist. Instead the
-constructor *registers* the asset with the cloner, the cloner files the
-registration into a queue, and later — when the session exits and the cloner
-runs replication — that registration is handed to the backend code that knows
-how to replicate a PhysX articulation, with the plan telling it where each
-clone goes.
-
-The story has to look like this because the engines underneath disagree about
-*when* and *how* replication actually happens:
-
-* **PhysX** defers the real work to physics runtime. At construction time the
-  only thing user code can do is register intent; PhysX replays those
-  registrations entity by entity when the simulation comes up.
-* **USD** is declarative and immediate — calling :func:`~isaaclab.cloner.usd_replicate`
-  materializes the clones in place, right then and there.
-* **Newton** is also declarative and immediate, but it insists on replicating
-  the whole world in one shot rather than asset by asset, so the framework
-  cannot just hand it one cfg at a time — everything Newton-related has to be
-  assembled first.
-
-Isaac Lab reconciles these into one surface with two small pieces of plumbing.
-Every backend supplies its own :class:`~isaaclab.cloner.UsdReplicateContext` /
-``PhysxReplicateContext`` / ``NewtonReplicateContext``, a class that hides the
-timing and granularity differences above behind a single uniform interface. A
-shared :data:`~isaaclab.cloner.REPLICATION_QUEUE` then remembers which asset
-cfgs participate until it is time to run. The three
-subsections below explain the queue, the contexts, and the function that joins
-them against a plan.
-
-The registration queue
-~~~~~~~~~~~~~~~~~~~~~~
-
-Assets do not replicate inline. Construction only registers the asset's cfg
-through :func:`~isaaclab.cloner.queue_replication`; the queue records *which*
-cfgs participate:
+Planning maps each cfg to rows in ``cfg_rows`` and each participating backend to
+its subset in ``context_rows``. The active physics manager registers its clone
+context during simulation initialization. Assets use that context by default;
+:attr:`~isaaclab.assets.AssetBaseCfg.cloning_contexts` can select an explicitly
+registered context instead. Planning also registers
+:class:`~isaaclab.cloner.UsdReplicateContext` for spawned assets when Kit is
+available.
 
-.. code-block:: text
-
-    REPLICATION_QUEUE
-        cartpole_cfg
-        cube_cfg
-        camera_cfg
-        ...
-
-*How* each cfg is cloned is resolved by :func:`~isaaclab.cloner.replicate` at
-dispatch. The physics side comes from the cfg's
-:attr:`~isaaclab.assets.AssetBaseCfg.cloning_contexts` when set, otherwise the
-active backend's default physics context, which each backend cloner exports as
-``PHYSICS_CONTEXT`` (PhysX and Newton replicate natively; OvPhysX replays its
-own clones). :func:`~isaaclab.cloner.replicate` adds
-:class:`~isaaclab.cloner.UsdReplicateContext` automatically whenever a cfg has a
-spawner and Kit is available, so USD clones accompany physics replication under
-Kit and are skipped by default in headless runs. An explicit cfg override may
-still request USD replication without Kit. With
-:attr:`~isaaclab.cloner.CloneCfg.replicate_physics` disabled, cloning is
-USD-only: every physics context is dropped and the physics engine parses the
-per-env USD prims directly.
-
-Deferring the work like this buys three things at once:
-
-* Replication can wait until the plan is fully built, so the final layout is
-  known before any prims are spawned.
-* Every asset's request is batched into a single backend call instead of one
-  call per asset.
-* Asset code stays free of any branching on which backend is active — it just
-  registers and lets the framework take it from there.
-
-:attr:`~isaaclab.scene.InteractiveSceneCfg.replicate_physics` is piped into
-:attr:`~isaaclab.cloner.CloneCfg.replicate_physics` and applied at dispatch;
-an asset whose only cloning mechanism is physics replication is then simply
-not cloned.
-
-Backend contexts
-~~~~~~~~~~~~~~~~
-
-Each backend ships a small adapter class — its *replicate context* — that
-knows how to take a registered cfg and replicate it on the backend's specific
-runtime:
+The backend packages expose different context implementations behind one
+execution contract:
 
 .. code-block:: text
 
     UsdReplicateContext      # replicates USD prim subtrees
     PhysxReplicateContext    # replicates PhysX rigid bodies and articulations
     NewtonReplicateContext   # replicates Newton bodies in its parallel pipeline
 
-A single asset usually resolves to more than one context — PhysX pairs its
-context with USD so physics and visuals both follow; Newton's default stack
-includes USD only under Kit, so kitless runs skip the authoring cost. This is
-where backend differences are absorbed: swapping a scene from PhysX to Newton
-swaps which default stack resolves at dispatch, while the cfgs and the rest of
-the user code stay unchanged.
-
-Running replication
-~~~~~~~~~~~~~~~~~~~
-
-:func:`~isaaclab.cloner.replicate` is what actually runs the registered work.
-The dispatch shape is roughly:
+:func:`~isaaclab.cloner.replicate` resolves these types through the
+:class:`~isaaclab.sim.SimulationContext` backend registry, orders them by
+``replicate_priority``, and passes the same plan to each one:
 
 .. code-block:: python
 
-    def replicate(plan, stage):
-        for context_cls, rows in group_queue_by_context(plan):
-            context_cls().replicate(rows=rows, stage=stage)
+    def replicate(plan):
+        for context_type in plan.context_rows:
+            simulation_backends[context_type].replicate(plan)
         publish(plan)
 
-Contexts run in a priority order that puts physics ahead of visuals, and the
-plan is published to :class:`~isaaclab.sim.SimulationContext` so the rest of the
-framework can read the per-env layout back.
+USD runs before native physics contexts so the destination topology exists when
+they consume it. No fallback context is constructed during dispatch. The plan is
+then published to the simulation context for downstream consumers.
 
 Collision Filtering
 -------------------

docs/source/how-to/cloning.rst

@@ -20,7 +20,7 @@ Quick Start
 ~~~~~~~~~~~
 
 Every property on asset and sensor data classes (e.g., ``robot.data.joint_pos``,
-``sensor.data.net_forces_w``) returns a ``ProxyArray``:
+``sensor.data.net_normal_forces_w``) returns a ``ProxyArray``:
 
 .. code-block:: python