[Backport release/3.0.0-beta2] FrameView local poses (#5677) + view-scoped Fabric selections (#6805) (#7342)

# Description

Backports two already-merged FrameView PRs to `release/3.0.0-beta2`,
cherry-picked in merge order:

- #5677 — GPU-accelerated FrameView local poses + `xform_space_writer`
API (upstream `69888c34e471`)
- #6805 — scale `FabricFrameView` selections to the view, not the stage
(upstream `480370cdc5b0`, nvbug 6535498)

Two support commits, because this branch predates some prerequisites:

- `Shim needed for Arena compatibility.` (@amillane) adds
`split_clone_template`, backported from #6071. Ordered first so no
commit imports it before it exists.
- `fix: use the OVPhysX tensor binding API…` restores this branch's
`physx.create_tensor_binding()` / `binding.read()`. The #5677 pick
referenced `OvPhysxView`, which arrives with #6224 and is absent here,
so `OvPhysxFrameView` init raised `ModuleNotFoundError`.

Conflict resolutions converge each touched file on its post-#6805 state
from `develop`, so the backported code is byte-identical to what has
been running and tested there. Two consequences of that choice:

- `get_scales()` now returns `ProxyArray` in all backends, as on
`develop`. On this branch it previously returned `wp.array`, so callers
that pass the result straight to Warp APIs need `.warp` (Torch users:
`.torch`). This is the one API change the backport carries.
- The Fabric-hierarchy import stays unconditional (the optional import
is #6499, not backported).

The only deviation from `develop`'s state is test-infra: device
parametrization uses literal `["cpu", "cuda:0"]` because
`test_devices()` does not exist on this branch.

## Type of change

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality) —
`FrameView.close()`, writer-scope API
- Breaking change (relative to this release branch only): `get_scales()`
returns `ProxyArray` instead of `wp.array`, matching `develop`

## Tests

- `test_views_xform_prim_ovphysx.py` — 49 passed (26 `cpu` + 24
`cuda:0`; needs one process per device, `ovphysx<=0.3.7` binds device
mode in C++ on first `PhysX(...)`)
- `test_views_xform_prim_newton.py` — 58 passed

---------

Co-authored-by: alex <amillane@nvidia.com>
Co-authored-by: pv-nvidia <197907000+pv-nvidia@users.noreply.github.com>
Co-authored-by: pv-nvidia <peter.verswyvelen@gmail.com>
Co-authored-by: yts-nv <yts-nv@users.noreply.github.com>
Co-authored-by: Kelly Guo <kellyg@nvidia.com>

Author: 5 people

docs/source/api/lab/isaaclab.utils.rst

@@ -188,3 +188,16 @@ Warp operations
    :members:
    :imported-members:
    :show-inheritance:
+
+Warp Fabric kernels
+^^^^^^^^^^^^^^^^^^^
+
+Warp kernels for reading and writing Fabric ``Matrix4d`` attributes
+(``omni:fabric:worldMatrix`` / ``omni:fabric:localMatrix``) via
+:class:`wp.fabricarray` and :class:`wp.indexedfabricarray`. Used by
+:class:`~isaaclab_physx.sim.views.FabricFrameView` to keep child world and
+local matrices consistent without round-tripping through USD.
+
+.. automodule:: isaaclab.utils.warp.fabric
+   :members:
+   :show-inheritance:

scripts/benchmarks/benchmark_view_comparison.py

@@ -284,7 +284,8 @@ def _run_pose_benchmarks(
 
     start_time = time.perf_counter()
     for _ in range(num_iterations):
-        view.set_world_poses(new_positions, orientations)
+        with view.xform_world_space_writer() as w:
+            w.set_poses(new_positions, orientations)
     timing_results["set_world_poses"] = (time.perf_counter() - start_time) / num_iterations
 
     ret_pos, ret_quat = view.get_world_poses()

scripts/benchmarks/benchmark_xform_prim_view.py

@@ -138,21 +138,35 @@ def benchmark_frame_view(  # noqa: C901
 
     is_newton = api == "isaaclab-newton-site"
 
+    # Synchronize around timed regions using Warp directly (rather than torch),
+    # since all backend kernels here are Warp launches and ``wp.synchronize()``
+    # covers CPU/CUDA Warp devices consistently.  We only need it for GPU runs,
+    # where kernel launches are asynchronous; guard on device to avoid paying
+    # it needlessly on CPU.
+    _needs_sync = str(device).startswith("cuda")
+
+    def _sync() -> None:
+        if _needs_sync:
+            wp.synchronize()
+
     def to_torch(a):
-        return wp.to_torch(a) if isinstance(a, wp.array) else a
+        if isinstance(a, wp.array):
+            return wp.to_torch(a)
+        if hasattr(a, "torch"):
+            return a.torch
+        return a
 
     try:
         # -- Warmup --------------------------------------------------------
         xform_view.get_world_poses()
+        xform_view.get_world_scales()
 
         # -- get_world_poses -----------------------------------------------
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         start_time = time.perf_counter()
         for _ in range(num_iterations):
             positions, orientations = xform_view.get_world_poses()
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         timing_results["get_world_poses"] = (time.perf_counter() - start_time) / num_iterations
 
         positions_t = to_torch(positions)
@@ -161,34 +175,36 @@ def to_torch(a):
         computed_results["initial_world_orientations"] = orientations_t.clone()
 
         # -- set_world_poses -----------------------------------------------
+        # ``.warp`` unwraps the ProxyArray returned by ``get_*_poses`` /
+        # ``get_*_scales`` to the underlying ``wp.array`` that ``wp.clone``
+        # requires.  ProxyArray was introduced in PR #5304 ("ProxyArray and
+        # Asset/Sensor level property caching") which changed the FrameView
+        # getter return type.  Applies to every ``wp.clone`` call below.
         if is_newton:
-            new_positions = wp.clone(positions)
+            new_positions = wp.clone(positions.warp)
             wp.to_torch(new_positions)[:, 2] += 0.1
         else:
             new_positions = positions_t.clone()
             new_positions[:, 2] += 0.1
 
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         start_time = time.perf_counter()
         for _ in range(num_iterations):
-            xform_view.set_world_poses(new_positions, orientations)
-        if is_newton:
-            torch.cuda.synchronize()
+            with xform_view.xform_world_space_writer() as w:
+                w.set_poses(new_positions, orientations)
+        _sync()
         timing_results["set_world_poses"] = (time.perf_counter() - start_time) / num_iterations
 
         pa, oa = xform_view.get_world_poses()
         computed_results["world_positions_after_set"] = to_torch(pa).clone()
         computed_results["world_orientations_after_set"] = to_torch(oa).clone()
 
         # -- get_local_poses -----------------------------------------------
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         start_time = time.perf_counter()
         for _ in range(num_iterations):
             translations, orientations_local = xform_view.get_local_poses()
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         timing_results["get_local_poses"] = (time.perf_counter() - start_time) / num_iterations
 
         translations_t = to_torch(translations)
@@ -198,45 +214,99 @@ def to_torch(a):
 
         # -- set_local_poses -----------------------------------------------
         if is_newton:
-            new_translations = wp.clone(translations)
+            new_translations = wp.clone(translations.warp)
             wp.to_torch(new_translations)[:, 2] += 0.1
         else:
             new_translations = translations_t.clone()
             new_translations[:, 2] += 0.1
 
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         start_time = time.perf_counter()
         for _ in range(num_iterations):
-            xform_view.set_local_poses(new_translations, orientations_local)
-        if is_newton:
-            torch.cuda.synchronize()
+            with xform_view.xform_local_space_writer() as w:
+                w.set_poses(new_translations, orientations_local)
+        _sync()
         timing_results["set_local_poses"] = (time.perf_counter() - start_time) / num_iterations
 
         ta, ola = xform_view.get_local_poses()
         computed_results["local_translations_after_set"] = to_torch(ta).clone()
         computed_results["local_orientations_after_set"] = to_torch(ola).clone()
 
-        # -- get_both (world + local) --------------------------------------
+        # -- get_world_scales ----------------------------------------------
+        _sync()
+        start_time = time.perf_counter()
+        for _ in range(num_iterations):
+            world_scales = xform_view.get_world_scales()
+        _sync()
+        timing_results["get_world_scales"] = (time.perf_counter() - start_time) / num_iterations
+
+        world_scales_t = to_torch(world_scales)
+        computed_results["initial_world_scales"] = world_scales_t.clone()
+
+        # -- set_world_scales ----------------------------------------------
+        if is_newton:
+            new_world_scales = wp.clone(world_scales.warp)
+            wp.to_torch(new_world_scales)[:] = 1.1
+        else:
+            new_world_scales = world_scales_t.clone()
+            new_world_scales[:] = 1.1
+
+        _sync()
+        start_time = time.perf_counter()
+        for _ in range(num_iterations):
+            with xform_view.xform_world_space_writer() as w:
+                w.set_scales(new_world_scales)
+        _sync()
+        timing_results["set_world_scales"] = (time.perf_counter() - start_time) / num_iterations
+
+        computed_results["world_scales_after_set"] = to_torch(xform_view.get_world_scales()).clone()
+
+        # -- get_local_scales ----------------------------------------------
+        _sync()
+        start_time = time.perf_counter()
+        for _ in range(num_iterations):
+            local_scales = xform_view.get_local_scales()
+        _sync()
+        timing_results["get_local_scales"] = (time.perf_counter() - start_time) / num_iterations
+
+        local_scales_t = to_torch(local_scales)
+        computed_results["initial_local_scales"] = local_scales_t.clone()
+
+        # -- set_local_scales ----------------------------------------------
         if is_newton:
-            torch.cuda.synchronize()
+            new_local_scales = wp.clone(local_scales.warp)
+            wp.to_torch(new_local_scales)[:] = 0.9
+        else:
+            new_local_scales = local_scales_t.clone()
+            new_local_scales[:] = 0.9
+
+        _sync()
+        start_time = time.perf_counter()
+        for _ in range(num_iterations):
+            with xform_view.xform_local_space_writer() as w:
+                w.set_scales(new_local_scales)
+        _sync()
+        timing_results["set_local_scales"] = (time.perf_counter() - start_time) / num_iterations
+
+        computed_results["local_scales_after_set"] = to_torch(xform_view.get_local_scales()).clone()
+
+        # -- get_both (world + local) --------------------------------------
+        _sync()
         start_time = time.perf_counter()
         for _ in range(num_iterations):
             xform_view.get_world_poses()
             xform_view.get_local_poses()
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         timing_results["get_both"] = (time.perf_counter() - start_time) / num_iterations
 
         # -- interleaved set -> get ----------------------------------------
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         start_time = time.perf_counter()
         for _ in range(num_iterations):
-            xform_view.set_world_poses(new_positions, orientations)
+            with xform_view.xform_world_space_writer() as w:
+                w.set_poses(new_positions, orientations)
             xform_view.get_world_poses()
-        if is_newton:
-            torch.cuda.synchronize()
+        _sync()
         timing_results["interleaved_world_set_get"] = (time.perf_counter() - start_time) / num_iterations
 
     finally:
@@ -267,15 +337,26 @@ def print_results(results_dict: dict[str, dict[str, float]], num_prims: int, num
     print(header)
     print("-" * 120)
 
-    operations = [
-        ("Initialization", "init"),
+    # ``init`` is the one-time view-construction cost.  We display it in the
+    # per-operation table but EXCLUDE it from the steady-state totals and the
+    # overall speedup -- otherwise a backend whose construction is dominated
+    # by stage population (e.g. Newton, where the first call materializes the
+    # site cache) shows a misleading "0.00x" overall and crushes the rest of
+    # the table.  The overall row is intended to compare per-iteration cost.
+    init_op = ("Initialization (one-time)", "init")
+    per_iter_operations = [
         ("Get World Poses", "get_world_poses"),
         ("Set World Poses", "set_world_poses"),
         ("Get Local Poses", "get_local_poses"),
         ("Set Local Poses", "set_local_poses"),
+        ("Get World Scales", "get_world_scales"),
+        ("Set World Scales", "set_world_scales"),
+        ("Get Local Scales", "get_local_scales"),
+        ("Set Local Scales", "set_local_scales"),
         ("Get Both (World+Local)", "get_both"),
         ("Interleaved World Set->Get", "interleaved_world_set_get"),
     ]
+    operations = [init_op, *per_iter_operations]
 
     for op_name, op_key in operations:
         row = f"{op_name:<28}"
@@ -286,15 +367,16 @@ def print_results(results_dict: dict[str, dict[str, float]], num_prims: int, num
 
     print("=" * 120)
 
-    total_row = f"{'Total':<28}"
+    total_row = f"{'Total (per-iter ops)':<28}"
     for name in api_names:
-        total_row += f" {sum(results_dict[name].values()) * 1000:>{col_width}.4f}"
+        per_iter_total = sum(results_dict[name].get(k, 0) for _, k in per_iter_operations)
+        total_row += f" {per_iter_total * 1000:>{col_width}.4f}"
     print(f"\n{total_row}")
 
     baseline = "isaaclab-usd"
     if baseline in results_dict and len(api_names) > 1:
         print("\n" + "=" * 120)
-        print(f"SPEEDUP vs {baseline.replace('-', ' ').title()}")
+        print(f"SPEEDUP vs {baseline.replace('-', ' ').title()} (per-iter ops; one-time init excluded)")
         print("=" * 120)
         header = f"{'Operation':<28}"
         for name in api_names:
@@ -304,7 +386,7 @@ def print_results(results_dict: dict[str, dict[str, float]], num_prims: int, num
         print("-" * 120)
 
         base = results_dict[baseline]
-        for op_name, op_key in operations:
+        for op_name, op_key in per_iter_operations:
             row = f"{op_name:<28}"
             base_t = base.get(op_key, 0)
             for name in api_names:
@@ -317,11 +399,11 @@ def print_results(results_dict: dict[str, dict[str, float]], num_prims: int, num
             print(row)
 
         print("=" * 120)
-        print(f"{'Overall':>28}", end="")
-        total_base = sum(base.values())
+        print(f"{'Overall (per-iter ops)':>28}", end="")
+        total_base = sum(base.get(k, 0) for _, k in per_iter_operations)
         for name in api_names:
             if name != baseline:
-                total_impl = sum(results_dict[name].values())
+                total_impl = sum(results_dict[name].get(k, 0) for _, k in per_iter_operations)
                 if total_base > 0 and total_impl > 0:
                     print(f" {total_base / total_impl:>{col_width}.2f}x", end="")
                 else:

source/isaaclab/changelog.d/fabric-local-poses.rst

@@ -0,0 +1,31 @@
+Added
+^^^^^
+
+* Added explicit local/world scale getters
+  :meth:`~isaaclab.sim.views.BaseFrameView.get_local_scales` and
+  :meth:`~isaaclab.sim.views.BaseFrameView.get_world_scales` to the FrameView
+  API, implemented for :class:`~isaaclab.sim.views.UsdFrameView`.  Scale
+  writes go through the writer scope (see the ``xform-space-writer``
+  fragment).
+
+* Added :func:`~isaaclab.utils.warp.fabric.decompose_indexed_fabric_transforms`,
+  :func:`~isaaclab.utils.warp.fabric.compose_indexed_fabric_transforms`,
+  :func:`~isaaclab.utils.warp.fabric.update_indexed_local_matrix_from_world`, and
+  :func:`~isaaclab.utils.warp.fabric.update_indexed_world_matrix_from_local`
+  Warp kernels operating on :class:`wp.indexedfabricarray` for reading and
+  writing Fabric ``Matrix4d`` attributes (``omni:fabric:worldMatrix`` /
+  ``omni:fabric:localMatrix``).
+
+Notes
+^^^^^
+
+* :meth:`~isaaclab.sim.views.BaseFrameView.get_scales` and
+  :meth:`~isaaclab.sim.views.BaseFrameView.set_scales` remain supported as
+  convenience helpers (not deprecated).  For reads where the space matters,
+  prefer the explicit ``get_local_scales`` (operates on ``xformOp:scale``) or
+  ``get_world_scales`` (composed world-space scale).  For writes that also
+  update poses, prefer batching inside one scope:
+  ``with view.xform_world_space_writer() as w: w.set_poses(...); w.set_scales(...)``
+  (or ``xform_local_space_writer``).
+  :class:`~isaaclab.sim.views.UsdFrameView` preserves prior behavior by
+  defaulting :meth:`get_scales` / :meth:`set_scales` to local scales.

source/isaaclab/changelog.d/fix-physx-newton-camera-pose-scaling.rst

@@ -0,0 +1,13 @@
+Fixed
+^^^^^
+
+* Fixed :class:`~isaaclab.scene_data.SceneDataProvider` transform mapping stalling
+  at high rigid-body counts, which delayed setup by minutes in scenes with
+  thousands of environments.
+
+Added
+^^^^^
+
+* Added :meth:`~isaaclab.sim.views.BaseFrameView.close` to release backend state
+  authored by a frame view. Backends also release best-effort on garbage
+  collection, but only an explicit close is deterministic.

source/isaaclab/changelog.d/xform-space-writer.rst

@@ -0,0 +1,35 @@
+Added
+^^^^^
+
+* Added :class:`~isaaclab.sim.views.FrameViewSpaceWriterBase`, the new context-managed
+  write API for ``FrameView``-managed prim transforms.  Open with
+  ``view.xform_world_space_writer()`` or ``view.xform_local_space_writer()`` and call
+  :meth:`~isaaclab.sim.views.FrameViewSpaceWriterBase.set_poses` /
+  :meth:`~isaaclab.sim.views.FrameViewSpaceWriterBase.set_scales` inside the scope;
+  the writer's ``__exit__`` derives the opposite-space matrices once and
+  synchronizes once.  Only one writer scope may be active per view at a
+  time.  View-level getters
+  (:meth:`~isaaclab.sim.views.BaseFrameView.get_world_poses` etc.) raise
+  :class:`RuntimeError` while a writer scope is active.
+
+* Added the two concrete tag classes
+  :class:`~isaaclab.sim.views.FrameViewWorldSpaceWriter` and
+  :class:`~isaaclab.sim.views.FrameViewLocalSpaceWriter` returned by
+  :meth:`~isaaclab.sim.views.BaseFrameView.xform_world_space_writer` /
+  :meth:`~isaaclab.sim.views.BaseFrameView.xform_local_space_writer`.
+
+Notes
+^^^^^
+
+* :meth:`~isaaclab.sim.views.BaseFrameView.set_world_poses`,
+  :meth:`~isaaclab.sim.views.BaseFrameView.set_local_poses`, and
+  :meth:`~isaaclab.sim.views.BaseFrameView.set_scales` remain supported as
+  convenience helpers (not deprecated).  Each opens a single-statement writer
+  scope internally, so updating poses and scales through separate calls derives
+  the opposite-space matrices and synchronizes twice.  For best performance,
+  update both inside one scope:
+  ``with view.xform_world_space_writer() as w: w.set_poses(...); w.set_scales(...)``
+  (or :meth:`~isaaclab.sim.views.BaseFrameView.xform_local_space_writer`).
+  The bundled examples use the writer scope to get this benefit; callers may
+  keep the convenience helpers when code simplicity matters more than shaving
+  a redundant derive/sync.