Skip to content

Commit 21bc111

Browse files
camevorooctipus
andauthored
[Newton] Streamline contact and raycast sensor startup (#7269)
## Summary This PR is now the contact/raycast part of the Newton startup work: - compile contact-sensor full-path expressions once and match them directly, without regex-to-glob conversion or stage globbing; - use Newton's current contact-sensing API names; - declare raycast collision-shape requirements before model finalization, so the model builds one correctly configured BVH instead of rebuilding it during sensor initialization. The contact-selector and sensing-API commits retain Chris's (`camevor`) original authorship. The BVH lifecycle change incorporates the review from #7296 while keeping this as Chris's PR. The related ownership work is intentionally split by component: #7292 handles explicit global ownership, and #7295 handles model/articulation startup. This PR contains no inactive-solver registration or articulation-view changes and supersedes #7296. Production code is `+29/-56` (net `-27`) against `909cc5decc5`. ## Startup benchmark RTX 5090, CUDA device 1, 4096 environments, three fresh processes per revision/task. Values are median end-to-end startup wall time. Base: `909cc5decc5`. PR: `5d0e1b1595e`. | Task | Base | PR | Change | |---|---:|---:|---:| | `Isaac-Cartpole` | 8.316 s | 8.410 s | +1.1% | | `Isaac-Velocity-Rough-UnitreeGo2` | 16.363 s | 15.682 s | -4.2% | | `Isaac-Lift-KukaAllegro-Camera` | 39.473 s | 37.605 s | -4.7% | Cartpole has no contact/raycast workload here and is neutral within process-startup noise. The sensor-heavy tasks show the intended gain: | Median phase | Go2 base | Go2 PR | Kuka base | Kuka PR | |---|---:|---:|---:|---:| | `newton_contact_sensor` | 0.10 s | 0.04 s | 1.32 s | 0.24 s | | `simulation_start` | 6.36 s | 5.91 s | 11.48 s | 10.39 s | Raw end-to-end totals: - Cartpole base: 9.368, 8.316, 8.195 s; PR: 8.410, 8.370, 8.476 s. - Go2 rough base: 19.900, 16.363, 16.091 s; PR: 15.669, 15.682, 15.984 s. - Kuka camera base: 41.328, 38.625, 39.473 s; PR: 37.580, 37.605, 38.785 s. ## Test plan - `261 passed, 8 xpassed` across the Newton manager abstraction, contact-sensor, and raycast-sensor suites. - Repository formatting and pre-commit checks pass. - Architecture checks reject the removed regex-to-glob path, deprecated sensing names, duplicate BVH state, and late BVH fallback. - The three 4096-environment benchmark tasks also provide end-to-end Newton MJWarp startup coverage. ## Type of change - Performance improvement - Bug fix --------- Co-authored-by: Octi Zhang <zhengyuz@nvidia.com>
1 parent 0a05bbd commit 21bc111

10 files changed

Lines changed: 172 additions & 58 deletions

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Docstring-only clarification of the shape-expression convention; behaviour change lives in isaaclab_newton.

source/isaaclab/isaaclab/sensors/contact_sensor/contact_sensor_cfg.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,9 @@ class ContactSensorCfg(SensorBaseCfg):
9898
**Newton backend only** (ignored by the PhysX and OvPhysX backends). A shape is an individual
9999
collision geometry attached to a body. If non-empty, :attr:`prim_path` is ignored for the
100100
sensing objects and these shape expressions are used instead.
101+
102+
Full-matched against shape paths, so an expression naming a body selects nothing:
103+
write ``{ENV_REGEX_NS}/Box[^/]*/.*`` to reach the shapes below it.
101104
"""
102105

103106
filter_shape_prim_expr: list[str] = []
@@ -106,6 +109,8 @@ class ContactSensorCfg(SensorBaseCfg):
106109
107110
**Newton backend only** (ignored by the PhysX and OvPhysX backends). If provided, the force
108111
matrix reports per-shape contact forces; mutually exclusive with :attr:`filter_prim_paths_expr`.
112+
113+
Matched against shape paths on the same terms as :attr:`sensor_shape_prim_expr`.
109114
"""
110115

111116
visualizer_cfg: VisualizationMarkersCfg = CONTACT_SENSOR_MARKER_CFG.replace(prim_path="/Visuals/ContactSensor")
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
Fixed
2+
^^^^^
3+
4+
* **Breaking:** Fixed Newton contact sensors matching their body and shape expressions as globs
5+
instead of regular expressions, which silently dropped alternation and let the segment-safe
6+
``[^/]*`` cross path separators. Expressions are now compiled and full-matched, as
7+
:func:`~isaaclab.utils.string.resolve_matching_names` already does elsewhere. An expression that
8+
relied on the widened wildcard to reach the shapes below a body now selects nothing and fails at
9+
sensor initialization; spell the descendant segments explicitly to migrate, so
10+
``sensor_shape_prim_expr=["{ENV_REGEX_NS}/Object[^/]*"]`` becomes
11+
``["{ENV_REGEX_NS}/Object[^/]*/.*"]``. The same applies to ``filter_shape_prim_expr``.
12+
* **Breaking:** Removed the contact sensor's bare-label fallback, which rewrote a path expression
13+
down to its final segment when no model label contained a separator. It dated from Newton's
14+
pre-hierarchical label API. Spell body and shape expressions as full paths to migrate, so
15+
``["fingertip_.*"]`` becomes ``["{ENV_REGEX_NS}/Robot/fingertip_[^/]*/.*"]``.
16+
* Migrated the Newton contact sensor off the deprecated ``sensing_obj_*`` names onto the
17+
replacements Newton 1.4 introduced.
18+
19+
Changed
20+
^^^^^^^
21+
22+
* Built the shared shape BVH with collision geometry during model finalization when a raycast sensor is present,
23+
instead of rebuilding the BVH when the sensor task initializes.

source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py

Lines changed: 22 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ def _paused_gc():
8989
)
9090
from isaaclab.sim import SimulationContext
9191
from isaaclab.sim.utils.newton_model_utils import replace_newton_builder_shape_colors
92-
from isaaclab.sim.utils.queries import has_deformable_curve_api, path_expr_to_glob
92+
from isaaclab.sim.utils.queries import has_deformable_curve_api
9393
from isaaclab.sim.utils.stage import get_current_stage
9494
from isaaclab.utils import checked_apply
9595
from isaaclab.utils.string import resolve_matching_names
@@ -120,6 +120,14 @@ def _paused_gc():
120120

121121
from isaaclab_newton.physics.newton_collision_cfg import NewtonCollisionPipelineCfg
122122

123+
124+
def _compile_label_pattern(expr: str | list[str] | None) -> re.Pattern[str] | None:
125+
"""Compile selector expressions for Newton's full label matching."""
126+
if not expr:
127+
return None
128+
return re.compile("|".join((expr,) if isinstance(expr, str) else expr))
129+
130+
123131
logger = logging.getLogger(__name__)
124132

125133
# Tagged union for entries in _cl_site_index_map.
@@ -435,7 +443,7 @@ class NewtonManager(PhysicsManager):
435443
_sensor_state: State | None = None
436444
_sensor_state_dirty: bool = True
437445
_sensor_graph_capture_failed: bool = False
438-
_sensor_bvh_has_collision_shapes: bool = False # set once a ray-cast sensor widens the shape BVH
446+
_sensor_bvh_shape_flags: ShapeFlags = ShapeFlags.VISIBLE
439447

440448
# USD/Fabric sync
441449
_newton_stage_path = None
@@ -1109,7 +1117,7 @@ def clear(cls):
11091117
NewtonManager._sensor_state = None
11101118
NewtonManager._sensor_state_dirty = True
11111119
NewtonManager._sensor_graph_capture_failed = False
1112-
NewtonManager._sensor_bvh_has_collision_shapes = False
1120+
NewtonManager._sensor_bvh_shape_flags = ShapeFlags.VISIBLE
11131121
NewtonManager._newton_stage_path = None
11141122
NewtonManager._usdrt_stage = None
11151123
NewtonManager._transforms_dirty = False
@@ -1178,6 +1186,7 @@ def create_builder(cls, up_axis: str | None = None, **kwargs) -> ModelBuilder:
11781186
mesh_constructor=cfg.bvh_constructor_geometry if isinstance(cfg, NewtonCfg) else None,
11791187
gaussian_constructor=cfg.bvh_constructor_gaussian if isinstance(cfg, NewtonCfg) else None,
11801188
shape_constructor=cfg.bvh_constructor_scene if isinstance(cfg, NewtonCfg) else None,
1189+
shape_flags=cls._sensor_bvh_shape_flags,
11811190
)
11821191

11831192
cls._register_builder_attributes(builder)
@@ -2577,32 +2586,21 @@ def get_contacts(cls) -> Contacts | None:
25772586
return cls._contacts
25782587

25792588
@classmethod
2580-
def _register_sensor_task(
2581-
cls, name: str, update_fn: Callable[[], None], *, include_collision_shapes: bool = False
2582-
) -> None:
2589+
def _register_sensor_task(cls, name: str, update_fn: Callable[[], None]) -> None:
25832590
"""Register a graph-capturable scene-query task.
25842591
25852592
Args:
25862593
name: Unique task name.
25872594
update_fn: Graph-capturable callable run by :meth:`_update_sensor_tasks`.
2588-
include_collision_shapes: Whether the task must see collision-only
2589-
geometry. Newton builds the shape BVH over visible shapes, which is
2590-
what renderers want; the first ray-cast sensor rebuilds it with
2591-
collision shapes added, since those must be hit even when they carry
2592-
no visual representation.
25932595
"""
25942596
if name in cls._sensor_tasks:
25952597
raise ValueError(f"Newton sensor task '{name}' is already registered.")
25962598
model = cls.get_model()
25972599
state = cls.get_state_0()
25982600
if model is None or state is None:
25992601
raise RuntimeError("Registering a Newton sensor task requires an initialized model and state.")
2600-
if model.shape_count > 0:
2601-
if include_collision_shapes and not cls._sensor_bvh_has_collision_shapes:
2602-
model.bvh_build_shapes(state, shape_flags=ShapeFlags.VISIBLE | ShapeFlags.COLLIDE_SHAPES)
2603-
NewtonManager._sensor_bvh_has_collision_shapes = True
2604-
elif model.bvh_shapes is None:
2605-
model.bvh_build_shapes(state)
2602+
if model.shape_count > 0 and model.bvh_shapes is None:
2603+
model.bvh_build_shapes(state)
26062604
if model.particle_count > 0 and model.bvh_particles is None:
26072605
model.bvh_build_particles(state)
26082606
cls._sensor_tasks[name] = update_fn
@@ -3357,8 +3355,9 @@ def add_contact_sensor(
33573355
) -> tuple[str | list[str] | None, str | list[str] | None, str | list[str] | None, str | list[str] | None]:
33583356
"""Add a contact sensor for reporting contacts between bodies/shapes.
33593357
3360-
Converts Isaac Lab pattern conventions (``.*`` regex, full USD paths) to
3361-
fnmatch globs and delegates to :class:`newton.sensors.SensorContact`.
3358+
Compiles the Isaac Lab regular expressions and delegates to
3359+
:class:`newton.sensors.SensorContact`, which full-matches compiled patterns
3360+
against model labels.
33623361
33633362
Args:
33643363
body_names_expr: Expression for body names to sense.
@@ -3386,48 +3385,20 @@ def add_contact_sensor(
33863385
def _hashable_key(x):
33873386
return tuple(x) if isinstance(x, list) else x
33883387

3389-
def _to_fnmatch(expr: str | list[str] | None) -> str | list[str] | None:
3390-
"""Convert Isaac Lab regex expressions (``.*``) to fnmatch glob (``*``)."""
3391-
if expr is None:
3392-
return None
3393-
if isinstance(expr, str):
3394-
return path_expr_to_glob(expr)
3395-
return [path_expr_to_glob(p) for p in expr]
3396-
3397-
def _normalize_for_labels(expr: str | list[str] | None, labels: list[str]) -> str | list[str] | None:
3398-
"""Strip leading path components from *expr* when labels are bare names.
3399-
3400-
Model labels may be full USD paths (``/World/envs/env_0/Robot/base``) or bare
3401-
names (``base``). When the labels are bare names but the user expression
3402-
contains slashes, we strip everything up to the last ``/``.
3403-
"""
3404-
if expr is None or not labels:
3405-
return expr
3406-
label_has_paths = any("/" in lbl for lbl in labels)
3407-
items = [expr] if isinstance(expr, str) else list(expr)
3408-
expr_uses_paths = any("/" in p for p in items)
3409-
if label_has_paths or not expr_uses_paths:
3410-
return expr
3411-
normalized = [p.rsplit("/", 1)[-1] for p in items]
3412-
return normalized[0] if isinstance(expr, str) else normalized
3413-
34143388
sensor_key = (
34153389
_hashable_key(body_names_expr),
34163390
_hashable_key(shape_names_expr),
34173391
_hashable_key(contact_partners_body_expr),
34183392
_hashable_key(contact_partners_shape_expr),
34193393
)
34203394

3421-
body_labels = list(cls._model.body_label)
3422-
shape_labels = list(cls._model.shape_label)
3423-
34243395
with Timer(name="newton_contact_sensor", msg="Contact sensor construction took:"):
34253396
sensor = NewtonContactSensor(
34263397
cls._model,
3427-
sensing_obj_bodies=_normalize_for_labels(_to_fnmatch(body_names_expr), body_labels),
3428-
sensing_obj_shapes=_normalize_for_labels(_to_fnmatch(shape_names_expr), shape_labels),
3429-
counterpart_bodies=_normalize_for_labels(_to_fnmatch(contact_partners_body_expr), body_labels),
3430-
counterpart_shapes=_normalize_for_labels(_to_fnmatch(contact_partners_shape_expr), shape_labels),
3398+
sensing_bodies=_compile_label_pattern(body_names_expr),
3399+
sensing_shapes=_compile_label_pattern(shape_names_expr),
3400+
counterpart_bodies=_compile_label_pattern(contact_partners_body_expr),
3401+
counterpart_shapes=_compile_label_pattern(contact_partners_shape_expr),
34313402
measure_total=True,
34323403
verbose=verbose,
34333404
)

source/isaaclab_newton/isaaclab_newton/sensors/contact_sensor/contact_sensor.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -329,14 +329,14 @@ def _create_buffers(self):
329329
body_labels = self._get_model_labels("body")
330330
shape_labels = self._get_model_labels("shape")
331331

332-
s_kind = self.contact_view.sensing_obj_type
332+
s_kind = self.contact_view.sensing_type
333333
if s_kind == "body":
334334
s_labels = body_labels
335335
elif s_kind == "shape":
336336
s_labels = shape_labels
337337
else:
338-
raise RuntimeError(f"Unexpected Newton sensing_obj_type {s_kind!r}; expected 'body' or 'shape'.")
339-
self._sensor_names = [s_labels[i].split("/")[-1] for i in self.contact_view.sensing_obj_idx]
338+
raise RuntimeError(f"Unexpected Newton sensing_type {s_kind!r}; expected 'body' or 'shape'.")
339+
self._sensor_names = [s_labels[i].split("/")[-1] for i in self.contact_view.sensing_indices]
340340
# Assumes the environments are processed in order.
341341
self._sensor_names = self._sensor_names[: self._num_sensors]
342342

source/isaaclab_newton/isaaclab_newton/sensors/contact_sensor/contact_sensor_cfg.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,8 @@ def from_base_cfg(cls, base_cfg: BaseContactSensorCfg, **kwargs) -> "ContactSens
6060
6161
Args:
6262
base_cfg: The base contact sensor configuration to copy from.
63-
**kwargs: Newton-specific fields, e.g. ``filter_shape_prim_expr=["fingertip_.*"]``.
63+
**kwargs: Newton-specific fields, e.g.
64+
``filter_shape_prim_expr=["{ENV_REGEX_NS}/Robot/fingertip_[^/]*/.*"]``.
6465
6566
Returns:
6667
A new :class:`ContactSensorCfg` instance.

source/isaaclab_newton/isaaclab_newton/sensors/ray_caster/newton_raycast_sensor.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,7 @@ class NewtonRaycastSensor(_NewtonRayCasterPoseMixin, BaseRayCaster):
273273
def __init__(self, cfg: NewtonRaycastSensorCfg):
274274
if cfg.max_distance <= 0.0:
275275
raise ValueError(f"max_distance must be positive, received {cfg.max_distance}.")
276+
NewtonManager._sensor_bvh_shape_flags |= newton.ShapeFlags.COLLIDE_SHAPES
276277
super().__init__(cfg)
277278
self._data = NewtonRaycastSensorData()
278279
self._sensor_task_name: str | None = None
@@ -326,7 +327,7 @@ def _initialize_impl(self) -> None:
326327
self._hit_normal = wp.empty(ray_count, dtype=wp.vec3f, device=self._device)
327328

328329
self._sensor_task_name = f"newton_raycast:{self.cfg.prim_path}:{id(self)}"
329-
NewtonManager._register_sensor_task(self._sensor_task_name, self._launch_raycast, include_collision_shapes=True)
330+
NewtonManager._register_sensor_task(self._sensor_task_name, self._launch_raycast)
330331

331332
def _launch_raycast(self) -> None:
332333
"""Sensor pose + ray transform + BVH query + hit resolve (graph-capturable)."""

source/isaaclab_newton/test/physics/test_newton_manager_abstraction.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525

2626
from __future__ import annotations
2727

28+
from inspect import signature
2829
from types import SimpleNamespace
2930

3031
import isaaclab_newton.physics.newton_manager as newton_manager_module
@@ -55,6 +56,7 @@
5556
XPBDSolverCfg,
5657
)
5758
from isaaclab_newton.physics.mpm_manager import _make_solver_config
59+
from newton import ShapeFlags
5860
from newton.solvers import SolverFeatherstone, SolverImplicitMPM, SolverKamino, SolverMuJoCo, SolverVBD, SolverXPBD
5961

6062
from isaaclab.physics import PhysicsManager
@@ -339,6 +341,30 @@ def render():
339341
assert status["rendered"]
340342

341343

344+
def test_sensor_bvh_shape_flags_are_fixed_before_builder_creation(monkeypatch):
345+
"""Builder finalization includes collision-only shapes without a later BVH rebuild."""
346+
import newton
347+
348+
flags = ShapeFlags.VISIBLE | ShapeFlags.COLLIDE_SHAPES
349+
monkeypatch.setattr(NewtonManager, "_sensor_bvh_shape_flags", flags)
350+
monkeypatch.setattr(PhysicsManager, "_cfg", NewtonCfg())
351+
builder = NewtonManager.create_builder()
352+
body = builder.add_body()
353+
builder.add_shape_sphere(body, cfg=newton.ModelBuilder.ShapeConfig(is_visible=False))
354+
355+
model = builder.finalize(device="cpu")
356+
357+
assert builder.default_bvh_cfg.shape_flags == flags
358+
assert model.bvh_shape_count_enabled == 1
359+
assert model.bvh_shapes is not None
360+
361+
362+
def test_sensor_task_registration_has_no_raycast_bvh_fallback():
363+
"""Raycast BVH requirements belong to builder creation, not task registration."""
364+
assert "include_collision_shapes" not in signature(NewtonManager._register_sensor_task).parameters
365+
assert not hasattr(NewtonManager, "_sensor_bvh_has_collision_shapes")
366+
367+
342368
def test_newton_shape_cfg_defaults_match_newton_shape_config():
343369
"""``NewtonShapeCfg`` contact defaults mirror Newton's ``ShapeConfig``.
344370
@@ -988,10 +1014,12 @@ def test_subclass_of_newton_manager(manager):
9881014
def test_clear_resets_rigid_body_force_capability(monkeypatch):
9891015
"""Teardown clears the canonical solver capability without subclass shadowing."""
9901016
monkeypatch.setattr(NewtonManager, "_supports_rigid_body_force_input", True)
1017+
monkeypatch.setattr(NewtonManager, "_sensor_bvh_shape_flags", ShapeFlags.COLLIDE_SHAPES)
9911018

9921019
NewtonManager.clear()
9931020

9941021
assert NewtonManager._supports_rigid_body_force_input is False
1022+
assert NewtonManager._sensor_bvh_shape_flags == ShapeFlags.VISIBLE
9951023
for manager in (
9961024
NewtonMJWarpManager,
9971025
NewtonXPBDManager,

0 commit comments

Comments
 (0)