[omniperf-agent] fix: PhysX KukaAllegro grasp via compliant contacts - #7511
[omniperf-agent] fix: PhysX KukaAllegro grasp via compliant contacts#7511yts-nv wants to merge 6 commits into
Conversation
Under rigid PhysX contacts the manipulated object is squirted out of the dexterous KukaAllegro hand before a sustained thumb+finger pinch forms, so the good_finger_contact grasp reward stays 0 and PPO never learns to lift; the object leaves the workspace and episodes terminate early. Newton (soft-constraint contacts) converges on the same task. Fix (core/lift/lift_env_cfg.py): - give the object a PhysX compliant (soft) contact material so the fingers can hold it (PhysxRigidBodyMaterialCfg, stiffness=300, damping=30); - disable the startup material domain randomization that rebuilds+rebinds the object material at startup and would otherwise discard the compliant spawn material (that DR path does not support compliant contact). Validated on this file (Isaac-Lift-KukaAllegro, env seed 42): success_rate 0 -> 0.40 @2000 iters, 0.78 @4000 (Newton-level), reproduced 2x. Note: OBJECT_PHYSICS / EventCfg here are shared by other lift configs (e.g. franka); reviewer should confirm desired scope. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…g DR (true fix) Replaces the earlier DR-disable workaround with a fix that keeps domain randomization AND compliant contact on PhysX. Root cause: the object needs PhysX compliant (soft) contacts to be graspable by the dexterous KukaAllegro hand, but randomize_rigid_body_material writes the object's material through the rigid 3-tuple API (RigidBodyView.set_material_properties: static/dynamic friction + restitution), overwriting the compliant contact authored at spawn -> rigid contacts -> the object is ejected before a grasp forms -> the policy never converges. Newton is unaffected: its DR touches only friction bindings and its contacts are soft by default. Fix (core/lift): - give the object a PhysX compliant material (PhysxRigidBodyMaterialCfg, stiffness=300, damping=30); - add core/lift/mdp/compliant_events.py: RandomizeFrictionKeepCompliant, a startup event that randomizes friction and writes it back through RigidBodyView.set_compliant_material_properties (the 4-tuple: friction + stiffness + damping), so compliant contact survives DR; - wire object_physics_material to it; robot_physics_material keeps the stock DR. Keeps full domain randomization (object friction still randomized) while preserving compliant contact. Validated (Isaac-Lift-KukaAllegro, PhysX, seed 42, 4096 envs): success_rate 0 -> 0.637 @2000 iters, Success Passed=1 (converged) -- higher than the DR-disable variant (0.40), because full randomization yields a more robust grasp. Validated on develop 8c3bc5e; targets current develop. Note: implemented task-scoped (a core/lift/mdp event); could be upstreamed into core randomize_rigid_body_material so all tasks benefit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Greptile SummaryThis PR fixes failed PhysX grasp learning in the KukaAllegro lift task by authoring compliant object contacts and preserving their stiffness and damping while friction is randomized.
Confidence Score: 4/5The backend-specific compliant-material event must be limited to PhysX or given backend-compatible behavior before this PR is safe to merge. The shared lift configuration invokes compliant-material methods during startup even for selectable backends such as OvPhysX whose root view does not implement those methods, preventing affected environments from initializing. Files Needing Attention: source/isaaclab_tasks/isaaclab_tasks/core/lift/lift_env_cfg.py, source/isaaclab_tasks/isaaclab_tasks/core/lift/mdp/compliant_events.py Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Lift EventCfg startup] --> B[RandomizeFrictionKeepCompliant]
B --> C{Selected backend}
C -->|PhysX| D[Compliant material tensor API]
C -->|OvPhysX or Newton| E[Backend root view]
E --> F[Unsupported compliant-material call]
F --> G[Environment startup failure]
Reviews (1): Last reviewed commit: "[omniperf-agent] fix: PhysX KukaAllegro ..." | Re-trigger Greptile |
|
|
||
| object_physics_material = EventTerm( | ||
| func=mdp.randomize_rigid_body_material, | ||
| func=RandomizeFrictionKeepCompliant, |
There was a problem hiding this comment.
Backend-specific startup event
When a lift environment uses the OvPhysX or default Newton backend, the shared startup event calls PhysX-specific compliant-material methods on that backend's root view, causing environment initialization to fail. Install this event only for the PhysX configuration or provide backend-compatible behavior.
Knowledge Base Used: Physics backend extensions
There was a problem hiding this comment.
Isaac Lab Review Bot
The compliant-contact fix is currently applied through backend-shared lift configuration, introducing PhysX-specific material and tensor APIs into Newton and OV startup paths. The new event also retains configuration parameters whose semantics it does not implement, and the new source is missing required repository metadata and includes experimental logging. A package changelog fragment is also required.
- Design and architecture:
EventCfgand the shared object presets serve multiple physics backends, but now directly useRandomizeFrictionKeepCompliantandPhysxRigidBodyMaterialCfg. The compliant material and event should be scoped to the PhysX-specific configuration while shared paths retain backend-neutral materials and backend-dispatched randomization. - API: The replacement event accepts the stock randomizer's parameters but ignores
restitution_range,num_buckets,env_ids, and the call-timeasset_cfg, causing configured behavior to be silently dropped. Its supported parameter contract should match its implementation, with callers updated accordingly, or the missing semantics should be implemented. - Implementation: The compliant material write path is coherent, but the new module lacks the required BSD-3-Clause SPDX header and contains an experimental
[TRUEFIX]stdout message that should not remain in production. Because this visibly changesisaaclab_tasksphysics behavior, the required changelog fragment or.skipfragment must also be added.
Needs rework. Posted 5 actionable findings inline.
Automated review; human maintainers own approval decisions.
|
|
||
| object_physics_material = EventTerm( | ||
| func=mdp.randomize_rigid_body_material, | ||
| func=RandomizeFrictionKeepCompliant, |
There was a problem hiding this comment.
🔴 Critical · Design Architecture — PhysX-only event term in backend-shared EventCfg
EventCfg is documented as shared by all physics backends, yet RandomizeFrictionKeepCompliant calls root_view.max_shapes, get_compliant_material_properties() and set_compliant_material_properties(), which are PhysX RigidBodyView APIs. Newton and OV runs of this task now hit that startup term instead of the backend-dispatching mdp.randomize_rigid_body_material. Keep the shared config on the stock term and apply the compliant variant only in the PhysX-specific configuration.
|
|
||
| OBJECT_PHYSICS = { | ||
| "physics_material": RigidBodyMaterialCfg(static_friction=0.5), | ||
| "physics_material": PhysxRigidBodyMaterialCfg(static_friction=0.5, compliant_contact_stiffness=300.0, compliant_contact_damping=30.0), |
There was a problem hiding this comment.
🟡 Warning · Design Architecture — PhysX material type in shared spawn presets
OBJECT_PHYSICS (and ObjectCfg.cube at line 85) now spawn with PhysxRigidBodyMaterialCfg instead of the backend-neutral RigidBodyMaterialCfg, but these presets feed default = shapes and ovphysx = cube for all backends. This changes the authored material type for non-PhysX spawners. Override the material in the PhysX configuration and leave the shared preset backend-neutral.
| @@ -0,0 +1,50 @@ | |||
| # PhysX "true fix" experiment: randomize friction while PRESERVING compliant contact, | |||
There was a problem hiding this comment.
🟡 Warning · Implementation — Missing SPDX header and leftover experiment artifacts
New source files must carry the repository SPDX header used by sibling modules such as lift_env_cfg.py; this file instead opens with an "experiment" note and ends with an unconditional [TRUEFIX] stdout print guarded by a _logged flag. Add the standard BSD-3-Clause header, remove the debug print, and rewrite the header comment to state the intent of preserving compliant contact.
| self._dr = tuple(cfg.params.get("dynamic_friction_range", (0.5, 1.0))) | ||
| self._logged = False | ||
|
|
||
| def __call__(self, env, env_ids, static_friction_range=None, dynamic_friction_range=None, |
There was a problem hiding this comment.
🟡 Warning · Api — Term silently ignores configured randomization parameters
__call__ accepts env_ids, restitution_range, num_buckets, and asset_cfg but uses none of them: every environment and shape is written, restitution is never applied, and the configured num_buckets=250 friction discretization is dropped. Callers editing these params in EventCfg will see no effect and different behavior from the term being replaced. Honor the parameters or remove them from both the signature and the config.
| cube = sim_utils.CuboidCfg( | ||
| size=(0.05, 0.05, 0.05), | ||
| physics_material=RigidBodyMaterialCfg(static_friction=0.5), | ||
| physics_material=PhysxRigidBodyMaterialCfg(static_friction=0.5, compliant_contact_stiffness=300.0, compliant_contact_damping=30.0), |
There was a problem hiding this comment.
🔵 Suggestion · Implementation — Missing changelog fragment for isaaclab_tasks
This changes user-visible lift task physics (compliant object material and a different startup randomization term) in the isaaclab_tasks package, but no fragment was added under source/isaaclab_tasks/changelog.d/. Repository rules require one past-tense fragment per touched source package, or a .skip fragment when no release note is needed.
…ing DR fix Follow IsaacLab contribution format for the PhysX KukaAllegro compliant-contact fix: - add isaaclab_tasks CHANGELOG.rst entry (20.0.1) describing the fix; - bump isaaclab_tasks version 20.0.0 -> 20.0.1; - add the standard copyright/SPDX header + class docstring to core/lift/mdp/compliant_events.py and drop the debug print. No functional change to the fix itself. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…x general - Make ``RandomizeFrictionKeepCompliant`` backend-aware: PhysX uses ``RigidBodyView.set_compliant_material_properties``; Newton / OVPhysX delegate to the stock ``randomize_rigid_body_material``. The previous version called a PhysX-only tensor API unconditionally and would crash under the ``newton_mjwarp`` preset. - Anchor the compliant stiffness/damping to ``2500`` / ``100`` = Newton's own default rigid contact ``ke`` / ``kd``, so both backends run the identical, tuned contact model. This is more physical than the previous ``300`` / ``30`` (0.78 mm vs 6.5 mm static penetration for the 0.2 kg object) and converges higher on PhysX: success_rate 0.843 vs 0.637 @2000 iters, Passed 2x @4000. Compliant contact is a Kelvin-Voigt penalty spring-damper (stiffness [N/m], damping [N*s/m]) -- the same quantity as Newton's contact ke/kd, not a material Young's modulus. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ength) Run the repo's pinned ruff (v0.14.10) format + lint --fix on the two changed files so they pass pre-commit at line-length 120: wrap the long ``PhysxRigidBodyMaterialCfg`` material declarations and the friction-sampling calls. No functional change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Description
What & why
On
Isaac-Lift-KukaAllegro, the PhysX backend never learns to grasp (success_rate ~ 0) while Newton converges. Root cause is a domain-randomization interaction, not a reward/sensor bug:randomize_rigid_body_materialwrites the object's material through the rigid 3-tuple API —RigidBodyView.set_material_properties(static/dynamic friction + restitution) — which overwrites the compliant contact authored at spawn. -> rigid contacts -> the object is squirted out of the hand before a grasp forms ->good_finger_contactstays 0 -> the policy never converges.shape_material_mu/restitution), leaving contact stiffness/damping alone, and its contacts are soft by default.Fix (
core/lift)PhysxRigidBodyMaterialCfg(static_friction=0.5, compliant_contact_stiffness=300.0, compliant_contact_damping=30.0).core/lift/mdp/compliant_events.py->RandomizeFrictionKeepCompliant: a startup event that randomizes friction and writes it back throughRigidBodyView.set_compliant_material_properties(the 4-tuple: friction + stiffness + damping), so compliant contact survives domain randomization.object_physics_materialto it;robot_physics_materialkeeps the stock DR.Keeps full domain randomization (object friction still randomized) and preserves compliant contact — no engine change needed; the compliant setter already exists in
omni.physics.tensors.Validation (PhysX, seed 42, 4096 envs)
Converges higher than disabling DR (0.637 vs 0.40) because retaining randomization yields a more robust grasp. Validated on develop
8c3bc5e; targets current develop.Note for reviewers
Implemented task-scoped (a
core/lift/mdpevent). The same idea could be upstreamed into the corerandomize_rigid_body_material(write compliant viaset_compliant_material_propertieswhen the material has compliant fields) so all tasks benefit.Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Release backport
developScreenshots
Please attach before and after screenshots of the change if applicable.
Checklist
Docker and GPU tests run on demand. Push the commits you want tested, then
comment
run-cion the pull request.pre-commitchecks with./isaaclab.sh --formatsource/<pkg>/changelog.d/for every touched package (do not editCHANGELOG.rstor bumpextension.toml— CI handles that)CONTRIBUTORS.mdor my name already exists there