[FEAT] Add OVPhysX support to Franka deformable lift tasks - #7077
Conversation
There was a problem hiding this comment.
Isaac Lab Review Bot
Adds OVPhysX presets for Franka soft-body and cloth lift tasks, including camera variants, replicated scenes, fixed-gravity handling, tests, and a changelog fragment. One inconsistent path remains: the generic physx preset can now auto-select OVPhysX while retaining gravity terms that OVPhysX cannot execute.
- Design and architecture: The backend-specific preset structure consistently reuses the existing PhysX deformable schemas and materials and enables scene replication for OVPhysX. However, the gravity workaround is keyed only to the explicit
ovphysxpreset, whilePhysxAutoCfgnow permits the genericphysxbranch to select OVPhysX, leaving that auto-selection path inconsistent. - API: The explicit
ovphysx,isaacsim_physx, and Newton preset surfaces remain coherent, and the changelog records the new task presets. The genericphysxcompatibility surface needs correction because its newly expanded backend selection can expose unsupported gravity behavior. - Implementation: The explicit OVPhysX wiring across physics, deformable objects, scenes, and camera scenes is covered by preset-resolution tests, and Isaac Sim PhysX retains its gravity curriculum. The untested generic
physxbranch maps events and curriculum to gravity-enabled configurations even though itsPhysxAutoCfgcan now resolve to OVPhysX; it should use gravity-free terms when OVPhysX is selected.
Minor fixes needed. Posted 1 actionable finding inline.
Automated review; human maintainers own approval decisions.
| ovphysx: OvPhysxCfg = OvPhysxCfg() | ||
|
|
||
| physx: PhysxAutoCfg = PhysxAutoCfg(isaacsim_physx=isaacsim_physx) | ||
| physx: PhysxAutoCfg = PhysxAutoCfg(isaacsim_physx=isaacsim_physx, ovphysx=ovphysx) |
There was a problem hiding this comment.
🟡 Warning · Design Architecture — Auto physx preset keeps unsupported gravity terms
PhysxAutoCfg now also carries ovphysx (and likewise at line 97 in the cloth config), so selecting the generic physx preset can resolve to OVPhysX. The new events/curriculum presets map physx to the unmodified configs (lines 663, 672), leaving variable_gravity and the gravity curriculum active on a backend that this PR states has no runtime gravity setter. Map physx to the gravity-free variants, or key the workaround on the resolved backend.
Greptile SummaryThe PR adds OVPhysX physics, deformable, and replicated-scene presets for Franka soft-body and cloth lift tasks, including camera variants. It also disables unsupported runtime gravity changes for the explicit OVPhysX selection and adds focused configuration tests.
Confidence Score: 4/5The broad In Kit-less training, Files Needing Attention: source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_soft_env_cfg.py Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Select physics preset] --> B{Preset name}
B -->|ovphysx| C[OvPhysxCfg]
B -->|physx| D[PhysxAutoCfg]
D --> E{Kit available?}
E -->|Yes| F[Isaac Sim PhysX]
E -->|No| C
C --> G[OVPhysX runtime]
B -->|ovphysx| H[Gravity event disabled]
B -->|physx| I[Gravity event retained]
I --> J[Unsupported runtime gravity update]
Reviews (1): Last reviewed commit: "Add OVPhysX support to Franka deformable..." | Re-trigger Greptile |
| default_events = self.events | ||
| self.events = preset( | ||
| default=default_events, | ||
| physx=default_events, |
There was a problem hiding this comment.
PhysX auto-selection retains gravity
When a Kit-less training run selects the broad physx preset, PhysxAutoCfg chooses OVPhysX but this variant retains variable_gravity and the gravity curriculum, causing resets to invoke a runtime gravity setter that OVPhysX does not provide and abort training.
Knowledge Base Used: isaaclab_tasks: Task Registration and Organization
## Summary Cherry-picks the following merged PRs from `develop` onto `release/3.0.0`, preserving each as an individual commit with `-x` provenance: - #7193 — Bake CI pytest deps into the built Docker image - #7077 — Add OVPhysX support to Franka deformable lift tasks - #7175 — Raise on unsupported camera renderer outputs - #7174 — Fix benchmark play inference scope - #7173 — Report resolved backend for play benchmarks - #7172 — Add video recording to play benchmarks - #7215 — Reduce GearAssembly default environment count - #7214 — Remove Kamino preset from open drawer - #7213 — Fix OvPhysX scene gravity randomization - #7212 — Fix OVPhysX material binding device selection - #7189 — Make preview surfaces renderer agnostic - #7206 — Fix OVPhysX benchmark articulation setup All cherry-picks and the final rebase onto the latest `release/3.0.0` tip completed without conflicts. ## Validation - Verified all 12 backported commits have patch IDs identical to their source squash commits and retain their `cherry picked from` footers. - `git diff --check upstream/release/3.0.0..HEAD` - Bash syntax validation for the modified CI shell scripts. - YAML parsing for the modified composite actions. - Changelog validation against `release/3.0.0`. - Targeted pytest coverage for benchmark APIs/play, video recording, GearAssembly defaults, OVPhysX presets and runtime semantics, and renderer-agnostic materials: **81 passed, 4 skipped, 1 deselected**. - The deselected case is an unchanged test that hard-codes a POSIX `/tmp` path and fails on Windows path normalization. - `uv run isaaclab -f` passed all hooks except the changelog hook's known release-branch baseline finding for `source/isaaclab/changelog.d/core-test-config-fixtures.skip`; the targeted release-base changelog check passed. --------- Co-authored-by: Matthew Taylor <mataylor@nvidia.com> Co-authored-by: Antoine RICHARD <antoiner@nvidia.com> Co-authored-by: Mike Yan Michelis <46975745+mmichelis@users.noreply.github.com> Co-authored-by: Maximilian Krause <99733341+maxkra15@users.noreply.github.com> Co-authored-by: Octi Zhang <zhengyuz@nvidia.com> Co-authored-by: Mustafa H <34825877+StafaH@users.noreply.github.com>
Description
Adds
ovphysxphysics presets to the Franka soft-body and cloth lift tasks, including their camera variants. The presets reuse the existing PhysX deformable schemas and materials, enable scene replication for OVPhysX, and leave the existing Isaac Sim PhysX and Newton presets unchanged.OVPhysX does not currently expose a runtime gravity setter, so its preset disables the variable-gravity event and gravity curriculum and trains at the configured fixed gravity. The Isaac Sim PhysX preset retains the existing gravity curriculum.
Short RSL-RL smoke runs completed for 10 iterations with 16 environments and seed 42. To account for the gravity limitation, OVPhysX was compared with Isaac Sim PhysX using the same fixed gravity:
All runs completed 3,840 steps without NaNs, out-of-bounds terminations, or joint-velocity-limit terminations. These short runs validate configuration and training stability, not converged training parity.
No new external dependencies are required.
Type of change
Screenshots
Not applicable.
Validation
uv run python tools/changelog/cli.py check developuv run isaaclab -fgit diff --checkChecklist
pre-commitchecks withuv run isaaclab -fsource/<pkg>/changelog.d/for every touched packageCONTRIBUTORS.md