Skip to content

[FEAT] Add OVPhysX support to Franka deformable lift tasks - #7077

Merged
kellyguo11 merged 4 commits into
isaac-sim:developfrom
mmichelis:mmichelis/franka-deformable-ovphysx
Aug 20, 2026
Merged

[FEAT] Add OVPhysX support to Franka deformable lift tasks#7077
kellyguo11 merged 4 commits into
isaac-sim:developfrom
mmichelis:mmichelis/franka-deformable-ovphysx

Conversation

@mmichelis

@mmichelis mmichelis commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds ovphysx physics presets to the Franka soft-body and cloth lift tasks, including their camera variants. The presets reuse the existing PhysX deformable schemas and materials, enable scene replication for OVPhysX, and leave the existing Isaac Sim PhysX and Newton presets unchanged.

OVPhysX does not currently expose a runtime gravity setter, so its preset disables the variable-gravity event and gravity curriculum and trains at the configured fixed gravity. The Isaac Sim PhysX preset retains the existing gravity curriculum.

Short RSL-RL smoke runs completed for 10 iterations with 16 environments and seed 42. To account for the gravity limitation, OVPhysX was compared with Isaac Sim PhysX using the same fixed gravity:

Task OVPhysX mean reward Isaac Sim PhysX mean reward OVPhysX throughput Isaac Sim PhysX throughput
Cloth lift 3.90 4.08 259 steps/s 324 steps/s
Soft lift 1.97 2.03 296 steps/s 339 steps/s

All runs completed 3,840 steps without NaNs, out-of-bounds terminations, or joint-velocity-limit terminations. These short runs validate configuration and training stability, not converged training parity.

No new external dependencies are required.

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Not applicable.

Validation

  • 10-iteration RSL-RL smoke runs for soft-body and cloth lift with OVPhysX and fixed-gravity Isaac Sim PhysX
  • Python syntax checks for the changed modules
  • uv run python tools/changelog/cli.py check develop
  • uv run isaaclab -f
  • git diff --check

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab -f
  • Documentation changes are not required for this preset-only change
  • My changes generate no new warnings
  • I have added tests that prove my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • My name already exists in CONTRIBUTORS.md

@mmichelis
mmichelis requested a review from a team August 13, 2026 16:10
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 13, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

Adds OVPhysX presets for Franka soft-body and cloth lift tasks, including camera variants, replicated scenes, fixed-gravity handling, tests, and a changelog fragment. One inconsistent path remains: the generic physx preset can now auto-select OVPhysX while retaining gravity terms that OVPhysX cannot execute.

  • Design and architecture: The backend-specific preset structure consistently reuses the existing PhysX deformable schemas and materials and enables scene replication for OVPhysX. However, the gravity workaround is keyed only to the explicit ovphysx preset, while PhysxAutoCfg now permits the generic physx branch to select OVPhysX, leaving that auto-selection path inconsistent.
  • API: The explicit ovphysx, isaacsim_physx, and Newton preset surfaces remain coherent, and the changelog records the new task presets. The generic physx compatibility surface needs correction because its newly expanded backend selection can expose unsupported gravity behavior.
  • Implementation: The explicit OVPhysX wiring across physics, deformable objects, scenes, and camera scenes is covered by preset-resolution tests, and Isaac Sim PhysX retains its gravity curriculum. The untested generic physx branch maps events and curriculum to gravity-enabled configurations even though its PhysxAutoCfg can now resolve to OVPhysX; it should use gravity-free terms when OVPhysX is selected.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

ovphysx: OvPhysxCfg = OvPhysxCfg()

physx: PhysxAutoCfg = PhysxAutoCfg(isaacsim_physx=isaacsim_physx)
physx: PhysxAutoCfg = PhysxAutoCfg(isaacsim_physx=isaacsim_physx, ovphysx=ovphysx)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Design Architecture — Auto physx preset keeps unsupported gravity terms

PhysxAutoCfg now also carries ovphysx (and likewise at line 97 in the cloth config), so selecting the generic physx preset can resolve to OVPhysX. The new events/curriculum presets map physx to the unmodified configs (lines 663, 672), leaving variable_gravity and the gravity curriculum active on a backend that this PR states has no runtime gravity setter. Map physx to the gravity-free variants, or key the workaround on the resolved backend.

@greptile-apps

greptile-apps Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds OVPhysX physics, deformable, and replicated-scene presets for Franka soft-body and cloth lift tasks, including camera variants. It also disables unsupported runtime gravity changes for the explicit OVPhysX selection and adds focused configuration tests.

  • Adds OvPhysxCfg alternatives while retaining existing Isaac Sim PhysX and Newton presets.
  • Reuses the existing PhysX deformable schemas and materials for OVPhysX.
  • Enables replicated physics for OVPhysX deformable scenes.
  • Tests backend resolution, scene replication, schema selection, and gravity-term behavior.

Confidence Score: 4/5

The broad physx path needs to disable gravity terms whenever it auto-selects OVPhysX before this is safe to merge.

In Kit-less training, PhysxAutoCfg selects OVPhysX while the parallel physx event and curriculum presets retain a runtime gravity operation that requires the unavailable PhysX simulation-view setter.

Files Needing Attention: source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_soft_env_cfg.py

Important Files Changed

Filename Overview
source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_soft_env_cfg.py Adds OVPhysX physics, deformable, scene, and gravity presets, but the broad physx path can auto-select OVPhysX without disabling unsupported gravity changes.
source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_cloth_env_cfg.py Adds OVPhysX cloth physics and replicated scene alternatives that inherit the shared gravity-preset behavior from the soft environment.
source/isaaclab_tasks/test/core/test_franka_deformable_ovphysx_cfg.py Adds focused tests for explicit OVPhysX and Isaac Sim PhysX selections but does not cover Kit-less auto-selection through the broad physx preset.
source/isaaclab_tasks/changelog.d/mmichelis-franka-deformable-ovphysx.minor.rst Documents the new OVPhysX presets for soft-body, cloth, and camera task variants.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Select physics preset] --> B{Preset name}
    B -->|ovphysx| C[OvPhysxCfg]
    B -->|physx| D[PhysxAutoCfg]
    D --> E{Kit available?}
    E -->|Yes| F[Isaac Sim PhysX]
    E -->|No| C
    C --> G[OVPhysX runtime]
    B -->|ovphysx| H[Gravity event disabled]
    B -->|physx| I[Gravity event retained]
    I --> J[Unsupported runtime gravity update]
Loading

Reviews (1): Last reviewed commit: "Add OVPhysX support to Franka deformable..." | Re-trigger Greptile

default_events = self.events
self.events = preset(
default=default_events,
physx=default_events,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 PhysX auto-selection retains gravity

When a Kit-less training run selects the broad physx preset, PhysxAutoCfg chooses OVPhysX but this variant retains variable_gravity and the gravity curriculum, causing resets to invoke a runtime gravity setter that OVPhysX does not provide and abort training.

Knowledge Base Used: isaaclab_tasks: Task Registration and Organization

@kellyguo11
kellyguo11 merged commit a9eb1ff into isaac-sim:develop Aug 20, 2026
55 of 74 checks passed
kellyguo11 added a commit that referenced this pull request Aug 21, 2026
## Summary

Cherry-picks the following merged PRs from `develop` onto
`release/3.0.0`, preserving each as an individual commit with `-x`
provenance:

- #7193 — Bake CI pytest deps into the built Docker image
- #7077 — Add OVPhysX support to Franka deformable lift tasks
- #7175 — Raise on unsupported camera renderer outputs
- #7174 — Fix benchmark play inference scope
- #7173 — Report resolved backend for play benchmarks
- #7172 — Add video recording to play benchmarks
- #7215 — Reduce GearAssembly default environment count
- #7214 — Remove Kamino preset from open drawer
- #7213 — Fix OvPhysX scene gravity randomization
- #7212 — Fix OVPhysX material binding device selection
- #7189 — Make preview surfaces renderer agnostic
- #7206 — Fix OVPhysX benchmark articulation setup

All cherry-picks and the final rebase onto the latest `release/3.0.0`
tip completed without conflicts.

## Validation

- Verified all 12 backported commits have patch IDs identical to their
source squash commits and retain their `cherry picked from` footers.
- `git diff --check upstream/release/3.0.0..HEAD`
- Bash syntax validation for the modified CI shell scripts.
- YAML parsing for the modified composite actions.
- Changelog validation against `release/3.0.0`.
- Targeted pytest coverage for benchmark APIs/play, video recording,
GearAssembly defaults, OVPhysX presets and runtime semantics, and
renderer-agnostic materials: **81 passed, 4 skipped, 1 deselected**.
- The deselected case is an unchanged test that hard-codes a POSIX
`/tmp` path and fails on Windows path normalization.
- `uv run isaaclab -f` passed all hooks except the changelog hook's
known release-branch baseline finding for
`source/isaaclab/changelog.d/core-test-config-fixtures.skip`; the
targeted release-base changelog check passed.

---------

Co-authored-by: Matthew Taylor <mataylor@nvidia.com>
Co-authored-by: Antoine RICHARD <antoiner@nvidia.com>
Co-authored-by: Mike Yan Michelis <46975745+mmichelis@users.noreply.github.com>
Co-authored-by: Maximilian Krause <99733341+maxkra15@users.noreply.github.com>
Co-authored-by: Octi Zhang <zhengyuz@nvidia.com>
Co-authored-by: Mustafa H <34825877+StafaH@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants