Skip to content

Retune Newton and PhysX environments for Franka cloth lift - #6998

Merged
kellyguo11 merged 15 commits into
isaac-sim:developfrom
mmichelis:lift-cloth
Aug 13, 2026
Merged

Retune Newton and PhysX environments for Franka cloth lift#6998
kellyguo11 merged 15 commits into
isaac-sim:developfrom
mmichelis:lift-cloth

Conversation

@mmichelis

@mmichelis mmichelis commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds isaacsim_physx support to the Franka cloth lift tasks as a follow-up to #6831, while retaining newton_mjwarp_vbd_proxy as the default physics preset.

This PR:

  • Adds PhysX physics, deformable material, collision, and scene presets for the standard and camera cloth tasks.
  • Disables scene replication for PhysX deformables and increases the GPU found/lost pair capacity for large environments.
  • Matches backend-specific cloth density and material conventions, lowers support friction, adjusts reset clearance, and fully closes the gripper on the thin cloth.
  • Updates the state-machine demo to use the task-specific IK preset, allow enough time for the low-PD Franka, and grasp the raised cloth fold.
  • Updates the environment catalog and generated browser data to advertise PhysX support.
  • Restores the already released removal of NewtonCfg.simplify_meshes, which was reintroduced during branch integration, and records the Newton-only cleanup with a skip fragment.

No new dependencies are required.

Validation

  • uv run isaaclab -f
  • uv run python tools/changelog/cli.py check develop in a clean clone
  • uv run --isolated --extra test -- make -C docs current-docs in a clean clone
  • PYTHONPATH="$PWD/source/isaaclab_tasks:$PYTHONPATH" uv run python scripts/environments/state_machine/lift_franka_soft.py --task Isaac-Lift-Cloth-Franka --num_envs 1 --num_steps 500 --viz none presets=isaacsim_physx

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Documentation update

Screenshots

Not applicable.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab -f
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@mmichelis
mmichelis requested a review from a team August 10, 2026 11:40
@mmichelis mmichelis added documentation Improvements or additions to documentation enhancement New feature or request isaac-lab Related to Isaac Lab team labels Aug 10, 2026
@mmichelis mmichelis changed the title Add PhysX support to Franka cloth lift Retune Newton and PhysX environments for Franka cloth lift Aug 10, 2026
@greptile-apps

greptile-apps Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds PhysX presets for the Franka cloth-lifting tasks while retaining Newton as the default.

  • Adds backend-specific cloth physics, material, collision, and non-replicated scene configurations.
  • Updates the scripted lifting demo’s timing, IK selection, termination handling, and cloth grasp pose.
  • Advertises PhysX support in the environment documentation and generated browser data.
  • Removes the restored obsolete Newton option, adjusts rendering-test configuration, and refreshes editable package versions in the lockfile.

Confidence Score: 5/5

The PR appears safe to merge, with the PhysX preset wiring and scripted-demo changes consistent with existing deformable-task conventions.

The new backend aliases resolve coherently across physics, deformable, and scene configurations; PhysX scenes disable unsupported replication, Newton remains the default, and no concrete blocking or non-blocking defect remains.

Important Files Changed

Filename Overview
source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_cloth_env_cfg.py Adds coherent PhysX cloth, physics, material, collision, and scene presets while preserving the Newton default.
scripts/environments/state_machine/lift_franka_soft.py Updates the demo to select task-specific IK actions, allow longer motion, and target the raised cloth fold.
source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_soft_env_cfg.py Adjusts shared PhysX defaults and the beam gripper close command without exposing a concrete regression.
source/isaaclab_tasks/test/rendering_test_utils.py Preserves each resolved scene preset’s replication setting during deterministic camera-test setup.
source/isaaclab_newton/isaaclab_newton/physics/newton_manager_cfg.py Removes the obsolete Newton mesh-simplification configuration field.
uv.lock Refreshes editable Isaac Lab workspace-package versions without changing the affected third-party dependency versions.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  CLI["Task + preset selection"] --> Resolve["Resolve task configuration"]
  Resolve --> Newton["newton_mjwarp_vbd_proxy<br/>default"]
  Resolve --> PhysX["isaacsim_physx"]
  Newton --> NewtonScene["Replicated cloth scene<br/>Newton material"]
  PhysX --> PhysXScene["Non-replicated cloth scene<br/>PhysX material and collision"]
  NewtonScene --> Env["Franka cloth environment"]
  PhysXScene --> Env
  Env --> Demo["IK state-machine demo"]
  Env --> Train["RL workflow"]
  Env --> Camera["Camera task variant"]
Loading

Reviews (1): Last reviewed commit: "Add Newton changelog skip fragment" | Re-trigger Greptile

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The cloth-specific PhysX preset architecture and documentation are consistent, but two changes in the shared Franka soft-task configuration alter existing soft-beam PhysX and IK behavior without being required for cloth support or recorded in the changelog.

  • Design and architecture: The backend-specific physics, deformable, and scene presets correctly isolate cloth configuration and disable scene replication for PhysX deformables. However, the shared soft-beam base still receives two cloth-adjacent tuning changes even though the cloth environment overrides both relevant configuration paths.
  • API: The advertised isaacsim_physx, physx, and newton_mjwarp_vbd_proxy preset keys agree across task configuration, documentation, and browser data. The changelog covers newly added cloth support but does not disclose the changed PhysX contact defaults or IK gripper close width for the existing soft-beam tasks.
  • Implementation: FrankaClothEnvCfg.__post_init__ installs its own physics configuration and overrides the cloth IK close command to 0.0. Consequently, removing the shared PhysX friction parameters and changing the shared IK close width to 0.01 affect the soft-beam task rather than implementing cloth support; these values should be restored or the behavioral changes explicitly documented.

Minor fixes needed. Posted 2 actionable findings inline.

Automated review; human maintainers own approval decisions.

friction_offset_threshold=0.005,
friction_correlation_distance=0.01,
)
isaacsim_physx: PhysxCfg = PhysxCfg()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Implementation — Soft-beam PhysX friction tuning silently dropped

FrankaClothEnvCfg.__post_init__ replaces sim.physics with the cloth module's own PhysicsCfg, so this preset is consumed only by the already-released Isaac-Lift-Soft-Franka(-Camera) tasks. Clearing friction_offset_threshold=0.005 and friction_correlation_distance=0.01 reverts that task's PhysX contact tuning to defaults, is not needed for cloth PhysX support, and is not covered by the Added-only changelog fragment. Restore the values or record the change under Changed.

@@ -369,7 +366,7 @@ class _IkActionsCfg:
asset_name="robot",
joint_names=["panda_finger_joint1"],
open_command_expr={"panda_finger_joint1": 0.04},
close_command_expr={"panda_finger_joint1": 0.015},
close_command_expr={"panda_finger_joint1": 0.01},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Suggestion · Implementation — Shared IK gripper close width changed

_IkActionsCfg is shared with the soft-beam task, and the cloth env already overrides close_command_expr to 0.0 in its __post_init__. Narrowing the shared default from 0.015 to 0.01 therefore only alters the existing beam task's scripted grasp. Keep the shared value at 0.015 and confine cloth-specific tuning to the cloth config, or document the beam behavior change.

@kellyguo11

Copy link
Copy Markdown
Contributor

looks like a failure in rendering correctness test

| /workspace/isaaclab/source/isaaclab_tasks/test/core/test_rendering_franka_cloth.py     | cuda:0 | FAILED |   208.77 |   521.28 |  41/50  |

@kellyguo11 kellyguo11 moved this to In progress in Isaac Lab Aug 11, 2026
@kellyguo11 kellyguo11 moved this from In progress to In review in Isaac Lab Aug 11, 2026
@mmichelis
mmichelis force-pushed the lift-cloth branch 2 times, most recently from 5fd4ca0 to 7828001 Compare August 11, 2026 16:47
@kellyguo11
kellyguo11 merged commit 6bc3603 into isaac-sim:develop Aug 13, 2026
75 of 77 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in Isaac Lab Aug 13, 2026
@mmichelis
mmichelis deleted the lift-cloth branch August 13, 2026 07:10
nvsekkin added a commit that referenced this pull request Aug 25, 2026
# Description

Restore meaningful Franka cloth motion-vector regression coverage after
#6998 changed the task to clamp the cloth between supports.

- Raise the cloth by 1 cm for the Newton + OVRTX motion-vector case so
it undergoes visible motion.
- Preserve the OVRTX third-frame workaround tracked by NVBug 6565960.
- Re-enable Newton + OVRTX cloth motion-vector coverage and refresh its
golden.
- Scope the controlled fall and extra frame strictly to Newton + OVRTX,
leaving all other backend/renderer combinations unchanged.
- Keep Newton + Isaac RTX quarantined under its existing nondeterminism
policy.

The third-frame workaround can be removed after the OVRTX fix is
available in a post-GA release.

## Type of change

- Bug fix (non-breaking change which fixes an issue)

## Screenshots
Updated Newton + OVRTX motion-vector golden:
![Updated Franka cloth motion-vector
golden](https://github.com/nvsekkin/IsaacLab/blob/esekkin/cloth-motion-vectors/source/isaaclab_tasks/test/golden_images/franka_cloth/newton-ovrtx_renderer-motion_vectors.png?raw=true)


## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/` for every touched package (do **not** edit
`CHANGELOG.rst` or bump `extension.toml` — CI handles that)
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`
isaaclab-bot Bot pushed a commit that referenced this pull request Aug 25, 2026
# Description

Restore meaningful Franka cloth motion-vector regression coverage after
#6998 changed the task to clamp the cloth between supports.

- Raise the cloth by 1 cm for the Newton + OVRTX motion-vector case so
it undergoes visible motion.
- Preserve the OVRTX third-frame workaround tracked by NVBug 6565960.
- Re-enable Newton + OVRTX cloth motion-vector coverage and refresh its
golden.
- Scope the controlled fall and extra frame strictly to Newton + OVRTX,
leaving all other backend/renderer combinations unchanged.
- Keep Newton + Isaac RTX quarantined under its existing nondeterminism
policy.

The third-frame workaround can be removed after the OVRTX fix is
available in a post-GA release.

## Type of change

- Bug fix (non-breaking change which fixes an issue)

## Screenshots
Updated Newton + OVRTX motion-vector golden:
![Updated Franka cloth motion-vector
golden](https://github.com/nvsekkin/IsaacLab/blob/esekkin/cloth-motion-vectors/source/isaaclab_tasks/test/golden_images/franka_cloth/newton-ovrtx_renderer-motion_vectors.png?raw=true)


## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/` for every touched package (do **not** edit
`CHANGELOG.rst` or bump `extension.toml` — CI handles that)
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

(cherry picked from commit 553efa6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request isaac-lab Related to Isaac Lab team

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants