Skip to content

Commit d44331e

Browse files
authored
Add 5 more kitchen envs and droid stand xy scaling supports (#1149)
## Summary Short description of the change (max 50 chars) ## Detailed description - Added stand xy scaling into droid params, allowing fitting kitchens with narrower stand footprint - Added LW kitchen envs: 1x multi-level PnP, 1 open-cabinet(hard), 2x fridge (left door) opening, 1x freezer(right door) opening. - Tested with PI, only cabinet env has 0 SR but small joint (0.25) movement --------- Signed-off-by: Xinjie Yao <xyao@nvidia.com>
1 parent 149ff46 commit d44331e

16 files changed

Lines changed: 535 additions & 19 deletions
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading

docs/pages/example_workflows/example_environments.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Kitchen benchmark environment graph YAMLs live under
3535
``isaaclab_arena_environments/kitchen_bench/``. They define DROID manipulation
3636
tasks across Lightwheel RoboCasa and Replicator kitchen layouts.
3737

38-
See :doc:`kitchen_bench_catalog` for all 21 environment specs and their Pi
38+
See :doc:`kitchen_bench_catalog` for all 26 environment specs and their Pi
3939
policy executions.
4040

4141
Python Environment Catalog

docs/pages/example_workflows/kitchen_bench_catalog.rst

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Kitchen Benchmark Catalog
22
=========================
33

4-
The kitchen benchmark contains 21 DROID tasks defined as environment graph
4+
The kitchen benchmark contains 26 DROID tasks defined as environment graph
55
YAML specs under ``isaaclab_arena_environments/kitchen_bench/``. Pass a spec
66
to ``policy_runner.py`` with ``--env_spec`` to run that environment.
77
Each row links to the source YAML and shows a Pi policy execution recorded from
@@ -14,10 +14,26 @@ the maintained DROID external camera.
1414

1515
* - Environment graph YAML
1616
- Pi policy execution
17+
* - `kitchen_bench_lightwheel_open_cabinet_u_shaped_with_island_farmhouse2.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_open_cabinet_u_shaped_with_island_farmhouse2.yaml>`_
18+
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_open_cabinet_u_shaped_with_island_farmhouse2.gif
19+
:alt: Pi policy opening a cabinet in the U-shaped farmhouse kitchen
20+
:width: 100%
21+
* - `kitchen_bench_lightwheel_open_freezer_g_shaped_large_scandinavian.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_open_freezer_g_shaped_large_scandinavian.yaml>`_
22+
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_open_freezer_g_shaped_large_scandinavian.gif
23+
:alt: Pi policy opening the freezer in the G-shaped Scandinavian kitchen
24+
:width: 100%
1725
* - `kitchen_bench_lightwheel_open_fridge.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_open_fridge.yaml>`_
1826
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_open_fridge_pi.gif
1927
:alt: Pi policy opening the refrigerator in the Lightwheel kitchen
2028
:width: 100%
29+
* - `kitchen_bench_lightwheel_open_fridge_g_shaped_large_scandinavian.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_open_fridge_g_shaped_large_scandinavian.yaml>`_
30+
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_open_fridge_g_shaped_large_scandinavian.gif
31+
:alt: Pi policy opening the refrigerator in the G-shaped Scandinavian kitchen
32+
:width: 100%
33+
* - `kitchen_bench_lightwheel_open_fridge_u_shaped_with_island_farmhouse2.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_open_fridge_u_shaped_with_island_farmhouse2.yaml>`_
34+
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_open_fridge_u_shaped_with_island_farmhouse2.gif
35+
:alt: Pi policy opening the refrigerator in the U-shaped farmhouse kitchen
36+
:width: 100%
2137
* - `kitchen_bench_lightwheel_open_microwave.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_open_microwave.yaml>`_
2238
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_open_microwave_pi.gif
2339
:alt: Pi policy opening the microwave in the Lightwheel kitchen
@@ -26,6 +42,10 @@ the maintained DROID external camera.
2642
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_pick_and_place_pi.gif
2743
:alt: Pi policy placing mustard in a bowl in the Lightwheel kitchen
2844
:width: 100%
45+
* - `kitchen_bench_lightwheel_pick_lemon_shelf_on_plate_u_shaped_with_island_farmhouse2.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_pick_lemon_shelf_on_plate_u_shaped_with_island_farmhouse2.yaml>`_
46+
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_pick_lemon_shelf_on_plate_lightwheel_kitchen_u_shaped_with_island_farmhouse2.gif
47+
:alt: Pi policy picking a lemon from a shelf onto a plate in the U-shaped farmhouse kitchen
48+
:width: 100%
2949
* - `kitchen_bench_lightwheel_place_bananas_in_wooden_bowl_u_shaped_with_island_farmhouse2.yaml <https://github.com/isaac-sim/IsaacLab-Arena/blob/main/isaaclab_arena_environments/kitchen_bench/kitchen_bench_lightwheel_place_bananas_in_wooden_bowl_u_shaped_with_island_farmhouse2.yaml>`_
3050
- .. image:: ../../images/kitchen_bench/kitchen_bench_lightwheel_place_bananas_in_wooden_bowl_u_shaped_with_island_farmhouse2_pi.gif
3151
:alt: Pi policy placing bananas in a wooden bowl in the U-shaped farmhouse kitchen

isaaclab_arena/embodiments/droid/droid.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@
5757
ref_prim_path="/World/franka_table",
5858
payload_child_name="franka_table",
5959
footprint_translate_xyz=(-0.05, 0.0, 0.0),
60-
footprint_scale_xy=(1.2, 1.2),
60+
stand_default_footprint_xy_m=(1.08, 0.91032),
6161
stand_default_height=1.35,
6262
)
6363
_DROID_JOINT_NAMES = (
@@ -86,6 +86,7 @@ class DroidEmbodimentBase(EmbodimentBase, ABC):
8686
``initial_pose`` / ``set_initial_pose`` set the base of the robot in world frame.
8787
``stand_height_m`` sets the height of the stand mesh under the robot base link,
8888
which changes how far the stand extends below the root link.
89+
``stand_footprint_xy_m`` sets the stand footprint dimensions in the robot-base X/Y plane.
8990
When manually placing the robot on floor, ``set_initial_pose`` z value and
9091
``stand_height_m`` should be adjusted together to keep the bottom of stand fixed.
9192
``placement_bbox_stand_only`` uses the stand footprint for ``On`` / ``NextTo`` placement
@@ -103,6 +104,7 @@ def __init__(
103104
concatenate_observation_terms: bool = False,
104105
arm_mode: ArmMode | None = None,
105106
stand_height_m: float = _DROID_STAND_PRIM.stand_default_height,
107+
stand_footprint_xy_m: tuple[float, float] | list[float] = _DROID_STAND_PRIM.stand_default_footprint_xy_m,
106108
placement_bbox_stand_only: bool = False,
107109
collision_mode: CollisionMode | str | None = None,
108110
):
@@ -114,12 +116,18 @@ def __init__(
114116
collision_mode=collision_mode,
115117
)
116118
self.stand_height_m = stand_height_m
119+
assert len(stand_footprint_xy_m) == 2, f"stand_footprint_xy_m must have 2 values, got {stand_footprint_xy_m!r}"
120+
assert all(
121+
value > 0.0 for value in stand_footprint_xy_m
122+
), f"stand_footprint_xy_m must be positive, got {stand_footprint_xy_m}"
123+
self.stand_footprint_xy_m = tuple(stand_footprint_xy_m)
117124
self.placement_bbox_stand_only = placement_bbox_stand_only
118125
self.scene_config = DroidSceneCfg()
119126
self.scene_config.robot.spawn.usd_path = compose_on_stand_usd(
120127
_DROID_ROBOT_PRIM,
121128
_DROID_STAND_PRIM,
122129
stand_height_m=stand_height_m,
130+
stand_footprint_xy_m=self.stand_footprint_xy_m,
123131
output_basename="droid_franka_robotiq_on_stand",
124132
)
125133
self.action_config = None
@@ -181,6 +189,7 @@ def __init__(
181189
concatenate_observation_terms: bool = False,
182190
arm_mode: ArmMode | None = None,
183191
stand_height_m: float = _DROID_STAND_PRIM.stand_default_height,
192+
stand_footprint_xy_m: tuple[float, float] | list[float] = _DROID_STAND_PRIM.stand_default_footprint_xy_m,
184193
placement_bbox_stand_only: bool = False,
185194
collision_mode: CollisionMode | str | None = None,
186195
):
@@ -191,6 +200,7 @@ def __init__(
191200
concatenate_observation_terms=concatenate_observation_terms,
192201
arm_mode=arm_mode,
193202
stand_height_m=stand_height_m,
203+
stand_footprint_xy_m=stand_footprint_xy_m,
194204
placement_bbox_stand_only=placement_bbox_stand_only,
195205
collision_mode=collision_mode,
196206
)
@@ -212,6 +222,7 @@ def __init__(
212222
concatenate_observation_terms: bool = False,
213223
arm_mode: ArmMode | None = None,
214224
stand_height_m: float = _DROID_STAND_PRIM.stand_default_height,
225+
stand_footprint_xy_m: tuple[float, float] | list[float] = _DROID_STAND_PRIM.stand_default_footprint_xy_m,
215226
placement_bbox_stand_only: bool = False,
216227
collision_mode: CollisionMode | str | None = None,
217228
):
@@ -222,6 +233,7 @@ def __init__(
222233
concatenate_observation_terms=concatenate_observation_terms,
223234
arm_mode=arm_mode,
224235
stand_height_m=stand_height_m,
236+
stand_footprint_xy_m=stand_footprint_xy_m,
225237
placement_bbox_stand_only=placement_bbox_stand_only,
226238
collision_mode=collision_mode,
227239
)
@@ -244,6 +256,7 @@ def __init__(
244256
concatenate_observation_terms: bool = False,
245257
arm_mode: ArmMode | None = None,
246258
stand_height_m: float = _DROID_STAND_PRIM.stand_default_height,
259+
stand_footprint_xy_m: tuple[float, float] | list[float] = _DROID_STAND_PRIM.stand_default_footprint_xy_m,
247260
placement_bbox_stand_only: bool = False,
248261
collision_mode: CollisionMode | str | None = None,
249262
):
@@ -254,6 +267,7 @@ def __init__(
254267
concatenate_observation_terms=concatenate_observation_terms,
255268
arm_mode=arm_mode,
256269
stand_height_m=stand_height_m,
270+
stand_footprint_xy_m=stand_footprint_xy_m,
257271
placement_bbox_stand_only=placement_bbox_stand_only,
258272
collision_mode=collision_mode,
259273
)

isaaclab_arena/embodiments/franka/franka.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@
6060
ref_prim_path="/Stand",
6161
payload_child_name="Stand",
6262
footprint_translate_xyz=(-0.05, 0.0, 0.0),
63-
footprint_scale_xy=(1.2, 1.2),
63+
stand_default_footprint_xy_m=(0.3888, 0.3888),
6464
stand_default_height=0.8755,
6565
)
6666
_FRANKA_JOINT_NAMES = (

isaaclab_arena/embodiments/robot_on_stand_utils.py

Lines changed: 46 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
_ROBOT_ON_STAND_USD_CACHE_DIR = "robot_on_stand"
2222

2323
_HEIGHT_ATOL = 1e-3
24+
_FOOTPRINT_ATOL = 1e-3
2425
_ALIGN_ATOL = 5e-2
2526

2627

@@ -52,17 +53,17 @@ class StandPrimSpec:
5253
ref_prim_path: str
5354
payload_child_name: str
5455
footprint_translate_xyz: tuple[float, float, float]
55-
footprint_scale_xy: tuple[float, float]
56+
stand_default_footprint_xy_m: tuple[float, float]
5657
stand_default_height: float
5758

5859

59-
@functools.cache
6060
def compose_on_stand_usd(
6161
robot: RobotPrimSpec,
6262
stand: StandPrimSpec,
6363
*,
6464
stand_height_m: float,
6565
output_basename: str,
66+
stand_footprint_xy_m: tuple[float, float] | None = None,
6667
) -> str:
6768
"""Build a robot+stand USD with stand under the robot base link.
6869
@@ -74,15 +75,39 @@ def compose_on_stand_usd(
7475
stand: Stand reference and footprint parameters.
7576
stand_height_m: Target absolute stand height after align.
7677
output_basename: Stable basename for the composed USD (embodiment-specific).
78+
stand_footprint_xy_m: Target footprint ``(x, y)`` dimensions in meters. ``None`` uses the default.
7779
7880
Returns:
7981
Local path to the composed on-stand USD.
8082
"""
8183
assert stand_height_m > 0.0, f"stand_height_m must be positive, got {stand_height_m}"
84+
if stand_footprint_xy_m is None:
85+
stand_footprint_xy_m = stand.stand_default_footprint_xy_m
86+
assert len(stand_footprint_xy_m) == 2, f"stand_footprint_xy_m must have 2 values, got {stand_footprint_xy_m!r}"
87+
assert all(
88+
value > 0.0 for value in stand_footprint_xy_m
89+
), f"stand_footprint_xy_m must be positive, got {stand_footprint_xy_m}"
90+
return _compose_on_stand_usd_cached(
91+
robot,
92+
stand,
93+
stand_height_m,
94+
output_basename,
95+
stand_footprint_xy_m,
96+
)
97+
8298

99+
@functools.cache
100+
def _compose_on_stand_usd_cached(
101+
robot: RobotPrimSpec,
102+
stand: StandPrimSpec,
103+
stand_height_m: float,
104+
output_basename: str,
105+
stand_footprint_xy_m: tuple[float, float],
106+
) -> str:
83107
cache_root = get_arena_asset_cache_dir().parent / "usd" / _ROBOT_ON_STAND_USD_CACHE_DIR
84108
cache_root.mkdir(parents=True, exist_ok=True)
85-
out_path = cache_root / f"{output_basename}_{stand_height_m:.3f}.usd"
109+
footprint_x_m, footprint_y_m = stand_footprint_xy_m
110+
out_path = cache_root / f"{output_basename}_{stand_height_m}_{footprint_x_m}x{footprint_y_m}.usd"
86111

87112
with tempfile.NamedTemporaryFile(suffix=".usd", dir=cache_root, delete=False) as tmp_file:
88113
tmp_path = Path(tmp_file.name)
@@ -94,7 +119,7 @@ def compose_on_stand_usd(
94119
root.GetReferences().AddReference(robot_resolved, robot.root_prim_path)
95120
stage.SetDefaultPrim(root)
96121

97-
_mount_stand_normalized(stage, robot, stand, stand_height_m)
122+
_add_stand(stage, robot, stand, stand_height_m, stand_footprint_xy_m)
98123
assert stage.GetRootLayer().Save(), f"failed to save composed on-stand USD to {tmp_path}"
99124
os.replace(tmp_path, out_path)
100125
except Exception:
@@ -104,19 +129,20 @@ def compose_on_stand_usd(
104129
return str(out_path)
105130

106131

107-
def _mount_stand_normalized(
132+
def _add_stand(
108133
stage: Usd.Stage,
109134
robot: RobotPrimSpec,
110135
stand: StandPrimSpec,
111136
stand_height_m: float,
137+
stand_footprint_xy_m: tuple[float, float],
112138
) -> None:
113-
"""Parent a stand payload under the robot base link, scale, and align to the robot base.
139+
"""Add a stand under the robot base link with the requested size and alignment.
114140
115141
Composed USD hierarchy::
116142
117143
/panda/panda_link0/stand_instanceable # outer mount
118144
translate: (footprint_xy, align_z)
119-
scale: (footprint_xy, stand_height_m / native_height)
145+
scale: (stand_footprint_xy_m / native_footprint_xy_m, stand_height_m / native_height)
120146
/<payload_child_name>/ # inner payload
121147
reference: stand USD @ ref_prim_path
122148
@@ -125,6 +151,7 @@ def _mount_stand_normalized(
125151
robot: Robot prim layout under the composed default prim.
126152
stand: Stand reference and footprint parameters.
127153
stand_height_m: Target stand height after align.
154+
stand_footprint_xy_m: Target footprint ``(x, y)`` dimensions in meters.
128155
"""
129156
# Robot-only stage: bottom of the base link is the align target (before stand exists).
130157
robot_base = stage.GetPrimAtPath(robot.robot_base_prim_path)
@@ -136,36 +163,41 @@ def _mount_stand_normalized(
136163

137164
stand_resolved = retrieve_file_path(stand.stand_usd_path)
138165
tx, ty, tz = stand.footprint_translate_xyz
139-
sx, sy = stand.footprint_scale_xy
166+
footprint_x_m, footprint_y_m = stand_footprint_xy_m
140167
stand_prim_path = robot.stand_prim_path
141168

142-
# Outer mount under link0; Z scale stays at 1 until native height is measured.
169+
# Outer mount under link0 stays at unit scale until the source stand dimensions are measured.
143170
stand_xf = UsdGeom.Xform.Define(stage, stand_prim_path)
144171
translate_op = stand_xf.AddTranslateOp()
145172
translate_op.Set(Gf.Vec3d(tx, ty, tz))
146173
scale_op = stand_xf.AddScaleOp()
147-
scale_op.Set(Gf.Vec3d(sx, sy, 1.0))
174+
scale_op.Set(Gf.Vec3d(1.0, 1.0, 1.0))
148175

149176
payload_prim = stage.DefinePrim(f"{stand_prim_path}/{stand.payload_child_name}")
150177
payload_prim.GetReferences().AddReference(stand_resolved, stand.ref_prim_path)
151178

152179
stand_prim = stand_xf.GetPrim()
153180
bbox_cache = UsdGeom.BBoxCache(Usd.TimeCode.Default(), [UsdGeom.Tokens.default_])
154181

155-
# Native height at footprint XY scale; then scale Z to the requested stand height.
182+
# Scale the source stand to the requested physical dimensions.
156183
stand_range = bbox_cache.ComputeWorldBound(stand_prim).ComputeAlignedRange()
157184
assert not stand_range.IsEmpty(), f"empty stand bounds at {stand_prim.GetPath()}"
158-
native_height = float(stand_range.GetSize()[2])
185+
native_size = stand_range.GetSize()
186+
native_x_m, native_y_m, native_height = (float(value) for value in native_size)
187+
assert native_x_m > 0.0 and native_y_m > 0.0, f"non-positive stand footprint at {stand_prim.GetPath()}"
159188
assert native_height > 0.0, f"non-positive stand height at {stand_prim.GetPath()}"
160-
scale_op.Set(Gf.Vec3d(sx, sy, stand_height_m / native_height))
189+
scale_op.Set(Gf.Vec3d(footprint_x_m / native_x_m, footprint_y_m / native_y_m, stand_height_m / native_height))
161190

162191
# Raise/lower outer translate so stand top meets robot_min_z.
163192
translate_op.Set(Gf.Vec3d(tx, ty, robot_min_z))
164193

165194
# Verify stand/robot alignment and stand height on a fresh BBoxCache.
166195
verify_cache = UsdGeom.BBoxCache(Usd.TimeCode.Default(), [UsdGeom.Tokens.default_])
167196
stand_range = verify_cache.ComputeWorldBound(stand_prim).ComputeAlignedRange()
168-
stand_height = float(stand_range.GetSize()[2])
197+
stand_size = stand_range.GetSize()
198+
stand_x_m, stand_y_m, stand_height = (float(value) for value in stand_size)
169199
stand_max_z = float(stand_range.GetMax()[2])
200+
assert abs(stand_x_m - footprint_x_m) < _FOOTPRINT_ATOL, stand_x_m
201+
assert abs(stand_y_m - footprint_y_m) < _FOOTPRINT_ATOL, stand_y_m
170202
assert abs(stand_height - stand_height_m) < _HEIGHT_ATOL, stand_height
171203
assert abs(stand_max_z - robot_min_z) < _ALIGN_ATOL, (stand_max_z, robot_min_z)

0 commit comments

Comments
 (0)