Skip to content

Add minimal ArenaWorld scene queries - #1173

Open
cvolkcvolk wants to merge 13 commits into
mainfrom
cvolk/feature/arena-world-mvp
Open

Add minimal ArenaWorld scene queries#1173
cvolkcvolk wants to merge 13 commits into
mainfrom
cvolk/feature/arena-world-mvp

Conversation

@cvolkcvolk

@cvolkcvolk cvolkcvolk commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Arena configuration objects describe the world before simulation, while predicates need a consistent way to query the live scene after it is spawned. An environment-owned, read-only ArenaWorld separates runtime access from configuration and avoids predicate-specific state and cache workarounds.

Current scope

  • Read rigid-object poses and root linear velocities fresh from the live scene, and read live poses for scene extras.
  • Lazily cache spawned AABBs for rigid objects and scene extras for one environment lifetime.
  • Move object_on_destination() onto this boundary, remove its stateful manager-term workaround, and have sorting combine one placement term per object. Sorting now requires destination-footprint containment, upward support, and low velocity for every object, so success-rate results are not directly comparable with older sorting runs.

Broader direction

This is the first deliberately small slice of the broader ArenaWorld design.

  • Add articulation root state, deformable nodal positions and velocities, and filtered contact-force access.
  • Add RigidObjectSet geometry variants and migrate further runtime consumers as their semantics are defined.

Live state remains uncached; only immutable derived geometry and access helpers are cached. Predicate semantics and simulator writes remain outside ArenaWorld.

References: ArenaWorld design and deformable support design.

Move live pose and derived geometry lookup behind the environment-owned ArenaWorld. Keep ObjectOnDestinationTerm as a stateless manager adapter and cache only local AABBs for the environment lifetime.

Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Document and assert the supported scene categories for pose and geometry queries. Use the initialized ArenaWorld field directly during cleanup.

Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Name the rigid-object and scene-extra lookup paths explicitly, hide scene-extra reader caching behind a helper, and apply one output contract before returning the pose.

Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
cvolkcvolk marked this pull request as ready for review September 2, 2026 16:55
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces an environment-owned ArenaWorld for live scene queries and lifetime-scoped geometry caching, then migrates placement predicates away from their stateful manager term.

  • Adds live rigid-object pose and velocity queries plus live scene-extra poses.
  • Lazily derives and caches entity-frame AABBs.
  • Updates pick-and-place and sorting success predicates to use destination geometry, upward support force, and velocity.
  • Adds coverage for entity access, caching, lifecycle, and placement semantics.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issues identified.

The updated task callers supply the new placement arguments, live and cached scene data remain aligned for established task paths, and ArenaWorld is guarded across initialization and closure.

Important Files Changed

Filename Overview
isaaclab_arena/environments/arena_world.py Adds the environment-owned query boundary, live-state reads, lazy AABB and scene-extra reader caches, and explicit closure behavior.
isaaclab_arena/environments/arena_world_entity_access.py Moves spawned-geometry extraction and scene-extra FrameView pose access into the environment layer while preserving clone coverage checks.
isaaclab_arena/environments/isaaclab_arena_manager_based_env.py Initializes ArenaWorld before manager loading and clears it on load failure or environment closure.
isaaclab_arena/tasks/predicates/spatial.py Migrates single- and multi-object placement predicates to ArenaWorld and requires explicit object-destination-sensor pairing.
isaaclab_arena/tasks/pick_and_place_task.py Replaces the stateful placement manager term with the plain predicate for termination and progress evaluation.
isaaclab_arena/tasks/sorting_task.py Supplies destination configurations for each object in multi-object placement success evaluation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Task[Task termination or progress predicate] --> World[ArenaWorld]
    World --> Pose[Live entity poses]
    World --> Velocity[Live rigid-object velocity]
    World --> Bounds[Lifetime-cached entity-frame AABBs]
    Task --> Sensor[Filtered contact force]
    Pose --> Placement[Placement evaluation]
    Velocity --> Placement
    Bounds --> Placement
    Sensor --> Placement
    Placement --> Result[Boolean result per environment]
Loading

Reviews (1): Last reviewed commit: "Explain ArenaWorld pose dispatch" | Re-trigger Greptile

Comment thread isaaclab_arena/tasks/sorting_task.py Outdated
Comment thread isaaclab_arena/environments/isaaclab_arena_manager_based_env.py Outdated
Comment thread isaaclab_arena/tasks/predicates/spatial.py Outdated
Comment thread isaaclab_arena/tests/test_arena_world.py Outdated
@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

This adds an environment-owned ArenaWorld for live pose / root-velocity reads and lifetime-cached spawned AABBs, moves the geometry module out of tasks/predicates/ into environments/, and collapses ObjectOnDestinationTerm back into a plain object_on_destination predicate. The slice is well scoped and the layering is right — geometry lookup and caching now sit in the layer that owns the live scene, and the stateful manager-term workaround is gone. My main concern is that folding the term's logic into the shared predicate also changes what SortMultiObjectTask counts as success, which the PR description does not mention.

Findings

🟡 Warning: isaaclab_arena/tasks/sorting_task.py:84objects_on_destinations previously only checked contact-force magnitude and speed; it now also requires footprint containment and a 45° upward support cone. SortMultiObjectTask success rates will drop against previously recorded runs. Worth calling out explicitly so old and new sorting numbers are not compared.

🔵 Improvement: isaaclab_arena/environments/isaaclab_arena_manager_based_env.py:75-83load_managers() is only called from ManagerBasedEnv.__init__, so the re-entry assert and the try/except here are already covered by the handler in __init__.

🔵 Improvement: isaaclab_arena/tasks/predicates/spatial.py:192 — signature still says ManagerBasedRLEnv, but the body needs IsaacLabArenaManagerBasedRLEnv.arena_world. Same for objects_on_destinations.

🔵 Improvement: isaaclab_arena/tests/test_arena_world.py:60 — the explicit arena_world.close() is a no-op after env.close(), and it means the closed-state assertion would still pass if the env's release hook were removed.

Test Coverage

Good. test_arena_world.py covers the live/cached split end-to-end (pose follows write_root_pose_to_sim, AABB identity survives a reset), test_arena_world_entity_access.py unit-tests the USD bounds math and the FrameView reader with doubles, and test_object_on_destination.py carries over the boundary cases from the deleted term test plus the wrapped and unwrapped call paths. All use the persistent-sim-app inner/outer pattern and belong in phase 1 (no camera/subprocess markers needed). The test_sorting_task.py switch from terminated to termination_manager.get_term("success") is a real improvement — the old assertions could pass via object_dropped.

One gap: nothing covers a scene-extra destination through the full object_on_destination path. SceneExtraPoseReader is exercised only against doubles, and ArenaWorld.get_pose_w only against a rigid object.

CI

Run tests with subprocess failed on a submodule checkout flake (fatal: shallow file has changed since we read it), not on this code — needs a rerun. Run tests and the GR00T E2E job were still pending at review time.

Verdict

Minor fixes needed

@qianl-nv qianl-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating this minimal interface PR first! This should unblock adding deformable state on top.
I have a few nits mostly on naming. Will take a look again after update.

Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/tasks/predicates/spatial.py Outdated
Comment thread isaaclab_arena/tasks/predicates/spatial.py Outdated
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com>

@alexmillane alexmillane left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. A few smaller comments.

Comment thread isaaclab_arena/environments/arena_world.py
Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/environments/arena_world.py Outdated
Comment thread isaaclab_arena/environments/arena_world_entity_access.py Outdated
Comment thread isaaclab_arena/environments/isaaclab_arena_manager_based_env.py Outdated
Comment thread isaaclab_arena/environments/isaaclab_arena_manager_based_env.py Outdated
Comment thread isaaclab_arena/environments/isaaclab_arena_manager_based_env.py Outdated

@qianl-nv qianl-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Use scene keys and local-frame bounds throughout the ArenaWorld API. Leave simulation lifecycle cleanup to Isaac Lab and document the returned tensor shapes.

Signed-off-by: Clemens Volk <cvolk@nvidia.com>

@alexmillane alexmillane left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants