Add flat mesh slots for heterogeneous multi-mesh raycasts

[ooctipus]

Motivation

PR #5510 adds ClonePlan-based multi-mesh target discovery as part of the RayCaster backend split. To keep that PR conservative, heterogeneous layouts still use the existing rectangular dynamic-mesh kernel with dummy padding for environments that have fewer meshes.

This PR removes that padding cost. Instead of launching raycasts over num_envs * max_meshes_per_env, it launches over the actual flat list of real mesh targets.

What Changed

• Added raycast_dynamic_mesh_slots_kernel, launched as (num_slots, num_rays).
• Switched MultiMeshRayCaster and MultiMeshRayCasterCamera from rectangular padded tables to flat mesh slots.
• Added slot_env_ids so each mesh slot maps back to its owning environment.
• Updated PhysX and Newton dynamic target pose refresh to write into flat slot buffers.
• Updated ClonePlan integration coverage to assert heterogeneous slot counts [1, 2, 1] without USD object clones.

Why

The rectangular kernel is correct, but for heterogeneous ClonePlan layouts it does work for dummy meshes. Flat slots match the actual scene layout: one slot per real mesh, no fake mesh ids, no far-away padding poses, and less wasted kernel work.

Dependency

Depends on #5510. Please review and land #5510 first, then review this PR as the follow-up flat-slot optimization. After #5510 lands, this PR should be rebased onto develop so the diff only shows the flat-slot change.

Test Plan

• ./isaaclab.sh -p -m pytest source/isaaclab/test/sensors/test_ray_caster_integration.py::test_multi_mesh_consumes_clone_plan_without_usd_object_clones -q
• ./isaaclab.sh -p -m pytest source/isaaclab/test/sensors/test_multi_mesh_ray_caster_camera.py -q
• ./isaaclab.sh -p -m pytest source/isaaclab/test/sensors/test_ray_caster_integration.py::test_update_mesh_transforms_non_identity_offset -q
• ./isaaclab.sh -f

[greptile-apps]

Greptile Summary

This PR replaces the rectangular padded-table layout for multi-mesh raycasts with a compact flat-slot Warp kernel (raycast_dynamic_mesh_slots_kernel), enabling heterogeneous per-environment mesh counts without padding. It also restructures ClonePlan from a per-group object (with dest_template/prototype_paths) into a single flat plan (sources/destinations/clone_mask), moves heterogeneous clone-plan construction into InteractiveScene, and extracts shared ray-caster logic into backend-agnostic base classes.

• New raycast_dynamic_mesh_slots_kernel dispatches over a flat (num_slots, num_rays) grid; each slot carries its owning environment id, eliminating per-env padding.
• ClonePlan is refactored to carry flat parallel sources/destinations lists; clone_from_template is removed and make_clone_plan now returns a ClonePlan directly.
• BaseMultiMeshRayCaster and BaseRayCaster are extracted as backend-neutral base classes; PhysX and Newton subclasses are thin mixins.

Confidence Score: 3/5

Not safe to merge as-is: two files carry stale references to removed ClonePlan fields that will raise AttributeError at runtime, and the ray_mesh_id semantic change in the new flat-slot kernel may silently corrupt per-env mesh-id lookups in downstream consumers.

The _target_tracker_utils.py module references plan.dest_template — a field that was deleted in this PR — meaning any caller of walk_target_prototypes crashes immediately. The check_multi_mesh_ray_caster.py test helper similarly constructs a ClonePlan with the old field names and calls the renamed set_clone_plans method, so the script fails before exercising any sensor logic. Additionally, the new raycast_dynamic_mesh_slots_kernel records the global slot_id in ray_mesh_id rather than the per-env mesh index used by the existing rectangular kernel, which can silently produce wrong indices for any code that treats the output as a dense per-env mesh selector.

_target_tracker_utils.py (stale dest_template access), check_multi_mesh_ray_caster.py (old ClonePlan constructor and set_clone_plans call), and kernels.py (undocumented ray_mesh_id semantic change in the flat-slot kernel).

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/utils/warp/kernels.py	Adds raycast_dynamic_mesh_slots_kernel — flat slot layout for heterogeneous scenes; ray_mesh_id stores global slot index (not per-env mesh index), diverging from the existing rectangular-kernel convention without documentation.
source/isaaclab/isaaclab/sensors/ray_caster/_target_tracker_utils.py	New shared target-tracker utility; _find_covering_plan accesses plan.dest_template — a field that no longer exists on the refactored ClonePlan, causing AttributeError at runtime for any caller.
source/isaaclab/isaaclab/sensors/ray_caster/base_multi_mesh_ray_caster.py	New base class for multi-mesh ray-casting using compact flat slots; ClonePlan integration and mesh slot initialization look structurally correct.
source/isaaclab/isaaclab/cloner/clone_plan.py	Replaces dest_template/prototype_paths fields with flat sources/destinations lists; adds eq=False to keep hash identity-based. Several consumers were not fully updated.
source/isaaclab/isaaclab/scene/interactive_scene.py	Migrates heterogeneous clone-plan construction into the scene itself; correctly uses the new flat ClonePlan API and publishes via set_clone_plan (singular).
source/isaaclab/test/sensors/check_multi_mesh_ray_caster.py	Check script uses old set_clone_plans (plural) and constructs ClonePlan with removed fields dest_template/prototype_paths; will fail immediately on execution.
source/isaaclab/isaaclab/cloner/cloner_utils.py	Removes clone_from_template; make_clone_plan now returns a ClonePlan directly instead of a tuple — clean simplification.
source/isaaclab_physx/isaaclab_physx/scene_data_providers/physx_scene_data_provider.py	Correctly migrated to flat ClonePlan; reads plan.sources/plan.destinations directly instead of reconstructing paths from dest_template.

Sequence Diagram

sequenceDiagram
    participant Scene as InteractiveScene
    participant Cloner as cloner_utils
    participant SimCtx as SimulationContext
    participant Sensor as BaseMultiMeshRayCaster
    participant Kernel as raycast_dynamic_mesh_slots_kernel

    Scene->>Cloner: make_clone_plan(sources, dests, num_envs, strategy)
    Cloner-->>Scene: ClonePlan(sources[], destinations[], clone_mask)
    Scene->>SimCtx: set_clone_plan(plan)
    Scene->>Scene: clone_environments() → usd_replicate / physics_clone_fn

    Sensor->>SimCtx: get_clone_plan()
    SimCtx-->>Sensor: ClonePlan
    Sensor->>Sensor: _initialize_warp_meshes_from_clone_plan(plan)
    Note over Sensor: Builds flat slot arrays

    loop each simulation step
        Sensor->>Sensor: _update_mesh_transforms()
        Sensor->>Kernel: "launch(dim=(num_slots, num_rays))"
        Note over Kernel: slot_id → tid_env via slot_env_ids
        Kernel-->>Sensor: ray_hits, ray_distance, ray_mesh_id
    end

Loading

_{Reviews (1): Last reviewed commit: "Add flat mesh slots for multi-mesh rayca..." | Re-trigger Greptile}

[greptile-apps]

Stale dest_template reference on refactored ClonePlan

_find_covering_plan accesses plan.dest_template, but the ClonePlan dataclass was restructured in this PR to replace the single dest_template: str field with destinations: list[str] (one entry per source row). Any caller of walk_target_prototypes will hit an AttributeError at runtime. The function needs to iterate over plan.destinations rows and use each row's template string to compute the prefix, rather than calling plan.dest_template.

[greptile-apps]

Test helper uses old ClonePlan API and old set_clone_plans method

This block calls sim.set_clone_plans(...) (plural), which was renamed to set_clone_plan(...) (singular) in this PR, and constructs ClonePlan(dest_template=env_fmt, prototype_paths=[...], ...) with field names that no longer exist (replaced by sources and destinations). Running this check script would immediately fail with TypeError at ClonePlan construction and AttributeError on set_clone_plans. It should use sim.set_clone_plan(ClonePlan(sources=[...], destinations=[...], clone_mask=...)).

[greptile-apps]

ray_mesh_id stores global slot index, not per-env mesh index

In raycast_dynamic_meshes_kernel (the rectangular-table variant), ray_mesh_id is set to wp.int16(tid_mesh_id) — the per-env mesh index. Here the flat-slot kernel stores wp.int16(slot_id) — the global slot index, which is different for environments that start at a non-zero slot offset. Any consumer that treats ray_mesh_ids as a dense per-env mesh index (e.g. to index into a per-env property table) will receive incorrect results for environments whose slots do not start at 0. The docstring for raycast_dynamic_mesh_slots_kernel should call out this semantic difference, and callers building on ray_mesh_ids should be verified against the new convention.

[AntoineRichard]

Hi @ooctipus — thanks for putting this one up! 🙏

We're doing a cleanup pass over the Isaac Lab PR backlog, which had grown past 400 open pull requests, and we're closing out the ones that have gone quiet so the queue is reviewable again.

Why this PR is being closed: Here is exactly what we found on this PR when we reviewed the backlog:

Opened	2026-05-07 (about 4 months ago)
Last commit on the branch	2026-05-07
Last activity from the author	about 4 months ago
Target branch	develop
Review status	Never reviewed by a maintainer — nobody on the team got to it. Sorry about that.
Merge status	Unknown
Size	13 commit(s), 56 file(s) changed, +3557 / -2309

It was picked up by the sweep because it has been open for about 4 months. It was then put in the "close" bucket because the author has been silent for about 4 months — which is the signal we used to tell apart pull requests that are still being worked on from ones that have genuinely been set aside.

We deliberately did not close pull requests that were approved and ready to land, or that were small and clearly still fixing a live bug — there were 27 of those, and we are merging them rather than closing them.

No judgement on the change itself — this is purely backlog hygiene.

If this is still wanted, please reopen it or re-submit against develop. 💚

---

🤖 This comment was drafted with AI assistance as part of a maintainer-led sweep of the Isaac Lab pull request backlog. A maintainer is behind this cleanup — but if this closure looks wrong, it may well be, so please push back and we'll take another look.