[Newton] Refresh FK before ray-cast sensor reads

[ooctipus]

Description

Fixes #7236.

Newton ray-cast tasks read NewtonManager.get_state_0().body_q inside their graph-capturable query pipeline. After an in-step joint or root-state write, the reset masks are current but the derived body_q remains stale until forward kinematics runs. This made the first sensor observation after a reset use the previous pose.

This fixes the stale read at its ownership boundary instead of adding eager simulator synchronization to the RL environment loops:

• NewtonManager._update_sensor_tasks() obtains state through the guarded get_state() accessor before BVH refit and sensor graph capture/replay.
• The graph-captured raycast callback keeps using raw get_state_0() state, so forward() is never captured.
• The ray-caster's direct get_world_poses() accessor applies the same lazy-FK rule.
• The renderer's now-redundant state refresh is removed because the sensor-task scheduler owns freshness for both renderer and raycast consumers.

This is an alternative to the in-step synchronization part of #7516. Its regex and legacy tracked-target fixes are independent of this PR.

Architecture and performance

The state access boundary owns FK freshness; environment stepping remains unaware of individual sensor requirements. The guard runs only when a sensor or renderer update is requested. Rendering does not gain an additional refresh because its existing call moved into the shared scheduler, and Newton's device-resident reset masks limit the actual FK work to invalidated worlds/articulations.

Type of change

• Bug fix (non-breaking change which fixes an issue)

Release backport

• This PR already targets the active release branch; do not backport it again.

Screenshots

Not applicable.

Validation

• Added observable regressions for sensor-data reads and direct pose-getter reads immediately after a carrier root-pose write, with no intervening simulation step or FK-sensitive asset getter. Both tests failed before the fix by exactly the authored displacement in eager and CUDA-graph modes.
• uv run --frozen --extra test python -m pytest source/isaaclab_newton/test/sensors/test_newton_raycast_sensor.py -vv (16 passed)
• uv run --frozen --extra test python -m pytest source/isaaclab_newton/test/physics/test_newton_manager_abstraction.py source/isaaclab/test/sim/test_newton_manager_visualization_state.py -q (194 passed)
• uv run --frozen isaaclab -f (all checks passed)

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks
• No standalone documentation change is required; the ownership rule is documented at the access boundaries
• My changes generate no new warnings
• I have added tests that prove the fix is effective
• I have added a changelog fragment for the touched package
• My name already exists in CONTRIBUTORS.md

[greptile-apps]

Greptile Summary

This PR moves Newton FK freshness into the shared sensor-task scheduler and direct ray-caster pose accessor, preventing the first observation after a state write from using stale body transforms.

• Refreshes derived Newton state before BVH refits and sensor graph execution.
• Keeps graph-captured callbacks on the raw state accessor so forward kinematics is not captured.
• Removes the renderer’s now-redundant explicit refresh.
• Adds eager and CUDA-graph regression coverage for sensor and direct-pose reads.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking performance issue from unnecessarily refitting shared sensor BVHs on repeated unchanged-state reads.

The refreshed state reaches BVH refit and graph execution in the correct order, but the new scheduler call also marks sensor state dirty every time and therefore repeats avoidable refit work.

Files Needing Attention: source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py

Important Files Changed

Filename	Overview
source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py	Centralizes state refresh before sensor execution, but currently re-dirties and refits sensor BVHs on repeated unchanged-state reads.
source/isaaclab_newton/isaaclab_newton/renderers/newton_warp_renderer.py	Removes a redundant renderer refresh because the shared scheduler now refreshes state before refitting and rendering.
source/isaaclab_newton/isaaclab_newton/sensors/ray_caster/newton_raycast_sensor.py	Refreshes pending FK for direct pose reads while preserving raw-state access inside graph-captured callbacks.
source/isaaclab_newton/test/sensors/test_newton_raycast_sensor.py	Adds regressions covering sensor-data and direct-pose reads immediately after carrier pose writes.
source/isaaclab_newton/test/physics/test_newton_manager_abstraction.py	Extends scheduler-ordering coverage to require state refresh before BVH refit and rendering.

Sequence Diagram

sequenceDiagram
    participant C as Sensor or renderer
    participant M as NewtonManager
    participant FK as Forward kinematics
    participant B as Sensor BVH
    participant Q as Captured query callback
    C->>M: _update_sensor_tasks()
    M->>FK: get_state() / forward()
    FK-->>M: Refreshed body_q
    M->>B: Refit using current state
    M->>Q: Capture or replay
    Q->>M: get_state_0()
    Q-->>C: Current sensor output

Loading

_{Reviews (1): Last reviewed commit: "Refresh Newton sensor state before ray c..." | Re-trigger Greptile}

[greptile-apps]

Avoid redundant sensor BVH refits

Every sensor-task update now calls get_state(), whose Newton path runs forward() and marks sensor state dirty even when no FK work is pending. Repeated ray-caster or renderer reads without an intervening state change therefore rerun the shared shape and particle BVH refits, adding avoidable work for every consumer read.

Knowledge Base Used: Newton backend

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[isaaclab-review-bot]

Isaac Lab Review Bot

The PR coherently moves FK freshness into the Newton state-access boundary, but FK can mutate body_q without invalidating the sensor BVH cache, leaving immediate scene queries with fresh transforms and stale acceleration bounds.

• Design and architecture: Having NewtonManager.get_state() refresh derived transforms before the shared sensor-task pipeline is coherent and keeps forward() outside graph-captured callbacks. However, this ownership boundary must also propagate FK-driven transform changes to the BVH dirty state.
• API: Public signatures and return types remain unchanged, renderer refresh behavior is routed through the shared scheduler, and the Newton package includes the required changelog fragment.
• Implementation: At _update_sensor_tasks(), get_state() may run FK and mutate _state_0.body_q in place. The following identity comparison therefore does not set _sensor_state_dirty, so an already-cleared dirty flag can prevent BVH refitting even though body transforms changed. The FK refresh path should explicitly invalidate the sensor BVH when it resolves pending transforms.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

[isaaclab-review-bot]

🟡 Warning · Implementation — Resolved FK does not invalidate BVH cache

get_state() may run forward(), which rewrites _state_0.body_q in place, so the object identity check on the next line never fires and _sensor_state_dirty stays False after the previous update cleared it. The subsequent query then uses freshly transformed ray origins against BVH bounds refit from pre-FK poses, so targets moved by the same reset can be missed. Set _sensor_state_dirty = True when pending FK was resolved.

[kellyguo11]

run-ci

[ooctipus]

run-ci

[isaaclab-bot]

Backported to release/3.0.0 as 2af0251.