Skip to content

Add regression test for fabrics after rebuild - #1174

Merged
alexmillane merged 4 commits into
mainfrom
alex/test/regression_test_for_fabrics_after_rebuild
Sep 3, 2026
Merged

Add regression test for fabrics after rebuild#1174
alexmillane merged 4 commits into
mainfrom
alex/test/regression_test_for_fabrics_after_rebuild

Conversation

@alexmillane

Copy link
Copy Markdown
Collaborator

Summary

Adds a test that tests fabric poses of prims after a stage rebuild.

Detailed description

@alexmillane alexmillane left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self review.

Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_render_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py
@alexmillane
alexmillane marked this pull request as ready for review September 2, 2026 16:44
Comment thread isaaclab_arena/tests/test_render_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_render_after_stage_rebuild.py Outdated
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py
Comment thread isaaclab_arena/tests/test_prims_after_stage_rebuild.py
- Inline enable_cameras at call sites instead of an ENABLE_CAMERAS constant
- Use numpy for the identity matrix instead of a hardcoded tuple
- Report all stale prim paths; drop the truncation variable
- Replace the _describe_build helper with 0-indexed build labels
- Shorten failure messages and a docstring

Signed-off-by: Alex Millane <amillane@nvidia.com>
@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

Adds a second regression test for the Lab render-after-rebuild bug, checking Fabric omni:fabric:worldMatrix values directly instead of comparing rendered pixels — a genuinely cheaper failure to diagnose, since it names the offending prim paths. The test itself is well built. The concern is the two edits to test_render_after_stage_rebuild.py, which look like debug leftovers rather than intended changes.

Findings

🔴 Critical: isaaclab_arena/tests/test_render_after_stage_rebuild.py:177 — the @pytest.mark.skip is commented out, un-skipping the known-failing GPU+Fabric render test. That contradicts the PR description and the TODO directly above it, and will turn the with_cameras phase red. Restore the marker (or delete the line rather than commenting it out, if the un-skip is intended).

🟡 Warning: isaaclab_arena/tests/test_render_after_stage_rebuild.py:34SAVE_IMAGES flipped FalseTrue, while its own comment still describes True as the opt-in. Every CI run now dumps PNGs per camera.

🟡 Warning: isaaclab_arena/tests/test_prims_after_stage_rebuild.py:46_build_droid_env (plus HEADLESS/ENABLE_CAMERAS and the reset-and-step loop) is a near-copy of the render test's. Worth pulling into isaaclab_arena/tests/utils/ so the two tests can't drift. The two copies already differ: only the new one forces --device cpu.

🔵 Improvement: isaaclab_arena/tests/test_prims_after_stage_rebuild.py:41MIN_PRIMS_COMPARED = 1 is a thin vacuity floor given the four continue paths in the scan. Asserting that both builds compare the same number of prims would be stronger and drop the magic number.

Test Coverage

This PR is the test. Both new tests carry @pytest.mark.with_cameras and no subprocess marker, so they land in Phase 2 — correct for camera-enabled sim work. The inner/outer pattern is followed properly: the pytest-visible functions call run_function_with_persistent_simulation_app, and all sim-dependent imports (omni.usd, usdrt, pxr, isaaclab.*, and the Arena builder modules) are deferred inside the inner functions. Fabric-on variants of both tests are skipped, so what actually runs in CI is the Fabric-off path — reasonable while the Lab bug is open, and the TODOs carry the shared [lab-render-after-rebuild-bug] tag so they can be found together.

Verdict

Minor fixes needed — the un-skipped render test and the SAVE_IMAGES flip should be resolved before merge; the rest is optional cleanup.

@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds lower-level regression coverage for stale Fabric world matrices after rebuilding a stage and improves render-comparison image naming. It also inadvertently enables the known-failing GPU-and-Fabric render regression.

  • Adds a two-build prim-transform test for camera-enabled Fabric-on and Fabric-off configurations.
  • Saves render diagnostics with Fabric-specific filenames.
  • Removes the skip from the unresolved Fabric-on render regression.

Confidence Score: 4/5

The PR should not merge until the Fabric-on render regression is skipped again or the underlying rebuild defect is fixed.

The newly enabled test is selected by standard GPU camera-test jobs and deliberately exercises the unresolved Fabric path that its own TODO says renders rebuilt geometry incorrectly.

Files Needing Attention: isaaclab_arena/tests/test_render_after_stage_rebuild.py

Important Files Changed

Filename Overview
isaaclab_arena/tests/test_prims_after_stage_rebuild.py Adds focused transform-level regression coverage with the currently broken Fabric-on variant explicitly skipped.
isaaclab_arena/tests/test_render_after_stage_rebuild.py Improves diagnostic filenames but activates a regression test whose adjacent documentation says the exercised GPU-and-Fabric path remains broken.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Create camera-enabled environment] --> B[Reset and step simulation]
  B --> C[Read USD and Fabric transforms]
  C --> D[Close environment and rebuild stage]
  D --> E[Repeat transform comparison]
  E --> F{Fabric world matrix stale?}
  F -->|Yes| G[Report offending prim paths]
  F -->|No| H[Regression passes]
Loading

Reviews (1): Last reviewed commit: "Add regression test notes." | Re-trigger Greptile

Comment thread isaaclab_arena/tests/test_render_after_stage_rebuild.py Outdated
Co-authored-by: arena-review-bot[bot] <290456231+arena-review-bot[bot]@users.noreply.github.com>
@alexmillane
alexmillane merged commit e07d35e into main Sep 3, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants