[Tests] Trim redundant isaaclab_rl export and wrapper tests - #7604
Conversation
Drop the LEAPP export tasks that repeat an already covered export pattern, the checkpoint-fixture-only test, and a recurrent-state unit test that is duplicated in test_leapp_recurrent_state.py. Limit the finite-horizon wrapper check to two environments since the contract is task independent.
Greptile SummaryThis PR shortens the
Confidence Score: 4/5The PR appears safe to merge, with a non-blocking recommendation to preserve focused coverage of silent subprocess failure detection. The task-matrix and wrapper reductions retain representative coverage, but deleting the only failure-path test for Files Needing Attention: source/isaaclab_rl/test/export/test_rsl_rl_export_flow.py Important Files Changed
Reviews (1): Last reviewed commit: "Trim redundant isaaclab_rl export and wr..." | Re-trigger Greptile |
| @@ -287,39 +253,6 @@ def _run_export_batch_entrypoint() -> None: | |||
| _run_export_batch(tasks) | |||
There was a problem hiding this comment.
Traceback Detection Loses Coverage
Removing the only test of _fail_on_process_error leaves its traceback-detection branch untested, even though the export flow relies on this helper to reject simulator failures that exit successfully. A later change to the patterns or failure logic could make CI accept a subprocess containing a traceback. Please retain a small focused test for the zero-exit-code failure path.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Isaac Lab Review Bot
This test-only change reduces LEAPP/RSL-RL export matrices, removes test-local redundant checks, limits the finite-horizon wrapper test to two registered environments, and adds the required .skip changelog fragment. No candidate findings identified a concrete issue requiring action before merge.
- Design and architecture: The patch consistently adopts representative export-path coverage instead of robot-variant breadth. Retained tasks span the documented classic, navigation, locomotion, manipulation, and dexterous patterns, while production architecture and export paths remain unchanged.
- API: No production API, public symbol, task registration, or export script is modified. Removed classes and test functions are local to test modules, so the patch does not introduce a public compatibility or migration concern.
- Implementation: The reduced RSL-RL task list still forms two valid batches under the existing batch size, and slicing
registered_tasks[:2]preserves the wrapper test mechanics while reducing task breadth. The principal non-blocking tradeoff is reduced variant-specific regression coverage, with the patch explicitly relying on representative shared-pattern and per-backend tests instead.
No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.
Automated review; human maintainers own approval decisions.
|
run-ci |
Description
Trims the
isaaclab_rltest suite where it repeats an export path or a check that already exists elsewhere. On this workstation the suite took ~15 min, of whichtest_leapp_export_flow.pyalone was 7 to 9 min andtest_rsl_rl_export_flow.pyanother 2.5 min; every other file finishes in under 35 s.export/test_leapp_export_flow.pyIsaac-Humanoidfrom the RSL-RL task list.test_rsl_rl_humanoid_export_across_physics_backends[newton_mjwarp]already creates the same initialized checkpoint with the same preset and exports it, so the matrix entry was an exact duplicate (one ~20 s checkpoint subprocess and one ~14 s export subprocess).Isaac-Reach-UR10. The list is documented as "one example of each distinct observation / command / action pattern"; UR10 is the same pose-command + joint-position-arm pattern asIsaac-Reach-Franka, which stays.test_initialized_checkpoints. It only re-asserted that the module fixture wrote its path files; the fixture already fails the run on a non-zero subprocess and every export test asserts its own checkpoint path.export/test_rsl_rl_export_flow.pyReach-UR10. Note that the batches containing those variants finished in ~12 s for 8 tasks in both baseline runs, i.e. the exports short-circuit (checkpoint unavailable oractor_state_dictmismatch are silently skipped), so their nominal coverage was mostly not exercised.test_recurrent_state_helpers_support_modular_rnn_model_lstm, which is line-for-lineTestRslRlRecurrentState.test_modular_rnn_model_lstm_round_tripintest_leapp_recurrent_state.py, andtest_export_flow_fails_on_sim_traceback, which tested a helper of the test file itself.test_rsl_rl_wrapper.pytest_no_time_outsnow builds two environments instead of five. The finite-horizon contract it checks belongs to the wrapper, not the task;test_random_actionsstill runs all five.Before and after measurement
Per-file wall time from the CI-style per-file runner on the same workstation (RTX PRO 6000). Baseline numbers are from a second, warm-cache run of the unchanged files; the first cold run was 548 s / 160 s for the two export files.
export/test_leapp_export_flow.pyexport/test_rsl_rl_export_flow.pytest_rsl_rl_wrapper.pyThat is about 2.4 min (16%) off a ~15 min suite.
test_leapp_recurrent_state.pystill passes (7 cases) and carries the recurrent-state check removed from the export flow file.Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatsource/<pkg>/changelog.d/for every touched package (do not editCHANGELOG.rstor bumpextension.toml— CI handles that)CONTRIBUTORS.mdor my name already exists there