[Workflow] Remove unused/poor tests from isaaclab_tasks - #7088
Conversation
Greptile SummaryThis PR streamlines
Confidence Score: 4/5The PR appears safe to merge, with one non-blocking gap in coverage for automatic PhysX fallback when OvPhysX is unavailable. CI routing and consolidated tests remain functional, but the removed single-backend PhysX fallback case has no direct equivalent in the remaining suite. Files Needing Attention: source/isaaclab_tasks/test/core/test_preset_kit_decision.py Important Files Changed
|
There was a problem hiding this comment.
Isaac Lab Review Bot
Reviewed the test-suite cleanup, preset-test consolidation, and CI retargeting. The dedicated video job now runs test_video_recording.py, ordinary task shards exclude that dependency-specific test, and the removed training job targeted a permanently skipped module. No actionable defect is established by the patch.
- Design and architecture: The change is confined to tests and workflow organization. UR10 Reach coverage is consolidated into the existing Reach preset matrix, Shadow Hand camera checks retain preset resolution and Warp compatibility cases, and implementation-layout or configuration snapshot assertions are removed in favor of retained behavioral coverage. A non-blocking discoverability tradeoff is that UR10 checks now reside in a Franka-named test module.
- API: No production API, task registration, configuration field, export, or CLI contract is changed. The retained test imports match their remaining uses, and the package includes an appropriate
.skipchangelog fragment for this non-user-visible cleanup. - Implementation: The workflow path is internally consistent:
test_video_recording.pyis excluded from all three standard task shards and selected by the dedicated job that installs MoviePy, while the obsolete environment-training workflow and skipped test are removed together. The workflow still relies on the shared action correctly parsing the comma-separatedexclude-pattern, but the supplied patch does not demonstrate a failure in that path.
No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.
Automated review; human maintainers own approval decisions.
Changes
Removed
test_record_video.py.Reason: It stepped environments but never checked recorded output. Changed the dedicated video CI job to run
test_video_recording.py. It verifies video files and frame content for each source and simultaneous recorders. That test already existed before, so we were actually double-testing but doing a bad job at it...Excluded
test_video_recording.pyfrom ordinary task shards.Reason: It needs the movie dependency and has a dedicated CI job.
Removed the environments-training CI job and skipped test module.
Reason: The entire collected matrix was permanently skipped anyways, and was never actually run. This testing is also now in the hands of the perf/benchmarking team, and there is an expected regression test coming soon!
Removed
test_lift_package_structure.py.Reason: It locked a completed migration's file layout, and just asserted stuff. There was no real behaviour testing.
Removed the GearAssembly default-count and DR Legs tuning snapshot tests.
Reason: They restated hard-coded configuration values without exercising outcomes.
Removed Lift migration and runner configuration snapshots.
Reason: They repeated authored settings; the camera-normalization behavior test remains.
Removed the disabled deterministic-training E2E and backend-path assertions.
Reason: They were never run or only enforced internal file placement.
Consolidated runtime/Kit preset decision tests.
Reason: Two modules covered the same resolution outcomes.
Consolidated Franka and UR10 reach preset tests.
Reason: Shared preset behavior now uses one matrix; robot-specific contracts remain explicit.
Consolidated Shadow Hand camera preset tests and removed skipped OVRTX cases.
Reason: The prior cases duplicated validation and permanently skipped coverage.
Validation
uv run python -m pytest source/isaaclab_tasks/test/core/test_preset_kit_decision.py source/isaaclab_tasks/test/core/test_train_scripts_deterministic.py -q -k 'not registered_task_physx_presets_keep_auto_selection_explicit'uv run python -m pytest --collect-only -q source/isaaclab_tasks/test/core/test_reach_franka_presets.py source/isaaclab_tasks/test/core/test_runtime_compatibility.py source/isaaclab_tasks/test/core/test_shadow_hand_camera_presets.pyuv run isaaclab -f.skipfragment to be staged before it can detect it.