[rwiltz]

Keyboard, gamepad and SpaceMouse input now has one implementation in Isaac Capture. The six old SE(2)/SE(3) implementation classes, their SpaceMouse HID helpers, and the unused hidapi dependency are removed. The old config classes remain as data records deprecated in Isaac Lab 3.2: select_teleop_cfg translates them once into Isaac Capture configs before launch, preserving sensitivities, dead zones, gripper settings, activation and output device. There are no constructor adapters or automatic fallbacks for these devices. Other device types, including native OpenXR hand tracking, retain their own implementation and factory.

All repository callers move in this PR so removing the implementations leaves a working tree. teleop_se3_agent.py, record_demos.py, consolidated Mimic, pick-and-place and the manual keyboard check use Isaac Capture builders directly for their defaults. Replay and annotation use the visualizers' add_key_callbacks for hotkeys without starting a teleop session. Isaac Capture consumers receive their windows explicitly and close the device before the environment. The demo loops handle startup and pause; consolidated Mimic consumes the flat action directly and holds the gripper state while motion is paused.

The SO-101 stacking pipeline also uses the Isaac Capture 1.7 clutch API: the retired fixed orientation offset is removed, and the SDK owns engage-relative orientation seeded from the existing home transform. Its integration test executes real controller snapshots through the graph and checks numeric pose/gripper ordering, release/re-engage and reset.

The six public pipeline builders remain in one flat device_pipelines.py, with shared action packing only where required. This builds on merged #8158 and requires Isaac Capture 1.7 through the Linux teleop extra. It incorporates the remaining-caller migration previously planned for #8798; that follow-up overlaps this change and will need rebasing. There is no dependency on #8798.

Breaking: direct calls such as Se3Keyboard(cfg) are removed. Use se3_keyboard_teleop_cfg(...) (or the corresponding builder), pass the result to create_isaac_teleop_device, and enter the returned context manager. The feature guide documents standalone session setup and config migration. Existing environment config declarations continue to translate with a migration warning; they no longer point to removed constructors.

Validation:

• 137 focused tests passed with one skip, across 13 files run in separate processes against the pinned Isaac Capture 1.7 packages. Coverage includes real SDK numeric actions/config translation, window/session lifecycle, native factory callbacks, Haply, CLI entry points, replay, and demo startup/pause/cleanup.
• Seven deliberate regressions were rejected: a retained config constructor target, lost activation, skipped subclass migration, accepted missing Isaac Capture, cleared paused gripper, ignored demo pause, and repeated hotkeys.
• After the SO-101 compatibility fix, 25 focused pipeline/IK/gripper/selection tests passed. The new graph regression failed on the original constructor argument; deliberate missing-gripper and reordered-quaternion mutations are rejected by numeric output checks.
• Removed the file-existence assertions requested in review; the real pipeline tests retain config translation, versioned deprecation warnings and numeric output coverage.
• A centralized file audit confirmed config-only legacy packages, no imports of retired classes, no old constructor targets, no additional script wrapper, and no HID dependency.
• Full Sphinx HTML build passed with warnings treated as errors. Repository formatting/changelog checks, git diff --check, and uv lock --check --offline passed.

Physical devices and a full CloudXR session were not exercised locally. The legacy OpenXR constructor test could not collect in the bare test container because carb requires the Kit runtime; CI must cover it.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] No new blocking defect was established in the changes since the previous review.

Summary

Routes keyboard, gamepad, and SpaceMouse teleoperation through Isaac Capture while keeping the native-device fallback.

• Kit and Newton visualizers now report keyboard events through a shared input source.
• Keyboard pipelines now drive SE(2) and SE(3) teleoperation from the focused viewer.
• Gamepads now drive SE(2) and SE(3) teleoperation through Isaac Capture.
• SpaceMouse devices now drive SE(2) and SE(3) teleoperation through Isaac Capture.
• Teleoperation scripts now select matching Isaac Capture pipelines from device requests.

Previous-thread context:

• rwiltz explained that Isaac Capture tests device behavior upstream. Greptile conceded that broad request; the thread was resolved.
• The later-window finding is addressed by checking for unattached visualizers on each step. Its thread was manually resolved without an explanatory reply.
• The recording thread was manually resolved without an explanatory reply. The new guard prevents the direct demo discard, but it does not stop an operator reset from pausing the Capture session before the script checks the guard.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Config[Device configuration] --> Selection[Select Capture pipeline or native fallback]
    Window[Focused visualizer window] --> Keys[Key events]
    Keys --> Capture[Isaac Capture session]
    Plugins[Gamepad or SpaceMouse plugin] --> Capture
    Selection --> Capture
    Capture --> Device[IsaacTeleopDevice]
    Device --> Scripts[Teleoperation and recording scripts]

Loading

_{Reviews (4) · Last reviewed commit: "Relock with the repository's uv and tidy..." · Reviewed by Greptile}

[rwiltz]

@greptile

[isaaclab-review-bot]

Isaac Lab Review Bot

Needs attention: Design and architecture (1), Scope and over-engineering (2), Compatibility and deprecation (2), Style consistency (1), Test quality (1). Other reviewed categories look good.

• PR description: The body omits the isaacteleop-to-isaaccapture dependency and lockfile migration, retains the empty issue placeholder, and provides no validation commands; add these details, while no performance benchmark is required.
• Design and architecture: TeleopSessionLifecycle retrieves visualizers through SimulationContext.instance() and suppresses lookup failures, hiding keyboard loss; pass owned visualizers from the composition root and remove the broad guard.
• Scope and over-engineering: Minimal implementation: extend BaseVisualizer, KitVisualizer, NewtonVisualizer, TeleopSessionLifecycle, and both scripts; keep key_event_source, _plugins, device_selection, six builders, six export files, and Kit/Newton sources; conditionally forward SpaceMouse buttons and reuse reset_recording_instance. Suggested PR breakdown: 1) visualizer key API/backends; 2) dependency/lock migration; 3) builders/lifecycle controls; 4) script routing/docs, each with focused package tests.
• API: API changes look good.
• Compatibility and deprecation: Breaking changes: explicit device selection silently changes established routing; preserve old routing through a warned, documented deprecation with transition coverage, and regenerate uv.lock using repository uv.
• Implementation: Implementation looks good.
• Style consistency: IsaacTeleopDevice places four unannotated class constants among private methods; move them before instance methods and declare them as ClassVar[int].
• Test quality: The key-source, routing, session, and operator tests protect distinct contracts, but the RTX camera-key row repeats the same inherited GL-viewer path; keep one backend-independent case.

Significant concerns. Posted 7 inline findings.

Automated review; human maintainers own approval decisions.

[rwiltz]

@greptile

[rwiltz]

run-ci

[ooctipus]

run-ci

[AntoineRichard]

From a high-level the changes look good!

[ooctipus]

run-ci

[ooctipus]

run-ci

[ooctipus]

run-ci