Re-enable the Kit visualizer pause checks (#7049)

Temporary workaround for NVBUG 6570125 while we wait for the upstream
renderer fix.

## Problem

RTX responsiveDenoising continues refining the image after physics
stops, so fresh renders captured during a pause are no longer stable.
The affected Kit visualizer checks are currently green for the wrong
reasons:
Tiled camera: #6658 changed the paused captures to reuse the cached
sensor frame, so the assertion effectively compares the same frame
against itself and can't detect renderer instability.
Viewport: still performs a real comparison, but the known denoiser noise
can push it over the existing threshold and the failure is absorbed by
retries.

## What this changes

This PR restores fresh renders for the tiled pause check and adjusts the
per-channel difference threshold only for the affected Kit RTX pause
paths.
The existing 100-pixel gate stays unchanged. We're changing how large a
per-channel difference must be before a pixel is counted, rather than
simply allowing more differing pixels.

| site | @50 | @80 | @100 | @120 | @150 | @160 | outcome |
|---|---|---|---|---|---|---|---|
| Kit viewport (Newton) | 67 | **12** | 8 | 7 | 3 | — | 80 |
| Kit viewport (PhysX) | 14 | 4 | 3 | 1 | 0 | — | 80 |
| Kit tiled (Newton) | 999 | 604 | 226 | 120 | 59 | **49** | 160 |
| Kit tiled (PhysX) | 103 | 4 | 4 | 2 | 0 | — | 80 |
| Newton ViewerGL | 0 | 0 | 0 | 0 | 0 | — | 50 |

Values are the number of differing pixels at each per-channel threshold.
For the margin-critical case (Kit tiled camera, Newton backend),
threshold 160 leaves at most 49 px of paused denoiser noise (worst of
n=36) against at least 160 px with real motion (weakest of n=72), both
measured with the denoiser enabled. With the existing 100-pixel gate the
known renderer noise passes at 2.0× margin while real motion is still
detected at 1.6×.



## Changes

`_assert_frames_remain_stable` now accepts a per-call
`channel_diff_threshold`; the default remains 50.
Kit RTX pause checks use 80 for the viewport, 160 for the tiled camera
on Newton, and 80 for the tiled camera on PhysX — three separate
constants, since the tiled PhysX cell measures 103 px at the default
threshold and must not silently track a later change to the viewport
value. Newton ViewerGL remains at the strict default since it doesn't
exercise the affected DLSS path.
`max_differing_pixels` remains 100 everywhere.
The tiled pause check is restored to fresh renders instead of cached
frames, so the assertion is actually exercising renderer stability
again.


## Validation

With retries disabled, the workaround passed 40/40 runs across all four
affected configurations.

As a control, restoring the original threshold of 50 reproduced the
failure 6/6 times at 766–949 differing pixels. Those six failures are
all the **tiled** assertion: the control ran under `pytest -x` and the
viewport test is defined first in the same file, so the run aborted at
the tiled test and the viewport site passed at 50 in those reps. The
control therefore demonstrates that the tiled workaround filters real
denoiser noise rather than simply making the test pass; it is not
evidence for the viewport site.

The viewport raise from 50 to 80 rests instead on production CI, where
the assertion has really been failing (`107 pixels differed, expected at
most 100`, with a historical band reaching 161 px), plus n=5 bench
samples showing 67 px at threshold 50 falling to 12 px at 80.

The same change was validated on three independent L40S CI runners
through #7055, with all four visualizer integration tests passing
first-pass, no retries. Restoring fresh renders also had no meaningful
runtime impact (~27s median).

This is a temporary workaround for NVBUG 6570125 and should be removed
or re-measured once the upstream fix is available. That revert is
load-bearing rather than cosmetic: with the denoiser disabled — the
world after the renderer fix — the weakest real motion in the governing
cell measures 80 px at threshold 160, which is *below* the 100-pixel
gate. Leaving 160 in place after the fix ships would make the tiled
check blind to the weakest motion it exists to catch, so the removal
comment on the constants is the thing guarding that.

---------

Signed-off-by: fanes <74020209+fatimaanes@users.noreply.github.com>

Author: fatimaanes

source/isaaclab_visualizers/changelog.d/fanes-nvbug-6570125-adjust-threshold.skip

@@ -0,0 +1,3 @@
+Test-only change: raises the per-channel difference threshold in the visualizer integration
+test helper for the Kit RTX pause checks, and restores fresh renders in the paused tiled
+capture (NVBUG 6570125 work-around). No user-facing behavior change.

source/isaaclab_visualizers/test/visualizer_integration_utils.py

@@ -175,6 +175,20 @@
 _TILED_CAMERA_MOTION_MIN_DIFFERING_PIXELS = 25
 """Minimum differing pixels for tiled camera motion checks."""
 
+# NVBUG 6570125 — Remove these overrides once it ships the fix and paused frames are stable again.
+_KIT_PAUSED_VIEWPORT_CHANNEL_DIFF_THRESHOLD = 80
+"""Per-channel threshold for paused Kit viewport comparisons (0–255 space)."""
+
+_KIT_PAUSED_TILED_CAMERA_NEWTON_CHANNEL_DIFF_THRESHOLD = 160
+"""Per-channel threshold for paused Kit tiled camera comparisons on Newton (0–255 space)."""
+
+_KIT_PAUSED_TILED_CAMERA_PHYSX_CHANNEL_DIFF_THRESHOLD = 80
+"""Per-channel threshold for paused Kit tiled camera comparisons on PhysX (0–255 space).
+
+Matches the viewport value but is kept separate: this cell measures 103 differing pixels at the
+default threshold, so it needs its own floor rather than tracking whatever the viewport uses.
+"""
+
 _FRAME_MIN_CHANNEL_RANGE = 10
 """Minimum per-frame channel range to reject all-one-color images."""
 
@@ -569,12 +583,14 @@ def _assert_frames_remain_stable(
     phase: str,
     debug_phase: str,
     max_differing_pixels: int = 100,
+    channel_diff_threshold: float = _FRAME_MOTION_CHANNEL_DIFF_THRESHOLD,
 ) -> None:
     """Assert two viewport frames are effectively unchanged while simulation is paused."""
-    n_diff = _count_significantly_differing_pixels(frame_a, frame_b)
+    n_diff = _count_significantly_differing_pixels(frame_a, frame_b, channel_diff_threshold=channel_diff_threshold)
     assert n_diff <= max_differing_pixels, (
         f"{case_label} failed to pause during {phase}: {n_diff} pixels differed, expected at most "
-        f"{max_differing_pixels}. Frame shape={_frame_shape_for_message(frame_a)}. "
+        f"{max_differing_pixels} with per-channel threshold {channel_diff_threshold} in 0-255 space. "
+        f"Frame shape={_frame_shape_for_message(frame_a)}. "
         f"Debug frames: {_current_visualizer_debug_dir()}/*{debug_phase}*.png."
     )
 
@@ -1259,6 +1275,7 @@ def _attempt_kit_pause():
                 case_label=case_label,
                 phase="pausing",
                 debug_phase="pausing",
+                channel_diff_threshold=_KIT_PAUSED_VIEWPORT_CHANNEL_DIFF_THRESHOLD,
             )
 
         try:
@@ -1400,16 +1417,13 @@ def _run_visualizer_tiled_camera_motion_test(env, visualizer, *, physics_kind: s
 
     def _attempt_pause():
         _set_kit_simulation_paused(env, True)
-        # Read the sensor's last completed frame while paused. Forcing a new RTX
-        # render here introduces TAA edge jitter even though physics is frozen.
-        paused_start_frame = _capture_visualizer_tiled_camera_rgb(
-            visualizer, label="2a_pausing_frame_20", force_recompute=False
-        )
+        # Re-render both paused captures: comparing the sensor's cached frame with itself cannot
+        # detect a renderer that keeps changing the image after physics stops.  The denoiser residue
+        # that motivated caching is handled by the per-channel threshold below (NVBUG 6570125).
+        paused_start_frame = _capture_visualizer_tiled_camera_rgb(visualizer, label="2a_pausing_frame_20")
         for _ in range(PAUSE_VIZ_N_STEP):
-            env.sim.render(skip_app_pumping=isinstance(visualizer, KitVisualizer))
-        paused_end_frame = _capture_visualizer_tiled_camera_rgb(
-            visualizer, label="2b_pausing_frame_25", force_recompute=False
-        )
+            env.sim.render()
+        paused_end_frame = _capture_visualizer_tiled_camera_rgb(visualizer, label="2b_pausing_frame_25")
         _save_visualizer_debug_phase_images(
             paused_start_frame,
             paused_end_frame,
@@ -1425,6 +1439,13 @@ def _attempt_pause():
             case_label=case_label,
             phase="pausing",
             debug_phase="pausing_tiled",
+            channel_diff_threshold=(
+                _KIT_PAUSED_TILED_CAMERA_NEWTON_CHANNEL_DIFF_THRESHOLD
+                if isinstance(visualizer, KitVisualizer) and physics_kind == "newton"
+                else _KIT_PAUSED_TILED_CAMERA_PHYSX_CHANNEL_DIFF_THRESHOLD
+                if isinstance(visualizer, KitVisualizer)
+                else _FRAME_MOTION_CHANNEL_DIFF_THRESHOLD
+            ),
         )
 
     try: