fix result handling and metric labels in the benchmark scripts

generate_video returns a plain dict, so getattr(result, ...) always hit the
fallback: generation_time silently became wall time and peak_memory_mb was
always None. Use dict access.

Label the measured metric honestly: generation_time is the full pipeline
(text-encode + denoise + decode), not denoise. Also: the sm_121 runtime
allowlist landed via #1647; #1598 is the remaining kernel build.

Author: SolitaryThinker

docs/getting_started/installation/spark_performance.md

@@ -109,7 +109,7 @@ this hardware, for reasons specific to it:
 | Building FlashAttention | **no speedup** — Torch SDPA already hits an efficient flash kernel on `sm_121`, and FA2 ties it | ❌ not worth building |
 | `torch.compile` of the VAE decode | recompile storm (per-frame varying shapes) → ~1.1× | ❌ dead end |
 | Linear (fp8 / nvfp4) quantization on long-sequence models (e.g. Cosmos) | ~nothing — see below | ❌ wrong lever here |
-| FP4 attention (`ATTN_QAT_INFER`) | works on `sm_121` (#1598); helps, but needs a QAT-trained checkpoint | ⚠️ opt-in — see below |
+| FP4 attention (`ATTN_QAT_INFER`) | works on `sm_121` (runtime allowlist landed in #1647; kernel build is #1598); helps, but needs a QAT-trained checkpoint | ⚠️ opt-in — see below |
 | FP4 linear on short-sequence models (LTX2) | up to −24% denoise at 1080p (#1594) | ⚠️ model/resolution-dependent |
 
 ### Why linear quantization is the wrong lever on long-sequence models
@@ -132,7 +132,7 @@ there (#1594). The rule: **on the GB10, the lever that matters is attention
 Block-scaled FP4 works on `sm_121` under CUDA 13:
 
 - **FP4 attention** (`FASTVIDEO_ATTENTION_BACKEND=ATTN_QAT_INFER`, #1598) is
-  numerically correct on the GB10 and ~6% faster denoise, but it only preserves
+  numerically correct on the GB10 and ~6% faster end-to-end generation, but it only preserves
   quality on a **quantization-aware-distilled checkpoint** (e.g.
   `FastVideo/FastWan-QAD-1.3B`) — stock weights aren't trained to tolerate it.
 - **FP4 linear** helps only where sequences are short (LTX2, above).

examples/inference/optimizations/qad_fp4_ab.py

@@ -26,9 +26,10 @@
 invocation and dumps a C stack on any hard crash, so a single misbehaving arm
 can never take the others down with it; the runbook loops it four times with
 different env. Quality is the eye/ear on the saved mp4 + a matching-frame still;
-timing is the mean generation_time over the measured runs.
+timing is the mean generation_time (full pipeline: text-encode +
+denoise + VAE decode) over the measured runs.
 
-On the GB10, expect FP4 attention ~6% faster denoise vs bf16 and quality-neutral
+On the GB10, expect FP4 attention ~6% faster end-to-end generation vs bf16 and quality-neutral
 by eye on the QAD checkpoint (both share the 3-step distill's quality ceiling).
 FP4 *linear* is roughly break-even at 1.3B/480p (the per-call quantize overhead
 eats the small-GEMM saving in eager mode); its win shows at higher resolution
@@ -183,9 +184,12 @@ def _generate():
         last = _generate()
         torch.cuda.synchronize()
         wall = time.perf_counter() - t0
-        denoise_times.append(getattr(last, "generation_time", wall))
+        # generate_video returns a plain dict; attribute access would always
+        # fall back to wall time.
+        gen_t = last.get("generation_time") if isinstance(last, dict) else None
+        denoise_times.append(gen_t if gen_t is not None else wall)
         print(f"[qad] {tag} run {i + 1}/{runs}: {wall:.2f}s wall "
-              f"(denoise {denoise_times[-1]:.2f}s)")
+              f"(gen {denoise_times[-1]:.2f}s)")
 
     # The pipeline wrote the mp4 (full known-good encode) into arm_dir; report
     # it and pull a matching-frame still from the [b,c,t,h,w] samples tensor
@@ -206,7 +210,7 @@ def _generate():
         print("[qad] note: no 5-D samples tensor; grab a frame from the mp4 above")
 
     mean = sum(denoise_times) / len(denoise_times)
-    print(f"\n[qad][{tag}] denoise mean {mean:.2f}s over {runs} runs "
+    print(f"\n[qad][{tag}] generation mean {mean:.2f}s over {runs} runs "
           f"({warmup} warmup, {steps} steps)")
     generator.shutdown()
 

examples/inference/optimizations/spark_benchmark.py

@@ -93,11 +93,16 @@ def _gen():
                                           save_video=True,
                                           sampling_param=sampling_param)
         torch.cuda.synchronize()
-        dt = getattr(video, "generation_time", time.perf_counter() - t0)
+        # generate_video returns a plain dict (legacy result), not an object —
+        # attribute access would silently fall back to wall time / None.
+        dt = video.get("generation_time") if isinstance(video, dict) else None
+        if dt is None:
+            dt = time.perf_counter() - t0
         # Peak memory is measured *inside the worker process* that runs the
         # pipeline and surfaced on the result; reading torch's allocator in this
         # (main) process would report ~0 because the allocations aren't here.
-        return dt, getattr(video, "peak_memory_mb", None)
+        peak = video.get("peak_memory_mb") if isinstance(video, dict) else None
+        return dt, peak
 
     for _ in range(args.warmup):
         _gen()