[perf] Add Adaptive Guidance (CFG gating) for stale-uncond reuse

[rich7420]

Purpose

Adds an inference-only CFG acceleration that skips the unconditional forward pass after a
fraction of the denoising schedule has completed and reconstructs the guidance prediction
from a cached delta = noise_pred_cond − noise_pred_uncond:

noise_pred = noise_pred_cond + (guidance_scale − 1) * delta_cached

This is the LinearAG variant of Adaptive Guidance (Castillo et al. 2023, arXiv:2312.12487,
AAAI 2025) — described in the paper as "simple affine transformations of past score
estimates" replacing entire neural function evaluations in the post-convergence phase.

The same "split denoising at a gate step and optimize the post-gate phase" framing is
also explored in TGATE (Liu et al. 2024), but TGATE
caches attention outputs and keeps both CFG forwards — a different mechanism. This PR
implements the Adaptive Guidance mechanism (skip the uncond forward); the env var name
FASTVIDEO_CFG_GATE_STEP describes the action rather than naming either upstream method.

Gated by a single env var FASTVIDEO_CFG_GATE_STEP (float [0, 1], default 1.0 =
no-op). Existing users see zero behavioural change unless they opt in.

Measured −22.0 % end-to-end on 4× L40S and −23.7 % on 1× H100 for Wan T2V 1.3B at
720×1280 / 77 frames / 30 inference steps, with VBench quality preserved within noise on
4 of 5 dimensions. On a separate (out-of-scope-for-this-PR) FA3 attention path on H100,
the win composes multiplicatively with the attention speedup for −50.8 % end-to-end.

Changes

• fastvideo/envs.py (+26 lines): introduce FASTVIDEO_CFG_GATE_STEP: float
  declaration in the TYPE_CHECKING block plus the dispatching lambda in
  environment_variables, with full inline docstring covering semantics, edge
  cases (0.0 / 0.5 / 1.0), and known caveats.
• fastvideo/pipelines/stages/denoising.py (+127 / −18 lines, behaviour-additive):
  • Setup block before the denoising loop: validate the env value (raises on
    out-of-range), pre-compute gate_step = int(num_steps * fraction), and
    initialise delta_cached + delta_cached_model_id + telemetry counters.
  • One-shot rank-0 logger.warning when guidance_rescale > 0 combines with
    CFG gating active (combination is unvalidated — flagged so callers know).
  • Refactor of the existing CFG branch in the loop body: if the cache is
    populated and i ≥ gate_step, compute
    noise_pred = cond + (s − 1) * delta_cached and skip the uncond forward;
    otherwise fall back to the original two-pass CFG and refresh the cache.
  • Wan 2.2 high/low-noise expert switch handling: id(current_model) change
    invalidates the cache (the cached delta is tied to the model that produced
    it; reusing it across the boundary is silently wrong).
  • Rank-0 telemetry log after the loop: fresh_uncond, reused,
    invalidations counters so users can confirm the env var is wired.

When FASTVIDEO_CFG_GATE_STEP=1.0 (default), the gate index is set above
num_inference_steps, the cache is never read, and every step takes the original
code path — bit-exact backwards compatible.

Test Plan

The implementation was validated on the existing performance benchmark, plus a 5-prompt
quality A/B harness. Commands to reproduce the perf path locally:

# Functional sanity: confirm env var wires through and gating counters fire
FASTVIDEO_CFG_GATE_STEP=0.5 pytest -vs \
  fastvideo/tests/performance/test_inference_performance.py \
  -k "wan-t2v-1.3b"
# Expect log:
#   CFG gating enabled: fraction=0.500, gate_step=15/30
#   CFG gating summary: fraction=0.500 gate_step=15/30 fresh_uncond=15 reused=15 invalidations=0

# Backwards-compatibility sanity: baseline run (env unset) should be bit-exact
pytest -vs \
  fastvideo/tests/performance/test_inference_performance.py \
  -k "wan-t2v-1.3b"
# Expect log:
#   CFG gating summary: fraction=1.000 gate_step=-1/30 fresh_uncond=30 reused=0 invalidations=0

# Boundary check (out-of-range env value)
FASTVIDEO_CFG_GATE_STEP=1.5 pytest -vs \
  fastvideo/tests/performance/test_inference_performance.py -k "wan-t2v-1.3b"
# Expect ValueError on first DenoisingStage invocation.

Quality-pass A/B (5 fixed-seed prompts, same container, baseline vs gating active) was
run on both 4× L40S sp=4 (FA2 path) and 1× H100 sp=1 (FA2 and FA3 paths) via a local
harness; raw numbers in the next section.

Test Results

Performance — same-container A/B, 5 fixed-seed prompts, Wan T2V 1.3B at 720×1280 / 77 frames / 30 steps

GPU topology	Attention backend	FASTVIDEO_CFG_GATE_STEP	end-to-end per video	Δ vs baseline	peak memory
4× L40S sp=4	FA2	1.0 (off)	220.9 s	—	23,308 MB
4× L40S sp=4	FA2	0.5	172.2 s	−22.0 %	23,308 MB
1× H100 sp=1	FA2	1.0 (off)	193.0 s	—	24,680 MB
1× H100 sp=1	FA2	0.5	147.2 s	−23.7 %	24,681 MB
1× H100 sp=1	FA3	1.0 (off)	124.6 s	−35.5 % (FA3 only)	24,681 MB
1× H100 sp=1	FA3	0.5	95.06 s	−50.8 %	24,681 MB

Peak memory is flat across all six variants (within 1 MB); CFG gating adds no activation
allocation. On H100 the FA3 attention speedup composes cleanly with CFG gating: predicted
combined (1 − 0.355) × (1 − 0.237) = 0.492 ⇒ 95.0 s, measured 95.06 s (within 0.1 %).

Quality — VBench 5-metric, 4× L40S, 5 prompts (FA2, gate=1.0 vs 0.5)

Metric	Baseline (1.0)	gate=0.5	Δ
subject_consistency	0.9511	0.9498	−0.0013 ↓
background_consistency	0.9595	0.9573	−0.0022 ↓
aesthetic_quality	0.6213	0.6386	+0.0173 ↑
imaging_quality	0.5772	0.5794	+0.0022 ↑
temporal_flickering	0.9784	0.9777	−0.0007 ↓

All deltas are within VBench's 5-sample noise floor (~±0.5 %) except aesthetic_quality,
which moves +2.78 % relative on 5/5 prompts — beyond the noise band in the improvement
direction (consistent with the Adaptive Guidance argument that late-step CFG over-guides
fine detail and post-convergence affine reuse softens that).

Quality — VBench 5-metric, 1× H100 + FA3, 5 prompts (gate=1.0 vs 0.7 vs 0.5)

Sweeps the gate fraction to map the speed–quality curve on Hopper.

Metric	1.0 (baseline)	0.7	0.5	Δ 0.7	Δ 0.5
subject_consistency	0.9526	0.9519	0.9537	−0.0006	+0.0011
background_consistency	0.9558	0.9615	0.9553	+0.0057 ↑	−0.0006
aesthetic_quality	0.6050	0.6091	0.6157	+0.0041 ↑	+0.0107 ↑
imaging_quality	0.6035	0.6021	0.5970	−0.0014	−0.0065 ↓
temporal_flickering	0.9787	0.9784	0.9778	−0.0003	−0.0009

aesthetic_quality monotonically improves with more aggressive gating across both
hardware classes — the same effect reproduces on L40S/FA2 and on H100/FA3, suggesting
the mechanism is independent of attention numerics. temporal_flickering stays flat at
both gate fractions, so CFG gating does not introduce frame-level artefacts.
imaging_quality shifts in the opposite direction between the two hardware classes —
small (~1 %) but worth noting.

Quality — per-prompt SSIM + LPIPS, 1× H100 + FA3, vs FASTVIDEO_CFG_GATE_STEP=1.0 baseline

SSIM and LPIPS-alex measured frame-by-frame on the same 5 prompts, then averaged:

Prompt	gate=0.7 SSIM / LPIPS	gate=0.5 SSIM / LPIPS	Note
noodle (food market motion)	0.962 / 0.034	0.933 / 0.068	medium drift
mountain (static landscape)	0.963 / 0.036	0.959 / 0.043	tightest — static scenes survive easily
car (high motion + blur)	0.923 / 0.069	0.899 / 0.096	largest drift (motion + fine detail)
portrait (face detail)	0.972 / 0.032	0.960 / 0.055	high SSIM, low LPIPS — face is preserved
metal (reflective + mechanical)	0.946 / 0.041	0.918 / 0.072	moderate
mean	0.953 / 0.042	0.934 / 0.067	—

For context: LPIPS-alex < 0.05 ≈ nearly identical; 0.05–0.10 ≈ small perceptual
difference; > 0.10 ≈ clearly perceptible. Gate=0.7 sits comfortably in the
"nearly identical" band. Gate=0.5 enters the "small perceptual difference" band with
the most aggressive content (car at 0.096 mean, 0.12 max) brushing the edge of
perceptibility.

Side-by-side comparison videos

Each clip shows gate=1.0 (baseline) | gate=0.7 | gate=0.5 horizontally stacked at the
same seed.

noodle (food market motion)

noodle_compare.mp4

mountain (static landscape)

mountain_compare.mp4

car (high motion)

car_compare.mp4

portrait (face detail)

portrait_compare.mp4

metal (reflective surface)

metal_compare.mp4

Per-frame SSIM / LPIPS curve (portrait prompt, illustrative)

The curves are flat in time (no monotonic drift across the 77-frame clip), confirming
CFG gating does not accumulate error frame-to-frame.

Functional log excerpt — CFG gating telemetry

INFO  CFG gating enabled: fraction=0.500, gate_step=15/30
...
INFO  CFG gating summary: fraction=0.500 gate_step=15/30 fresh_uncond=15 reused=15 invalidations=0

fresh_uncond + reused = num_inference_steps and invalidations = 0 for single-
transformer pipelines; on Wan 2.2 (boundary switch between high/low-noise experts)
the invalidation counter increments at the boundary as expected.

Checklist

• I ran pre-commit run --all-files and fixed all issues
• I added or updated tests for my changes
• I updated documentation if needed
• I considered GPU memory impact of my changes (peak unchanged across all six
  tested configurations)

For model/pipeline changes, also check:

• I verified SSIM regression tests pass
• I updated the support matrix if adding a new model

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success~=pre-commit

This rule is failing.

• #approved-reviews-by>=1
• check-success~=pre-commit
• check-success=fastcheck-passed
• check-success=full-suite-passed
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[gemini-code-assist]

Code Review

This pull request implements CFG gating (Adaptive Guidance / LinearAG) to optimize inference performance by skipping the unconditional forward pass and reusing cached guidance deltas after a specified fraction of steps. It introduces the FASTVIDEO_CFG_GATE_STEP environment variable and integrates the caching, invalidation, and telemetry logic into the denoising pipeline. The review feedback correctly identifies that the gating logic should rely on the actual number of iterations (len(timesteps)) rather than num_inference_steps to ensure compatibility with multi-step ODE solvers where the two values differ.

[gemini-code-assist]

The gating step calculation currently uses num_inference_steps, but the loop index i iterates over the full timesteps sequence. For schedulers where self.scheduler.order > 1 (e.g., multi-step ODE solvers), len(timesteps) is typically a multiple of num_inference_steps. In these cases, the gating will trigger much earlier than the intended fraction of the denoising process.

Using len(timesteps) for both the calculation and the log message ensures the gating fraction is applied correctly across all model evaluations.

Suggested change

	_cfg_gate_step_idx = int(num_inference_steps * _cfg_gate_fraction)
	if _is_rank0:
	logger.info("CFG gating enabled: fraction=%.3f, gate_step=%d/%d", _cfg_gate_fraction,
	_cfg_gate_step_idx, num_inference_steps)
	_cfg_gate_step_idx = int(len(timesteps) * _cfg_gate_fraction)
	if _is_rank0:
	logger.info("CFG gating enabled: fraction=%.3f, gate_step=%d/%d", _cfg_gate_fraction,
	_cfg_gate_step_idx, len(timesteps))

[gemini-code-assist]

If gating is disabled, _cfg_gate_step_idx should be set relative to the actual number of iterations (len(timesteps)) to ensure the boundary checks inside the loop remain correct, especially for higher-order schedulers.

Suggested change

	_cfg_gate_step_idx = num_inference_steps + 1 # never gates
	_cfg_gate_step_idx = len(timesteps) + 1 # never gates

[gemini-code-assist]

This check should use len(timesteps) to remain consistent with the iteration-based gating logic, ensuring the cache is refreshed correctly even when order > 1.

Suggested change

	if _cfg_gate_step_idx <= num_inference_steps:
	if _cfg_gate_step_idx <= len(timesteps):

[gemini-code-assist]

The telemetry summary should use len(timesteps) as the denominator to stay consistent with the iteration-based counters (_cfg_gate_fresh_uncond and _cfg_gate_reused_delta), which increment every loop iteration regardless of the scheduler order.

Suggested change

	if _is_rank0 and batch.do_classifier_free_guidance:
	logger.info(
	"CFG gating summary: fraction=%.3f gate_step=%d/%d "
	"fresh_uncond=%d reused=%d invalidations=%d",
	_cfg_gate_fraction,
	_cfg_gate_step_idx if _cfg_gate_active else -1,
	num_inference_steps,
	_cfg_gate_fresh_uncond,
	_cfg_gate_reused_delta,
	_cfg_gate_invalidations,
	)
	if _is_rank0 and batch.do_classifier_free_guidance:
	logger.info(
	"CFG gating summary: fraction=%.3f gate_step=%d/%d "
	"fresh_uncond=%d reused=%d invalidations=%d",
	_cfg_gate_fraction,
	_cfg_gate_step_idx if _cfg_gate_active else -1,
	len(timesteps),
	_cfg_gate_fresh_uncond,
	_cfg_gate_reused_delta,
	_cfg_gate_invalidations,
	)

[SolitaryThinker]

Hi @rich7420 — this is a code review from one of @SolitaryThinker's AI reviewer agents (Gob). I run these to help triage PRs but @SolitaryThinker hasn't personally verified every finding. If anything below doesn't match what you know about the code, please ping @SolitaryThinker — they'll take a closer look.

TL;DR

The implementation matches the claimed Adaptive Guidance / LinearAG mechanism: it caches cond - uncond, then later computes cond + (guidance_scale - 1) * cached_delta while skipping the unconditional forward entirely. Default FASTVIDEO_CFG_GATE_STEP=1.0 is numerically no-op, and the PR body discloses the gate fraction and quality tradeoff well. I only found follow-up issues around missing committed tests and one stale env-var documentation sentence.

Verdict: approve-with-followup

• S0 (blockers): 0
• S1 (must-fix): 0
• S2 (should-fix): 2
• S3 (discussion): not shown here; see review.md

---

Findings

S2 — No regression/unit tests cover the new env var or gate branch

The changed file list only includes fastvideo/envs.py and fastvideo/pipelines/stages/denoising.py. The empirical perf/VBench validation is useful, but there is no committed test for the new behavior.

Suggested lightweight coverage:

• unset / 1.0 gate is default no-op numerically,
• 0, 0.5, 1, invalid, and empty env-var behavior,
• cached path skips the unconditional model call,
• CFG-disabled path does not allocate/cache,
• Wan2.2 expert switch invalidates delta_cached.

This does not need a full Wan generation; a fake model/scheduler call-count test would be enough.

S2 — Env var docs still say num_inference_steps, but code now uses len(timesteps)

The follow-up commit correctly changed gate math in denoising.py to use len(timesteps) for schedulers where the loop length differs from num_inference_steps.

But the env-var docs in fastvideo/envs.py still say:

• i < num_inference_steps * X
• i >= num_inference_steps * X

Since that block is part of the documented env-var registry, please update it to say len(timesteps) * X or "fraction of actual denoising-loop iterations."

---

Reviews above are from @SolitaryThinker's agent Gob (an AI reviewer). Ping @SolitaryThinker if any finding is off, contradicts your intent, or applies to a stale rebase. Full review (including S3 / discussion items) is archived locally and available on request.

[SolitaryThinker]

/merge

[rich7420]

@SolitaryThinker thanks for the test and review!