[feat] GenRL: add runtime and memory stability helpers

[Abecid]

Extracted from #1391.

GenRL-Stack: 3/6

Purpose

Add focused runtime stability helpers for GenRL sampling, evaluation, EMA cadence, launch entrypoints, and numerical SDE behavior.

This keeps the memory/runtime plumbing separate from reward compatibility, PPO loop changes, LoRA support, and recipe defaults.

Fixes #

Changes

• Make same-latent prompt seeding deterministic across Python processes.
• Move decoded rollout videos to CPU before reward/logging handoff.
• Support sync reward scoring mode for GPU reward execution.
• Add bounded evaluation controls: max batches and deterministic eval seed.
• Add EMA update interval support.
• Guard SDE log-prob computation against zero-variance deterministic steps.
• Launch training through -m fastvideo.train.entrypoint.train in shell scripts.

Test Plan

python -m py_compile \
  fastvideo/train/callbacks/ema.py \
  fastvideo/train/methods/rl/utils/evaluation.py \
  fastvideo/train/methods/rl/utils/pipeline.py \
  fastvideo/train/methods/rl/utils/sampling.py \
  fastvideo/train/methods/rl/utils/sde.py

Test Results

Test output

py_compile passed locally.

[mergify]

⚠️ PR title format required

Your PR title must start with a type tag in brackets. Examples:

• [feat] Add new model support
• [bugfix] Fix VAE tiling corruption
• [refactor] Restructure training pipeline
• [perf] Optimize attention kernel
• [ci] Update test infrastructure
• [infra] Add activation trace hooks
• [docs] Add inference guide
• [misc] Clean up configs
• [new-model] Port Flux2 to FastVideo
• [skill] Add add-model agent skill

Valid tags: feat, feature, bugfix, fix, refactor, perf, ci, infra, doc, docs, misc, chore, kernel, new-model, skill, skills

Please update your PR title and the merge protection check will pass automatically.

[gemini-code-assist]

Code Review

This pull request introduces several improvements and fixes across training scripts, EMA callbacks, evaluation, and sampling logic. Key changes include switching to module-based execution in run scripts, adding an update interval to the EMA callback, introducing deterministic seeding using Blake2b hashing, and adding support for synchronous reward scoring with CPU offloading. The review feedback highlights three main improvement opportunities: clamping std_scale to math.sqrt(tiny) instead of tiny in sde.py to prevent underflow during squaring, moving the expensive torch.cuda.empty_cache() call outside the loop in sampling.py, and instantiating the torch.Generator outside the evaluation loop in evaluation.py to reduce overhead.

[gemini-code-assist]

Clamping std_scale to torch.finfo(std_scale.dtype).tiny is insufficient to prevent division by zero because std_scale**2 will underflow to 0.0 for any std_scale < sqrt(tiny). For float32, tiny is 1.17e-38, so any std_scale below 1.08e-19 will cause std_scale**2 to underflow to 0.0, leading to division by zero (NaN or inf).

Additionally, extremely small values of std_scale can cause severe numerical instability (exploding log-probabilities) if there is even a tiny discrepancy between prev_sample and prev_sample_mean due to precision limits.

It is safer to clamp std_scale to math.sqrt(torch.finfo(std_scale.dtype).tiny) or a small epsilon like 1e-5 to ensure numerical stability.

Suggested change

	std_scale = torch.clamp(
	std_scale,
	min=torch.finfo(std_scale.dtype).tiny,
	)
	std_scale = torch.clamp(
	std_scale,
	min=math.sqrt(torch.finfo(std_scale.dtype).tiny),
	)

[gemini-code-assist]

Calling torch.cuda.empty_cache() inside the loop for every sample when async_reward_scoring is False is highly inefficient. empty_cache() is an expensive operation that forces CUDA synchronization and releases cached memory back to the OS, which can significantly slow down the epoch processing.

Instead, you should call torch.cuda.empty_cache() once before the loop starts (e.g., right after _t_reward_wait = time.perf_counter()) if not async_reward_scoring, and remove it from inside the loop.

        if async_reward_scoring:
            rewards, _ = sample["rewards"].result()
        else:
            videos_cpu, prompts, prompt_metadata = sample["rewards"]
            rewards, _ = reward_fn(
                videos_cpu, prompts, prompt_metadata, True
            )

[gemini-code-assist]

Instantiating a new torch.Generator(device=device) inside the loop for every evaluation batch introduces unnecessary overhead, especially on CUDA devices.

To optimize performance, instantiate the generator once outside the loop (e.g., right before for batch_idx, ... in enumerate(test_dataloader):) and simply re-seed it inside the loop using generator.manual_seed(seed + batch_idx).