[feat] GenRL: add explicit HPSv3 VideoAlign recipes

[Abecid]

Extracted from #1391.

GenRL-Stack: 6/6

Purpose

Update GenRL example recipes after the underlying reward/runtime/PPO/LoRA changes are split into focused PRs.

This also renames the previous LongCat-labelled config to avoid confusion with the actual LongCat-Video model. The recipe remains derived from GenRL's LongCat-style setup, but the filename now reflects the reward stack.

Fixes #

Changes

• Update GenRL OCR config paths and beta defaults for no-reference launches.
• Add HPSv3 + VideoAlign reward recipe defaults.
• Configure full-parameter 4-GPU debug settings separately from LoRA options.
• Set prompt dataset paths to GenRL/datasets/....
• Add eval, microbatch accumulation, EMA interval, and optimizer cadence defaults.
• Rename genrl_wan2.1_t2v_1.3B_longcat.yaml to genrl_wan2.1_t2v_1.3B_hpsv3_videoalign.yaml.

Test Plan

python - <<'PY'
import yaml
from pathlib import Path

for path in [
    Path("examples/train/configs/genrl_wan2.1_t2v_1.3B_hpsv3_videoalign.yaml"),
    Path("examples/train/configs/genrl_wan2.1_t2v_1.3B_ocr.yaml"),
]:
    with path.open() as f:
        yaml.safe_load(f)
print("GenRL configs parse")
PY

Test Results

Test output

GenRL configs parse

[mergify]

⚠️ PR title format required

Your PR title must start with a type tag in brackets. Examples:

• [feat] Add new model support
• [bugfix] Fix VAE tiling corruption
• [refactor] Restructure training pipeline
• [perf] Optimize attention kernel
• [ci] Update test infrastructure
• [infra] Add activation trace hooks
• [docs] Add inference guide
• [misc] Clean up configs
• [new-model] Port Flux2 to FastVideo
• [skill] Add add-model agent skill

Valid tags: feat, feature, bugfix, fix, refactor, perf, ci, infra, doc, docs, misc, chore, kernel, new-model, skill, skills

Please update your PR title and the merge protection check will pass automatically.

[gemini-code-assist]

Code Review

This pull request updates the training configuration files for GenRL/Video GRPO with Wan 2.1 T2V 1.3B, adjusting hyperparameters for full fine-tuning, updating dataset paths, and introducing new parameters. However, several of the newly added configuration parameters—including accumulate_ppo_microbatches, loss_reweighting: flash_tgr, optimizer_step_per_timestep, log_post_update_kl, and update_interval for the EMA callback—are not supported or implemented in the underlying training code. These unsupported parameters should be removed or corrected to prevent runtime errors and silent failures.

[gemini-code-assist]

The parameter accumulate_ppo_microbatches is not parsed or implemented in GenRLMethod (see fastvideo/train/methods/rl/genrl.py). Currently, the training loop in _ppo_train performs an optimizer step and zeros gradients for every micro-batch in batched_list rather than accumulating gradients across them. This means the behavior described in the comment (averaging PPO losses and applying one optimizer update) is not actually happening.

  # Sample multiple rollout microbatches. Note: gradient accumulation across
  # microbatches is not currently supported by the training method.
  num_batches_per_epoch: 4

[gemini-code-assist]

The loss_reweighting value "flash_tgr" is not implemented in GenRLMethod._ppo_train (see fastvideo/train/methods/rl/genrl.py). Only "longcat" is supported; any other value will silently default to rw_scale = 1.0 (no reweighting). If temporal gradient rectification is intended, it needs to be implemented in the training method first. Otherwise, set this to null to make it clear that no reweighting is applied.

  # No loss reweighting is applied (flash_tgr is not implemented in the training method).
  loss_reweighting: null
  loss_reweighting_clip: null

[gemini-code-assist]

The EMACallback class (defined in fastvideo/train/callbacks/ema.py) does not accept an update_interval parameter in its __init__ method, nor does it support **kwargs. Passing update_interval: 8 here will likely cause a TypeError at startup when the callback is instantiated. Additionally, the callback currently updates the EMA on every single step once started, ignoring any interval setting. You should remove this parameter.

  ema:
    decay: 0.9
    start_iter: 0