[ci] add per-method single-step training tests for fastvideo.train (PR 5a-i/9)

Phase 2 / PR 5/9 of the fastvideo.train CI plan, split into a
first-half ``5a-i`` slice that establishes the per-method test
pattern.  A follow-up ``5a-ii`` will layer a device-keyed
grad-norm regression on top of the same harness.

Adds two GPU smoke tests under a new
``fastvideo/tests/train/methods/`` directory:

* ``test_wan_finetune.py`` — ``WanModel`` + ``FineTuneMethod``
* ``test_wan_causal_dfsft.py`` — ``WanCausalModel`` +
  ``DiffusionForcingSFTMethod``

Both tests construct the method via its public constructor
(``method = FineTuneMethod(cfg=cfg.method, role_models=...)``),
build a tiny synthetic ``raw_batch`` (text embed + mask + vae
latent), and run one end-to-end step:

    method.on_train_start()
    loss_map, outputs, _ = method.single_train_step(batch, 0)
    method.backward(loss_map, outputs, grad_accum_rounds=1)

Asserts that ``loss_map["total_loss"]`` is finite and that the
first transformer block (``model.transformer.blocks[0]``) has
trainable parameters with finite, non-zero gradients.  The first
block's grad is computed *last* during backprop, so a healthy
grad there implies the full forward + chain-rule path is intact —
keeping the assertion surface to a single block keeps the
reference data tiny for the follow-up regression PR.

The harness deliberately avoids the full ``Trainer`` (no FSDP
wrap, no data loader, no checkpointing) so the tests stay focused
on the method-level wiring that the model-loading tests in
``tests/train/models`` don't cover.

CI plumbing:

* ``fastvideo/tests/modal/pr_test.py``: add
  ``./fastvideo/tests/train/methods`` to the
  ``run_train_framework_tests`` pytest path, and add a matching
  ``--ignore`` entry in ``run_unit_test`` so the CPU runner doesn't
  pick it up.
* ``docs/contributing/testing.md``: mention the new ``methods/``
  subdirectory in the Train Framework Tests entry.

Two new fixture YAMLs under ``fastvideo/tests/train/fixtures/``
mirror the structure of the existing 4/9 fixtures but flip
``trainable`` to ``true`` and include the minimum
``training.distributed`` + ``training.optimizer`` + ``training.loop``
keys needed by the method's ``__init__`` and ``on_train_start``.

Author: alexzms

docs/contributing/testing.md

@@ -6,7 +6,7 @@ This guide explains how to add and run tests in FastVideo. The testing suite is
 
 * **Unit Tests**: Located in `fastvideo/tests/api`, `fastvideo/tests/dataset`, `fastvideo/tests/entrypoints`, `fastvideo/tests/workflow`, and the CPU-only subset of `fastvideo/tests/train` (callbacks, utils). These test individual functions and classes.
 * **Component Tests**: Located in `fastvideo/tests/encoders`, `fastvideo/tests/transformers`, and `fastvideo/tests/vaes`. These verify the loading and basic functionality of model components.
-* **Train Framework Tests** (GPU): Located in `fastvideo/tests/train/models`. Cover model loading + forward smoke for the new `fastvideo/train/` framework. Triggered via `/test train-framework` or as part of the Full Suite.
+* **Train Framework Tests** (GPU): Located in `fastvideo/tests/train/models` (model loading + forward smoke) and `fastvideo/tests/train/methods` (per-method single training step). Cover the new `fastvideo/train/` framework end-to-end on real checkpoints with tiny synthetic batches. Triggered via `/test train-framework` or as part of the Full Suite.
 * **SSIM Tests**: Located in `fastvideo/tests/ssim`. These are regression tests that compare generated videos against reference videos using the Structural Similarity Index Measure (SSIM) to detect quality degradation.
 * **Training Tests**: Located in `fastvideo/tests/training`. These validate training loops, loss calculations, and specific training techniques like LoRA, Distillation, and VSA.
 * **Inference Tests**: Located in `fastvideo/tests/inference`. These test specialized inference pipelines and optimizations (e.g., VSA, V-MoBA).

fastvideo/tests/modal/pr_test.py

@@ -214,7 +214,7 @@ def run_self_forcing_tests():
 @app.function(gpu="L40S:1", image=image, timeout=900)
 def run_unit_test():
     run_test(
-        "pytest ./fastvideo/tests/api/ ./fastvideo/tests/contract/ ./fastvideo/tests/dataset/ ./fastvideo/tests/workflow/ ./fastvideo/tests/entrypoints/ ./fastvideo/tests/train/ --ignore=./fastvideo/tests/entrypoints/test_openai_api_integration.py --ignore=./fastvideo/tests/train/models -vs"
+        "pytest ./fastvideo/tests/api/ ./fastvideo/tests/contract/ ./fastvideo/tests/dataset/ ./fastvideo/tests/workflow/ ./fastvideo/tests/entrypoints/ ./fastvideo/tests/train/ --ignore=./fastvideo/tests/entrypoints/test_openai_api_integration.py --ignore=./fastvideo/tests/train/models --ignore=./fastvideo/tests/train/methods -vs"
     )
 
 
@@ -228,7 +228,7 @@ def run_unit_test():
               volumes={"/root/data": model_vol})
 def run_train_framework_tests():
     run_test(
-        "export HF_HOME='/root/data/.cache' && hf auth login --token $HF_API_KEY && pytest ./fastvideo/tests/train/models -vs"
+        "export HF_HOME='/root/data/.cache' && hf auth login --token $HF_API_KEY && pytest ./fastvideo/tests/train/models ./fastvideo/tests/train/methods -vs"
     )
 
 

fastvideo/tests/train/fixtures/wan_causal_t2v_dfsft_min.yaml

@@ -0,0 +1,47 @@
+# Minimum config to run a single training step of
+# DiffusionForcingSFTMethod on WanCausalModel for the per-method
+# smoke test.  Uses the real Wan 2.1 1.3B checkpoint (loaded as a
+# CausalWanTransformer3DModel by WanCausalModel) with tiny synthetic
+# latents.
+
+models:
+  student:
+    _target_: fastvideo.train.models.wan.WanCausalModel
+    init_from: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
+    trainable: true
+
+method:
+  _target_: fastvideo.train.methods.fine_tuning.dfsft.DiffusionForcingSFTMethod
+  chunk_size: 3
+
+training:
+  dit_precision: bf16
+
+  distributed:
+    num_gpus: 1
+    sp_size: 1
+    tp_size: 1
+
+  data:
+    seed: 42
+    train_batch_size: 1
+    training_cfg_rate: 0.0
+    # Multiple of chunk_size=3 so the diffusion-forcing chunker has
+    # whole chunks to work with.
+    num_latent_t: 6
+    num_height: 64
+    num_width: 64
+    num_frames: 21
+
+  optimizer:
+    learning_rate: 2.0e-6
+    betas: [0.9, 0.999]
+    weight_decay: 0.01
+    lr_scheduler: constant
+    lr_warmup_steps: 0
+
+  loop:
+    max_train_steps: 1
+    gradient_accumulation_steps: 1
+
+pipeline: {}

fastvideo/tests/train/fixtures/wan_t2v_finetune_min.yaml

@@ -0,0 +1,47 @@
+# Minimum config to run a single training step of FineTuneMethod on
+# WanModel for the per-method smoke test.  Uses the real Wan 2.1
+# 1.3B checkpoint (loaded by WanModel.__init__) but with tiny synthetic
+# latents and a single-rank distributed setup so the test fits on one
+# L40S.
+
+models:
+  student:
+    _target_: fastvideo.train.models.wan.WanModel
+    init_from: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
+    trainable: true
+
+method:
+  _target_: fastvideo.train.methods.fine_tuning.finetune.FineTuneMethod
+
+training:
+  dit_precision: bf16
+
+  distributed:
+    num_gpus: 1
+    sp_size: 1
+    tp_size: 1
+
+  data:
+    seed: 42
+    train_batch_size: 1
+    training_cfg_rate: 0.0
+    num_latent_t: 4
+    num_height: 64
+    num_width: 64
+    num_frames: 13
+
+  optimizer:
+    learning_rate: 1.0e-6
+    betas: [0.9, 0.999]
+    weight_decay: 0.01
+    lr_scheduler: constant
+    lr_warmup_steps: 0
+
+  loop:
+    max_train_steps: 1
+    gradient_accumulation_steps: 1
+
+# Empty dict (not omitted) so _parse_pipeline_config resolves the
+# Wan-specific dit_config from the model checkpoint instead of
+# falling back to a bare DiTConfig.
+pipeline: {}

fastvideo/tests/train/methods/__init__.py

@@ -0,0 +1,108 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Per-method GPU smoke test: ``WanCausalModel`` + ``DiffusionForcingSFTMethod``.
+
+Mirrors ``test_wan_finetune.py`` for the diffusion-forcing SFT
+(DFSFT) algorithm on the causal Wan transformer.  The harness is
+intentionally identical so the two tests are easy to compare and so
+future per-method tests can copy this template verbatim.
+
+DFSFT samples *inhomogeneous* timesteps per chunk (``chunk_size=3``
+in the fixture) and is the natural training counterpart of the
+``WanCausalModel`` plugin.
+"""
+
+from __future__ import annotations
+
+import os
+
+os.environ.setdefault("MASTER_ADDR", "localhost")
+os.environ.setdefault("MASTER_PORT", "29517")
+
+from pathlib import Path
+
+import pytest
+import torch
+
+from fastvideo.train.methods.fine_tuning.dfsft import (
+    DiffusionForcingSFTMethod, )
+from fastvideo.train.models.wan import WanCausalModel
+from fastvideo.train.utils.config import load_run_config
+
+
+_FIXTURE = str(
+    Path(__file__).resolve().parent.parent / "fixtures"
+    / "wan_causal_t2v_dfsft_min.yaml")
+
+
+def _build_synthetic_batch(
+    device: torch.device,
+    dtype: torch.dtype,
+) -> dict[str, torch.Tensor]:
+    """Tiny synthetic ``raw_batch`` for the causal Wan path.
+
+    ``num_latent_t`` in the fixture is 6 (= 2 * chunk_size) so the
+    diffusion-forcing chunker has two whole chunks to operate on
+    even after ``prepare_batch`` truncates.
+    """
+    batch_size = 1
+    return {
+        "text_embedding":
+        torch.randn(batch_size, 16, 4096, device=device, dtype=dtype),
+        "text_attention_mask":
+        torch.ones(batch_size, 16, device=device, dtype=dtype),
+        "vae_latent":
+        torch.randn(batch_size, 16, 6, 8, 8, device=device, dtype=dtype),
+    }
+
+
+@pytest.mark.usefixtures("distributed_setup")
+def test_wan_causal_dfsft_single_train_step() -> None:
+    if not torch.cuda.is_available():
+        pytest.skip("requires CUDA")
+
+    cfg = load_run_config(_FIXTURE)
+
+    device = torch.device("cuda:0")
+    dtype = torch.bfloat16
+
+    model = WanCausalModel(
+        init_from=cfg.models["student"]["init_from"],
+        training_config=cfg.training,
+        trainable=True,
+    )
+    model.transformer = model.transformer.to(device=device, dtype=dtype)
+
+    method = DiffusionForcingSFTMethod(
+        cfg=cfg,
+        role_models={"student": model},
+    )
+    method.on_train_start()
+
+    batch = _build_synthetic_batch(device, dtype)
+    loss_map, outputs, _metrics = method.single_train_step(batch, iteration=0)
+
+    loss = loss_map["total_loss"]
+    assert torch.is_tensor(loss), "total_loss must be a torch.Tensor"
+    assert torch.isfinite(loss).item(), (
+        f"total_loss is not finite: {loss.item()}")
+
+    method.backward(loss_map, outputs, grad_accum_rounds=1)
+
+    blocks = getattr(model.transformer, "blocks", None)
+    assert blocks is not None and len(blocks) > 0, (
+        "CausalWanTransformer is expected to expose ``.blocks``")
+    layer0 = blocks[0]
+
+    trainable = [p for p in layer0.parameters() if p.requires_grad]
+    assert len(trainable) > 0, "layer 0 has no trainable parameters"
+
+    for i, p in enumerate(trainable):
+        assert p.grad is not None, f"layer 0 param[{i}] has None grad"
+        assert torch.isfinite(p.grad).all().item(), (
+            f"layer 0 param[{i}] grad contains NaN/Inf")
+
+    any_nonzero = any(
+        p.grad.detach().float().norm().item() > 0.0 for p in trainable)
+    assert any_nonzero, (
+        "all layer-0 grads are exactly zero; backward did not "
+        "reach the first transformer block")

fastvideo/tests/train/methods/test_wan_causal_dfsft.py

@@ -0,0 +1,125 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Per-method GPU smoke test: ``WanModel`` + ``FineTuneMethod``.
+
+Establishes the per-method test pattern for ``fastvideo/train``:
+
+1. Instantiate the model + method via their public constructors
+   (no ``Trainer`` setup, no FSDP wrapping).
+2. Feed a synthetic ``raw_batch`` dict through
+   ``method.single_train_step()`` + ``method.backward()``.
+3. Assert that the loss is finite and that the first transformer
+   block received a finite, non-zero gradient.
+
+The first block's gradient is the *last* one computed during
+backprop, so a healthy grad there implies the full
+forward + chain-rule path is intact.  Keeping the assertion to a
+single block keeps the reference surface tiny — a later PR layers a
+device-keyed grad-norm regression on top of this same harness.
+"""
+
+from __future__ import annotations
+
+import os
+
+# Required by the ``distributed_setup`` fixture pulled from
+# ``fastvideo/tests/conftest.py``.  Set before any fastvideo import.
+os.environ.setdefault("MASTER_ADDR", "localhost")
+os.environ.setdefault("MASTER_PORT", "29516")
+
+from pathlib import Path
+
+import pytest
+import torch
+
+from fastvideo.train.methods.fine_tuning.finetune import (
+    FineTuneMethod, )
+from fastvideo.train.models.wan import WanModel
+from fastvideo.train.utils.config import load_run_config
+
+
+_FIXTURE = str(
+    Path(__file__).resolve().parent.parent / "fixtures"
+    / "wan_t2v_finetune_min.yaml")
+
+
+def _build_synthetic_batch(
+    device: torch.device,
+    dtype: torch.dtype,
+) -> dict[str, torch.Tensor]:
+    """Tiny synthetic ``raw_batch`` matching ``WanModel.prepare_batch``.
+
+    Shapes are intentionally small so the 1.3B base model + one
+    backward pass fits comfortably on a single L40S.  Wan uses T5
+    text embeddings (hidden=4096) and a VAE with ``z_dim=16``.
+    """
+    batch_size = 1
+    return {
+        "text_embedding":
+        torch.randn(batch_size, 16, 4096, device=device, dtype=dtype),
+        "text_attention_mask":
+        torch.ones(batch_size, 16, device=device, dtype=dtype),
+        # vae_latent in (B, C, T, H, W); ``prepare_batch`` will
+        # truncate T to ``training.data.num_latent_t``.
+        "vae_latent":
+        torch.randn(batch_size, 16, 4, 8, 8, device=device, dtype=dtype),
+    }
+
+
+@pytest.mark.usefixtures("distributed_setup")
+def test_wan_finetune_single_train_step() -> None:
+    if not torch.cuda.is_available():
+        pytest.skip("requires CUDA")
+
+    cfg = load_run_config(_FIXTURE)
+
+    device = torch.device("cuda:0")
+    dtype = torch.bfloat16
+
+    model = WanModel(
+        init_from=cfg.models["student"]["init_from"],
+        training_config=cfg.training,
+        trainable=True,
+    )
+    # Move transformer to device + training dtype.  Real training
+    # wraps the transformer with FSDP and shards across ranks; for a
+    # single-step smoke we just move it directly.
+    model.transformer = model.transformer.to(device=device, dtype=dtype)
+
+    method = FineTuneMethod(
+        cfg=cfg,
+        role_models={"student": model},
+    )
+    # cuda_generator + RNG seeding (normally done by ``Trainer``).
+    method.on_train_start()
+
+    batch = _build_synthetic_batch(device, dtype)
+    loss_map, outputs, _metrics = method.single_train_step(batch, iteration=0)
+
+    loss = loss_map["total_loss"]
+    assert torch.is_tensor(loss), "total_loss must be a torch.Tensor"
+    assert torch.isfinite(loss).item(), (
+        f"total_loss is not finite: {loss.item()}")
+
+    method.backward(loss_map, outputs, grad_accum_rounds=1)
+
+    blocks = getattr(model.transformer, "blocks", None)
+    assert blocks is not None and len(blocks) > 0, (
+        "Wan transformer is expected to expose ``.blocks``")
+    layer0 = blocks[0]
+
+    trainable = [p for p in layer0.parameters() if p.requires_grad]
+    assert len(trainable) > 0, "layer 0 has no trainable parameters"
+
+    for i, p in enumerate(trainable):
+        assert p.grad is not None, f"layer 0 param[{i}] has None grad"
+        assert torch.isfinite(p.grad).all().item(), (
+            f"layer 0 param[{i}] grad contains NaN/Inf")
+
+    # At least one layer-0 grad must have non-zero L2 norm so we
+    # catch the case where backward ran but the first block was
+    # detached from the loss (silent connectivity bug).
+    any_nonzero = any(
+        p.grad.detach().float().norm().item() > 0.0 for p in trainable)
+    assert any_nonzero, (
+        "all layer-0 grads are exactly zero; backward did not "
+        "reach the first transformer block")