[feat]: Stable-Video-Infinity inference on Wan 2.1 I2V 14B 480P

[H1yori233]

Purpose

• Adds an inference-only port of Stable-Video-Infinity (SVI) — a rank-128 LoRA on top of Wan 2.1 I2V 14B 480P that generates arbitrarily long videos clip-by-clip via motion-frame chaining.
• Selected at runtime via override_pipeline_cls_name="WanSVIImageToVideoPipeline" on top of the existing Wan-AI/Wan2.1-I2V-14B-480P-Diffusers model path. No new model path / no Tier-3 registry switch.
• Three SVI LoRA variants supported out of the box (Shot / Film / Tom) via a single example with a MODEL_VARIANT switch.

Changes

• fastvideo/pipelines/basic/wan/wan_svi_i2v_pipeline.py, add WanSVIImageToVideoPipeline.
• fastvideo/pipelines/stages/image_encoding.py, add SVIImageVAEEncodingStage.
• fastvideo/tests/stages/test_svi_image_vae_encoding.py, test script.
• examples/inference/basic/basic_svi_i2v.py
• fastvideo/utils.py — maybe_download_lora accepts org/repo/subdir/file.safetensors so LoRA HF repos with multiple variant files (like vita-video-gen/svi-model) can pick a specific one without an explicit hf_hub_download call in user code.
• fastvideo/pipelines/pipeline_batch_info.py
• fastvideo/api/sampling_param.py

Test Plan

Test Results

Stage unit test:

$ pytest fastvideo/tests/stages/test_svi_image_vae_encoding.py -q
4 passed in 0.26s

End-to-end inference, all 3 variants (Wan-AI/Wan2.1-I2V-14B-480P-Diffusers + SVI LoRA, 81 frames at 448×832, 20 denoise steps, 1× GB200).

A.sleek.white.motor.yacht.speeds.across.the.turquoise.blue.sea.leaving.a.dramatic.wake.of.white.foa.mp4

A.Siamese.kitten.rests.snugly.inside.a.straw.hat.its.head.slightly.tilted.as.it.gazes.curiously.to.mp4

Checklist

• I ran pre-commit run --all-files and fixed all issues
• I added or updated tests for my changes
• I updated documentation if needed
• I considered GPU memory impact of my changes

For model/pipeline changes, also check:

• I verified SSIM regression tests pass
• I updated the support matrix if adding a new model

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

	Protection	Waiting on
🔴	PR merge requirements	👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=full-suite-passed
• check-success=fastcheck-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[gemini-code-assist]

Code Review

This pull request introduces support for Stable-Video-Infinity (SVI) multi-clip image-to-video generation for the Wan 2.1 model, including a new example script and inference presets. Key changes include the implementation of the WanSVIImageToVideoPipeline, a specialized SVIImageVAEEncodingStage for motion-frame conditioning, and updates to the LoRA loading logic to support individual file downloads from the Hugging Face Hub. The review feedback recommends implementing file locking for downloads to prevent race conditions, improving visual continuity in multi-clip sequences by using the last frame of a clip as the next reference, and caching embeddings to optimize performance during long video generation.

[SolitaryThinker]

Hi @H1yori233 — automated review from Gob, one of @SolitaryThinker's AI reviewers. Findings aren't all human-verified; ping @SolitaryThinker if anything looks off.

Reviewed against the add-model-10-pr-review rubric on 815da1fe2bd58d5ab87561f5429051adf83ebebd. Overall this looks clean — preset registered, stage exported, stage-level parity test is genuinely tight (byte-exact mask + ~2e-7 abs tol on the video tensor vs a diffsynth reference). The pipeline subclassing is right and override_pipeline_cls_name is the idiomatic dispatch for a variant on existing Wan 2.1 weights, so the absence of a top-level registry entry is fine. Verdict: ship-with-fixes.

Three S2 findings, all on shared-path contract widening:

---

S2-1 — lora_pipeline.py key normalizers fire unconditionally on every LoRA, not just SVI's

fastvideo/pipelines/lora_pipeline.py:319-321

 name = name.replace("diffusion_model.", "")
+name = name.replace("pipe.dit.", "")
 name = name.replace(".weight", "")
+name = name.replace(".default", "")

lora_state_dict is only normalized in this one function, so this is the single source of truth for every LoRA adapter in the repo. pipe.dit. is SVI/diffsynth-specific and unlikely to collide. .default is more generic — it's actually the correct normalization for PEFT/diffusers LoRA keys (<layer>.lora_A.default.weight), but the contract is now silently widened for all callers.

Suggested fix — guard the new replacements so non-SVI behavior is provably unchanged:

if "pipe.dit." in name:
    name = name.replace("pipe.dit.", "")
if ".default" in name:
    name = name.replace(".default", "")

Or just confirm one existing in-tree Wan 2.1 / Wan 2.2 LoRA adapter still loads after this change. Either is sufficient evidence.

---

S2-2 — maybe_download_lora triple-slash-path branch has no test

fastvideo/utils.py:595-604

The new branch handling org/repo/<sub>/<file>.safetensors is cleanly motivated by the SVI HF repo layout (vita-video-gen/svi-model/version-1.0/svi-shot.safetensors) and the guards look fine — os.path.isfile above short-circuits any real local path. But there's no unit test for the new branch or for the fall-through behavior.

One monkeypatch-based test asserting hf_hub_download is called with repo_id="vita-video-gen/svi-model" and filename="version-1.0/svi-shot.safetensors" would lock the contract. Not blocking.

---

S2-3 — Multi-clip stitching path is untested + missing a num_motion < num_frames guard

fastvideo/pipelines/basic/wan/wan_svi_i2v_pipeline.py:144-148

concatenated = [clip_outputs[0]]
for video in clip_outputs[1:]:
    concatenated.append(video[:, :, num_motion:, :, :])
batch.output = torch.cat(concatenated, dim=2)

The shipped example uses svi_num_clips=1, so the entire multi-clip outer loop in forward() is unexercised at runtime. The unit test only covers SVIImageVAEEncodingStage in isolation, not this stitching path.

There's also no guard that num_motion < num_frames. If a caller misconfigures it, video[:, :, num_motion:, :, :] becomes an empty slice and torch.cat silently produces a video with content missing from the affected clips.

Suggested fix:

1. Add if num_motion >= num_frames: raise ValueError(...) in the multi-clip branch before the loop.
2. Add a multi-clip smoke test (even a tiny one with a mocked transformer + 2 clips) to lock the index math.

---

S3 items (non-blocking): no docstring on SVIImageVAEEncodingStage explaining the y = concat(mask, latent) channel layout, assert vs verify_* for shape checks, no docs entry. Full review (including S3 items) is archived locally.

— Gob (@SolitaryThinker's AI reviewer). Full review (including S3 items) is archived locally.

[H1yori233]

/merge

[H1yori233]

/test vsa

[H1yori233]

/test unit

[SolitaryThinker]

Full review at head (six lanes, findings adversarially verified). This is a clean way to ship SVI: override_pipeline_cls_name instead of a registry switch is right, and the stage test that pins the mask/pre-VAE math to the reference is exactly what we want — I checked it line-for-line against encode_images_adaptive in the SVI repo; it matches.

Before merge:

1. CI red at head is partly yours: add the four svi_* fields to docs/design/inference_schema_parity_inventory.yaml. The ltx2 preset failure is just a stale head — rebase onto main (conflict-free, I checked; 106 commits behind).
2. Every clip reuses the identical seed/noise; the reference varies seed per chunk. Offset per clip or justify the deviation.
3. No evidence the headline feature works: the example ships svi_num_clips=1 and all PR videos are single-clip. Attach one multi-clip run (any variant), and either add SSIM coverage or state the deferral explicitly.
4. Per-clip prompts are unreachable: generate_video rejects list prompts and generate() fans them into separate videos — wire a real per-clip prompt input or drop the dead branch and the film multi-prompt claim.

Minor: the three new test files aren't collected by any CI lane (run_unit_test doesn't include tests/stages/tests/pipelines); the .default strip in set_lora_adapter runs on every adapter in both stacks — anchor it to the PEFT .lora_A.default/.lora_B.default pattern; batch.sigmas is force-overwritten and the scheduler hardcodes shift=5.0, so flow_shift silently does nothing (schedule itself matches the reference — knob honesty only).

Two-stack note: the only shared-surface hunk is lora_pipeline.py key canonicalization, inherited by the legacy training stack — the PEFT-anchoring fix above closes that exposure.

[H1yori233]

Thanks for the review @SolitaryThinker. I've addressed the requested changes:

• Added all SVI fields to the schema inventory and synced the branch with main.
• Added per-clip seed offsets matching the reference (0, 42, 84, ...).
• Wired svi_clip_prompts through the public API; Shot repeats one prompt, while Film/Tom support prompt streams.
• Added multi-clip stitching, motion-window, prompt, seed, and conditioning tests.
• Anchored .default normalization to PEFT LoRA keys.
• Made flow_shift and caller-provided sigmas effective.

Multi-clip run:

https://github.com/user-attachments/assets/0ffefc2d-2dff-4bd0-93cb-db86d44dce8d
https://github.com/user-attachments/assets/88a38103-6dea-46cc-a5d1-13c207532810
https://github.com/user-attachments/assets/aabd2ea5-727a-43ff-91f2-42ee4806c67d

[H1yori233]

/test vsa

[mergify]

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease

[mergify]

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

	Protection	Waiting on
🔴	PR merge requirements	👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=full-suite-passed
• check-success=fastcheck-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]