[feat]: add Lucy Edit inference scaffold

[aryan5v]

Context

This adds initial Lucy Edit support as part of the FastVideo roadmap item tracked in #899.

I also looked at the earlier abandoned/draft Lucy Edit PR #1096 for reference while adapting the work to current main, but this PR keeps the implementation scoped to the current preset-based API and Wan pipeline structure.

Summary

• Add initial inference support for decart-ai/Lucy-Edit-Dev and decart-ai/Lucy-Edit-1.1-Dev
• Register Lucy Edit with the current preset-based API defaults
• Add a Wan-based LucyEditPipeline that uses input video latents without CLIP image encoder components
• Handle Lucy Edit denoising as [noise, video] channels instead of standard Wan V2V [noise, video, zeros]
• Use deterministic VAE latent retrieval for Lucy input video conditioning
• Add a basic inference example, support matrix entry, and focused API coverage

Review follow-up

• Added workload_type="t2v" to the Lucy Edit preset.
• Expanded API coverage for decart-ai/Lucy-Edit-1.1-Dev config and sampling-param resolution.
• Added config-time mutual-exclusion assertions for Wan TI2V vs Lucy Edit task flags.
• Verified both Lucy Edit HF model_index.json files do not declare image_encoder or image_processor; no loader code change needed for that item.

Validation

• API tests pass: fastvideo/tests/api/ -> 187 passed
• Ran video edit inference with FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA
• Full validation run: 81 frames, 50 steps, 480x832
• Output generated successfully and uploaded to W&B
• Conflict-resolution follow-up: python3 -m py_compile passed for touched Python files; git diff --check upstream/main passed. Local pytest could not run on macOS because the dependency solve requires triton==3.7.0, which has no wheel/source for this platform.

W&B

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[gemini-code-assist]

Code Review

This pull request introduces support for the Lucy Edit Dev 5B video editing model, adding a dedicated pipeline, configuration, and inference preset along with an example script and unit tests. The denoising logic was updated to accommodate the model's specific latent concatenation and timestep expansion requirements. Feedback from the review highlights a potential shape mismatch due to the in_channels configuration and suggests improving performance by hoisting constant calculations out of the denoising loop.

[aryan5v]

• API tests pass: fastvideo/tests/api/ -> 187 passed
• Ran video edit inference with FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA
• Full validation run: 81 frames, 50 steps, 480x832
• Output generated successfully and uploaded to W&B

W&B

[SolitaryThinker]

Hi @aryan5v — late surfacing of a review from one of @SolitaryThinker's AI reviewer agents (Gob) that we ran on 2026-05-18 but never got around to posting. I'm an AI reviewer, so @SolitaryThinker hasn't personally verified every finding — please ping @SolitaryThinker if anything below doesn't match what you know about the code.

TL;DR

Clean Wan2.2-5B-based scaffold for the Decart Lucy Edit family (V2V, no CLIP). LucyEditPipeline is a small, well-factored specialization of WanVideoToVideoPipeline. Architecture sound, test exercises the registry surface, no blockers. Four [S2] polish items worth addressing before this is rolled into the support matrix as ✅ (currently honestly ⭕ "not-yet-validated").

Verdict: ship-with-fixes

• S0 (blockers): 0
• S1 (must-fix): 0
• S2 (should-fix): 4
• S3 (discussion): 5 — not surfaced here; available locally on request

---

Findings

[S2-1] LUCY_EDIT_DEV preset is missing workload_type

What: fastvideo/pipelines/basic/wan/presets.py (added preset block ~lines 287-302) declares the new LUCY_EDIT_DEV preset without a workload_type field. Every peer Wan preset in the same file declares one (WAN_2_2_TI2V_5B at :240, FAST_WAN_2_2_TI2V_5B at :258 — both workload_type="t2v").

Why it matters: Depending on how InferencePreset.workload_type defaults, this is either a silent fallback that mis-routes UI workload selection, or a future KeyError when the schema validator runs. The preset is the registry's UI/dispatch entry; inconsistency with peer Wan presets is a foot-gun for the OpenAI-compatible serving layer and the CLI.

Suggested fix: Add workload_type="t2v" (matching the Wan 2.2 TI2V 5B precedent — Lucy is V2V by intent but FastVideo doesn't have a V2V workload enum yet, which is the same gap noted at test_lucy_edit.py:41).

Evidence: fastvideo/pipelines/basic/wan/presets.py:287-302, peers at :240 and :258.

---

[S2-2] Lucy-Edit-1.1-Dev registered but not covered by the test

What: fastvideo/registry.py:408-411 registers both decart-ai/Lucy-Edit-Dev and decart-ai/Lucy-Edit-1.1-Dev, but fastvideo/tests/api/test_lucy_edit.py:25-26 only asserts pipeline-config resolution + sampling-param resolution for Lucy-Edit-Dev. The 1.1-Dev variant only gets the default-preset assertion on line 26.

Why it matters: The two variants resolve through model_detectors=[lambda path: "lucy-edit" in path.lower()] — a substring match. The test doesn't lock that behavior in for 1.1-Dev, so a future rename or detector change could regress one variant but not the other silently.

Suggested fix: Mirror the Lucy-Edit-Dev assertions for Lucy-Edit-1.1-Dev — get_pipeline_config_cls_from_name("decart-ai/Lucy-Edit-1.1-Dev") + SamplingParam.from_pretrained("decart-ai/Lucy-Edit-1.1-Dev"). One extra block in the existing test, much higher confidence.

Evidence: fastvideo/registry.py:408-411, fastvideo/tests/api/test_lucy_edit.py:25-26.

---

[S2-3] lucy_edit_task: bool adds a third pipeline-mode boolean that should be an enum

What: fastvideo/configs/pipelines/base.py:108-112 adds lucy_edit_task: bool next to the existing ti2v_task: bool. The comment update ("Wan2.2 TI2V parameters" → "Wan2.2 task modifiers") is implicit acknowledgement that these are a family of mutually-exclusive flags. fastvideo/pipelines/stages/denoising.py:273-296 branches on is_lucy_edit first and falls through to ti2v_task in an elif — mutual exclusion enforced by reading order, not by the type system.

Why it matters: Every additional Wan-family special case (control variants, audio-conditioned variants, etc.) will keep adding *_task: bool flags, and the denoising stage will accumulate if/elif ladders. This is the "branch-on-pipeline-identity via boolean flags" anti-pattern from the add-model-10 rubric. The PR is consistent with the existing local convention (ti2v_task already does this), so it's not a regression, but it's compounding existing tech debt.

Suggested fix (medium effort, optional for this PR): Convert ti2v_task + lucy_edit_task into a single wan_task_mode: Literal["t2v","ti2v","lucy_edit"] | None enum (default None = standard V2V/T2V), update the denoising-stage branches to switch on the enum. Out-of-scope for a scaffold PR; could be a follow-up issue.

Suggested fix (low effort, acceptable for this PR): Leave the boolean pile as-is, but add a one-line assert not (self.ti2v_task and self.lucy_edit_task) invariant inside LucyEditDevConfig.__post_init__ and Wan2_2_TI2V_5B_Config.__post_init__ so the mutual exclusion is checked at config-build time rather than via reading-order luck in the denoising loop.

Evidence: fastvideo/configs/pipelines/base.py:108-112, fastvideo/pipelines/stages/denoising.py:273-296.

---

[S2-4] LucyEditPipeline._required_config_modules drops image_encoder/image_processor — verify the loader doesn't try to instantiate them

What: fastvideo/pipelines/basic/wan/lucy_edit_pipeline.py:27-33 (new) declares a 5-module required list (text_encoder, tokenizer, vae, transformer, scheduler) vs the parent WanVideoToVideoPipeline._required_config_modules at wan_v2v_pipeline.py:26-29 which includes 7 (adds image_encoder, image_processor). The override is correct in intent — Lucy has no CLIP image encoder. But the diff does not touch image_encoding.py's VideoVAEEncodingStage selection logic, and image_encoding.py:632-637 only changes the sample_mode flag, not the gating.

Why it matters: If the loader treats _required_config_modules as the source of truth and the HF repo decart-ai/Lucy-Edit-Dev has no image_encoder key in model_index.json, this is correct as-is. If the loader iterates over the HF model_index.json instead, a stray module in the HF repo will be loaded into a pipeline that doesn't use it (wasted memory + slower cold start).

Suggested fix: No code change needed if (1) holds. Manually confirm via huggingface_hub.HfApi().model_info("decart-ai/Lucy-Edit-Dev") that no image_encoder / image_processor keys exist in the HF model index, and add the verified result to the PR description so it's locked in for reviewers. If the HF repo DOES include these modules, the loader-skipping behavior needs an explicit test.

Evidence: fastvideo/pipelines/basic/wan/lucy_edit_pipeline.py:27-33, fastvideo/pipelines/basic/wan/wan_v2v_pipeline.py:26-29, fastvideo/pipelines/stages/image_encoding.py:632-637.

---

Review from @SolitaryThinker's agent Gob (an AI reviewer). Late surfacing — review was run 2026-05-18 against the current HEAD 964892ef; PR hasn't moved since, so the findings are still valid against the current state. Ping @SolitaryThinker if any finding is off, or if the loader behavior in S2-4 needs follow-up verification. Full S3 discussion items (5 of them: example VRAM tuning, missing SSIM regression, expand_timesteps double-counting risk, latents_mean/std provenance, create_pipeline_stages could compose via super() instead of fork) are in the local archive on request.

[aryan5v]

Hi @aryan5v — late surfacing of a review from one of @SolitaryThinker's AI reviewer agents (Gob) that we ran on 2026-05-18 but never got around to posting. I'm an AI reviewer, so @SolitaryThinker hasn't personally verified every finding — please ping @SolitaryThinker if anything below doesn't match what you know about the code.

TL;DR

Clean Wan2.2-5B-based scaffold for the Decart Lucy Edit family (V2V, no CLIP). LucyEditPipeline is a small, well-factored specialization of WanVideoToVideoPipeline. Architecture sound, test exercises the registry surface, no blockers. Four [S2] polish items worth addressing before this is rolled into the support matrix as ✅ (currently honestly ⭕ "not-yet-validated").

Verdict: ship-with-fixes

• S0 (blockers): 0
• S1 (must-fix): 0
• S2 (should-fix): 4
• S3 (discussion): 5 — not surfaced here; available locally on request

Findings

[S2-1] LUCY_EDIT_DEV preset is missing workload_type

What: fastvideo/pipelines/basic/wan/presets.py (added preset block ~lines 287-302) declares the new LUCY_EDIT_DEV preset without a workload_type field. Every peer Wan preset in the same file declares one (WAN_2_2_TI2V_5B at :240, FAST_WAN_2_2_TI2V_5B at :258 — both workload_type="t2v").

Why it matters: Depending on how InferencePreset.workload_type defaults, this is either a silent fallback that mis-routes UI workload selection, or a future KeyError when the schema validator runs. The preset is the registry's UI/dispatch entry; inconsistency with peer Wan presets is a foot-gun for the OpenAI-compatible serving layer and the CLI.

Suggested fix: Add workload_type="t2v" (matching the Wan 2.2 TI2V 5B precedent — Lucy is V2V by intent but FastVideo doesn't have a V2V workload enum yet, which is the same gap noted at test_lucy_edit.py:41).

Evidence: fastvideo/pipelines/basic/wan/presets.py:287-302, peers at :240 and :258.

[S2-2] Lucy-Edit-1.1-Dev registered but not covered by the test

What: fastvideo/registry.py:408-411 registers both decart-ai/Lucy-Edit-Dev and decart-ai/Lucy-Edit-1.1-Dev, but fastvideo/tests/api/test_lucy_edit.py:25-26 only asserts pipeline-config resolution + sampling-param resolution for Lucy-Edit-Dev. The 1.1-Dev variant only gets the default-preset assertion on line 26.

Why it matters: The two variants resolve through model_detectors=[lambda path: "lucy-edit" in path.lower()] — a substring match. The test doesn't lock that behavior in for 1.1-Dev, so a future rename or detector change could regress one variant but not the other silently.

Suggested fix: Mirror the Lucy-Edit-Dev assertions for Lucy-Edit-1.1-Dev — get_pipeline_config_cls_from_name("decart-ai/Lucy-Edit-1.1-Dev") + SamplingParam.from_pretrained("decart-ai/Lucy-Edit-1.1-Dev"). One extra block in the existing test, much higher confidence.

Evidence: fastvideo/registry.py:408-411, fastvideo/tests/api/test_lucy_edit.py:25-26.

[S2-3] lucy_edit_task: bool adds a third pipeline-mode boolean that should be an enum

What: fastvideo/configs/pipelines/base.py:108-112 adds lucy_edit_task: bool next to the existing ti2v_task: bool. The comment update ("Wan2.2 TI2V parameters" → "Wan2.2 task modifiers") is implicit acknowledgement that these are a family of mutually-exclusive flags. fastvideo/pipelines/stages/denoising.py:273-296 branches on is_lucy_edit first and falls through to ti2v_task in an elif — mutual exclusion enforced by reading order, not by the type system.

Why it matters: Every additional Wan-family special case (control variants, audio-conditioned variants, etc.) will keep adding *_task: bool flags, and the denoising stage will accumulate if/elif ladders. This is the "branch-on-pipeline-identity via boolean flags" anti-pattern from the add-model-10 rubric. The PR is consistent with the existing local convention (ti2v_task already does this), so it's not a regression, but it's compounding existing tech debt.

Suggested fix (medium effort, optional for this PR): Convert ti2v_task + lucy_edit_task into a single wan_task_mode: Literal["t2v","ti2v","lucy_edit"] | None enum (default None = standard V2V/T2V), update the denoising-stage branches to switch on the enum. Out-of-scope for a scaffold PR; could be a follow-up issue.

Suggested fix (low effort, acceptable for this PR): Leave the boolean pile as-is, but add a one-line assert not (self.ti2v_task and self.lucy_edit_task) invariant inside LucyEditDevConfig.__post_init__ and Wan2_2_TI2V_5B_Config.__post_init__ so the mutual exclusion is checked at config-build time rather than via reading-order luck in the denoising loop.

Evidence: fastvideo/configs/pipelines/base.py:108-112, fastvideo/pipelines/stages/denoising.py:273-296.

[S2-4] LucyEditPipeline._required_config_modules drops image_encoder/image_processor — verify the loader doesn't try to instantiate them

What: fastvideo/pipelines/basic/wan/lucy_edit_pipeline.py:27-33 (new) declares a 5-module required list (text_encoder, tokenizer, vae, transformer, scheduler) vs the parent WanVideoToVideoPipeline._required_config_modules at wan_v2v_pipeline.py:26-29 which includes 7 (adds image_encoder, image_processor). The override is correct in intent — Lucy has no CLIP image encoder. But the diff does not touch image_encoding.py's VideoVAEEncodingStage selection logic, and image_encoding.py:632-637 only changes the sample_mode flag, not the gating.

Why it matters: If the loader treats _required_config_modules as the source of truth and the HF repo decart-ai/Lucy-Edit-Dev has no image_encoder key in model_index.json, this is correct as-is. If the loader iterates over the HF model_index.json instead, a stray module in the HF repo will be loaded into a pipeline that doesn't use it (wasted memory + slower cold start).

Suggested fix: No code change needed if (1) holds. Manually confirm via huggingface_hub.HfApi().model_info("decart-ai/Lucy-Edit-Dev") that no image_encoder / image_processor keys exist in the HF model index, and add the verified result to the PR description so it's locked in for reviewers. If the HF repo DOES include these modules, the loader-skipping behavior needs an explicit test.

Evidence: fastvideo/pipelines/basic/wan/lucy_edit_pipeline.py:27-33, fastvideo/pipelines/basic/wan/wan_v2v_pipeline.py:26-29, fastvideo/pipelines/stages/image_encoding.py:632-637.

Review from @SolitaryThinker's agent Gob (an AI reviewer). Late surfacing — review was run 2026-05-18 against the current HEAD 964892ef; PR hasn't moved since, so the findings are still valid against the current state. Ping @SolitaryThinker if any finding is off, or if the loader behavior in S2-4 needs follow-up verification. Full S3 discussion items (5 of them: example VRAM tuning, missing SSIM regression, expand_timesteps double-counting risk, latents_mean/std provenance, create_pipeline_stages could compose via super() instead of fork) are in the local archive on request.

Thanks for the review. I pushed a follow-up that addresses the S2s:

• added workload_type="t2v" to the Lucy preset
• expanded coverage for Lucy-Edit-1.1-Dev
• added config-time mutual exclusion checks for ti2v_task / lucy_edit_task
• verified the Lucy HF model indexes don’t include image_encoder or image_processor, so no loader change was needed there

Also merged latest main; PR should be mergeable again.

[mergify]

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease

[SolitaryThinker]

/merge

[aryan5v]

/test performance

[mergify]

Pre-commit checks failed

Hi @aryan5v, the pre-commit checks have failed. To fix them locally:

# Install pre-commit if you haven't already
uv pip install pre-commit
pre-commit install

# Run all checks and auto-fix what's possible
pre-commit run --all-files

Common fixes:

• yapf: yapf -i <file> (formatting)
• ruff: ruff check --fix <file> (linting)
• codespell: codespell --write-changes <file> (spelling)

After fixing, commit and push the changes. The checks will re-run automatically.

For future commits, pre-commit will run automatically on changed files before each commit.