[feat] Add Cosmos Predict2.5 distilled profile to DreamVerse

[Mister-Raggs]

Purpose

Add an experimental DreamVerse runtime profile for the Cosmos Predict2.5 2B distilled T2W path introduced in #1767. This lets DreamVerse generate and browser-stream independent Cosmos clips while preserving the existing LTX2 defaults.

This PR is intentionally stacked on #1767 and should be reviewed after that core model PR. Until #1767 merges, GitHub will show its core commits in this diff as well; the DreamVerse-specific change is commit 056de6eb6.

Changes

• Add a cosmos25-distilled model profile with the validated BF16 + Torch SDPA settings: 704x1280, 77 frames, 16 FPS, four steps.
• Make generation parameters, attention backend, quantization, compile, refine, LoRA, audio, and continuation behavior model-aware while retaining the existing LTX2 values as defaults.
• Return decoded Cosmos frames directly to DreamVerse and mux a duration-matched silent 24 kHz track for the existing browser AV contract.
• Pass the active model FPS into fMP4 encoding and overlap calculations instead of assuming 24 FPS.
• Reject unsupported image input and LoRA stacks for the distilled Cosmos profile.
• Use a single independent segment for Cosmos startup warmup.
• Document the experimental startup configuration and limitations.

Test plan and results

Media

Segment 1 — initial robotic-arm prompt

cosmos25-dreamverse-2.mp4

Segment 2 — independently generated follow-up

cosmos25-dreamverse-1.mp4

Local focused tests:

• test_config.py, test_gpu_pool.py, and test_video_generation_cosmos25.py: 21 passed.
• Python compilation for the changed runtime modules: passed.
• YAPF, Ruff, codespell, PyMarkdown, and filename hooks: passed.

End-to-end on Spark / GB10:

• DreamVerse loaded the locally converted Cosmos package with BF16 + Torch SDPA.
• A 704x1280x77, four-step segment generated in about 136.5 seconds after model load.
• fMP4 encoding completed in 286 ms and streamed 31 chunks (about 2.9-3.1 MB).
• Main-process overhead was about 350 ms.
• The video played successfully in the browser and showed expected motion with the validated robotic-arm prompt.
• A second prompted segment also generated and played successfully as an independent clip.

Scope and limitations

• This is one-shot T2W integration, not rolling Cosmos generation. Each segment is independent.
• Cosmos does not generate audio here; the silent track exists only to satisfy DreamVerse's current AV streaming contract.
• This is not real-time on GB10: a full segment takes about 136 seconds to generate.
• A local converted package is required until [new-model] Add Cosmos Predict2.5 2B distilled T2W inference #1767 publishes/registers a model ID; set DREAMVERSE_MODEL_PATH to that directory.
• FFmpeg must be available through PATH or FASTVIDEO_FFMPEG_BIN.
• The existing frontend still has an LTX-specific model badge and LTX-oriented presets. Model-aware UI metadata and Cosmos-specific presets are follow-up work; the backend logs and selected runtime profile are authoritative for this PR.
• DreamVerse's existing prompt-enhancer startup requirements are unchanged.

Checklist

• Existing LTX2 generation defaults have regression coverage.
• Cosmos profile selection and unsupported-input behavior have focused tests.
• Browser AV streaming was validated end to end on Spark.
• Relevant pre-commit hooks pass.
• Merge/rebase after [new-model] Add Cosmos Predict2.5 2B distilled T2W inference #1767.
• Replace the provisional model path after a public converted package is available.

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

	Protection	Waiting on
🔴	PR merge requirements	👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed
• check-success~=pre-commit

This rule is failing.

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.