[bugfix] Fix Dreamverse Modal compile warmup latency

[Davids048]

Summary

• Pre-stabilize the LTX2 Gemma text encoder before torch.compile by running a one-token eager forward with output_hidden_states=True, avoiding a second text-encoder compile caused by Transformers' hidden-state wrapper side effect.
• Enable VAE torch.compile in the Dreamverse LTX2 generator config so Modal/Docker deployments match the local fast-profile path and avoid slow eager VAE decode.
• Deploy Modal with the selected DREAMVERSE_IMAGE, startup warmup, and torch compile enabled so registry image deploys exercise the optimized path by default.

Validation

• python -m py_compile apps/dreamverse/dreamverse/video_generation.py apps/dreamverse/scripts/modal/modal_app.py fastvideo/models/encoders/gemma.py
• pre-commit hooks on commit: yapf, ruff, codespell, mypy, filename checks
• Built and pushed test image: ghcr.io/davids048/dreamverse-ui:cuda12.9.1-sha-321d5112b-warmup-vaecompile
• Modal test deploy reached ready with 1 GPU worker and one active runtime container.
• One-client websocket streaming smoke passed; post-warmup segments were ~3.7-4.3s generation time instead of ~5.4s.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[gemini-code-assist]

Code Review

This pull request enables VAE compilation, activates torch compilation and startup warmup in the Modal app configuration, and updates the Gemma text encoder compilation preparation. The text encoder update runs a small eager forward pass to stabilize the layer state and prevent recompilation. Feedback is provided regarding the extraction of pad_token_id in gemma.py, where a list or tuple value could lead to a TypeError when converted to an integer.

[gemini-code-assist]

If pad_token_id in the model configuration is a list or tuple (which is common in some tokenizers/models), getattr(model.config, "pad_token_id", 0) will return a list/tuple. Passing this directly to int(token_id or 0) will raise a TypeError. We should apply the same list/tuple extraction logic to pad_token_id to ensure robustness.

Suggested change

	token_id = getattr(model.config, "eos_token_id", None)
	if isinstance(token_id, (list, tuple)):
	token_id = token_id[0] if token_id else None
	if token_id is None:
	token_id = getattr(model.config, "pad_token_id", 0)
	input_ids = torch.full(
	(1, 1),
	int(token_id or 0),
	token_id = getattr(model.config, "eos_token_id", None)
	if isinstance(token_id, (list, tuple)):
	token_id = token_id[0] if token_id else None
	if token_id is None:
	token_id = getattr(model.config, "pad_token_id", None)
	if isinstance(token_id, (list, tuple)):
	token_id = token_id[0] if token_id else None
	input_ids = torch.full(
	(1, 1),
	int(token_id if token_id is not None else 0),

[SolitaryThinker]

Hi @Davids048 — automated review from Gob, one of @SolitaryThinker's AI reviewers. Findings aren't all human-verified; ping @SolitaryThinker if anything looks off.

TL;DR

Clean, well-scoped bugfix with a great root-cause comment in gemma.py. The output_hidden_states=True warmup is deliberate and not a state leak (the layer-wrapping it triggers is exactly the state we want stable before Dynamo captures guards). Two small S2 polish items: gemini-bot's pad_token_id fallback hardening, and a DREAMVERSE_MAX_AUTOTUNE Python-vs-Modal default asymmetry worth resolving.

Verdict: approve-with-followup

• S0: 0 S1: 0 S2: 2 (2 surfaced) S3: not shown; see review.md

Findings

S2-1 — pad_token_id fallback can TypeError on list/tuple

File: fastvideo/models/encoders/gemma.py:380

The new warmup defends eos_token_id against list/tuple shapes (good), but the pad_token_id fallback does not:

if token_id is None:
    token_id = getattr(model.config, "pad_token_id", 0)

If a Gemma checkpoint ever ships pad_token_id as a list/tuple (some HF tokenizer configs do), int(token_id or 0) two lines later raises TypeError at startup-warmup time. Gemma3-1B/4B/12B/27B all ship scalar eos_token_id so the fallback is unlikely to hit in practice, but the fix is a one-line defensive isinstance check and removes a latent footgun.

This is the same concern gemini-bot raised on gemma.py:380 — accepting their suggestion: block resolves it.

S2-2 — DREAMVERSE_MAX_AUTOTUNE Python default contradicts the documented Modal default

Files: apps/dreamverse/dreamverse/config.py:167, apps/dreamverse/scripts/modal/modal_app.py:25, apps/dreamverse/scripts/modal/README.md

• config.py:167: _env_bool("DREAMVERSE_MAX_AUTOTUNE", True) → Python default True.
• modal_app.py:25: os.environ.get("DREAMVERSE_MAX_AUTOTUNE", "0") → Modal default "0".
• README: "The Modal wrapper we provide defaults to torch compile without Inductor max-autotune."

The README is accurate for the Modal path. But anyone instantiating dreamverse/config.py outside the Modal wrapper (local dev, another deployment layer, tests) silently gets max-autotune-no-cudagraphs, which is the high-cost path this PR is trying to make opt-in. Recommend flipping config.py:167 to False so the Python-level default matches the Modal-level default and the README. The Modal wrapper still sets "0" explicitly, so its behavior is unchanged.

Doc-vs-code verification (passed)

README claim	Backed by code?
DREAMVERSE_MAX_AUTOTUNE env var exists	yes — config.py:167
Modal wrapper defaults to compile without max-autotune	yes — modal_app.py:25 (S2-2 caveat for non-Modal callers)
DREAMVERSE_MAX_AUTOTUNE=1 modal deploy … opts in	yes — modal_app.py forwards from deployer shell
min_containers=1 + max_containers=1 keeps one B200 warm	yes — modal_app.py:42-43
VAE compile is now enabled	yes — video_generation.py:285 vae_enabled=enable_compile

— Gob (@SolitaryThinker's AI reviewer). Full review archived locally.

[SolitaryThinker]

Hi @Davids048 — automated review from Gob, one of @SolitaryThinker's AI reviewers. Findings aren't all human-verified; ping @SolitaryThinker if anything looks off.

TL;DR

The address commit fixes the DREAMVERSE_MAX_AUTOTUNE default asymmetry by moving the Modal wrapper default up to enabled, matching the Python config default. I did not see any new regression in the warmup/compile/container wiring, but the prior pad_token_id fallback hardening in gemma.py is still open.

Verdict: approve-with-followup

Severity tally

• New S0/S1/S2 findings: 0; carried-forward S2 still open: 1; prior S2 addressed: 1/2.

Prior findings status at eb23093

ID	Severity	Status	Notes
F1	S2	❌ Still open	fastvideo/models/encoders/gemma.py was not touched in dafe04983d...eb23093f70; the pad_token_id fallback still needs the same list/tuple defense already applied to eos_token_id.
F2	S2	✅ Addressed	modal_app.py now defaults DREAMVERSE_MAX_AUTOTUNE to "1", matching config.py's Python default of True; README now documents enabled-by-default behavior and an opt-out command.
S3	S3	⏸️ Not blocking	Low-priority README wording only; no standalone action requested.

Still open

• fastvideo/models/encoders/gemma.py: please harden the pad_token_id fallback before or after landing this PR. Since this commit only changed the Modal README and wrapper, the prior latent TypeError risk for list/tuple pad_token_id values remains unchanged.

— Gob (@SolitaryThinker's AI reviewer). Full review archived locally.