Fix Lucy Edit review items

Author: aryan5v

.agents/lessons/2026-05-22_dreamverse-ci-streaming-imports-need-gpu.md

@@ -0,0 +1,41 @@
+---
+date: 2026-05-22
+experiment: PR #1386 DreamVerse app CI backend tests
+category: infrastructure
+severity: important
+---
+
+# DreamVerse App CI Streaming Imports Need GPU
+
+## What Happened
+
+DreamVerse app CI backend pytest collection imports FastVideo streaming surfaces.
+When those tests run in a CPU-only Modal environment, collection can fail before
+any app assertions run with Triton reporting:
+
+```text
+RuntimeError: 0 active drivers
+```
+
+## Root Cause
+
+Some streaming import paths can import `fastvideo_kernel` at module import time.
+Triton then probes for an active GPU driver during pytest collection. A CPU-only
+Modal container has no active driver, so the failure appears as an import-time
+collection error rather than a DreamVerse app behavior failure.
+
+## Fix / Workaround
+
+For PR #1386, use a surgical CI fix: allocate a GPU to
+`run_dreamverse_app_tests`. Do not refactor core streaming/kernel imports just to
+unstick this app CI path.
+
+Keep `build_kernel=False` for this job. The DreamVerse app backend test imports
+streaming surfaces but does not need to rebuild or exercise custom kernels.
+
+## Prevention
+
+When adding or modifying DreamVerse app CI jobs that import FastVideo streaming
+modules, make the GPU requirement explicit if the import graph may touch
+`fastvideo_kernel`. Prefer small CI resource fixes for app test collection issues
+unless the product code genuinely requires lazy import cleanup.

.agents/memory/codebase-map/README.md

@@ -31,7 +31,7 @@ FastVideo-WorldModel/
 │   │   │   └── distribution_matching/  # DMD2Method, SelfForcingMethod
 │   │   ├── models/            #   Per-role model wrappers (ModelBase, CausalModelBase)
 │   │   │   ├── wan/           #     WanModel, WanCausalModel
-│   │   │   └── matrixgame/    #     MatrixGameModel, MatrixGameCausalModel
+│   │   │   └── matrixgame2/   #     MatrixGame2Model, MatrixGame2CausalModel
 │   │   ├── callbacks/         #   Composable hooks (grad_clip, ema, validation)
 │   │   └── utils/             #   Config, builder, checkpoint, optimizer, tracking
 │   ├── training/              # Legacy training infrastructure (being phased out)
@@ -44,7 +44,7 @@ FastVideo-WorldModel/
 │   │   ├── wan_distillation_pipeline.py # Wan distillation
 │   │   ├── self_forcing_distillation_pipeline.py # Self-forcing distill
 │   │   ├── ltx2_training_pipeline.py   # LTX-2 training
-│   │   └── matrixgame_training_pipeline.py # MatrixGame training
+│   │   └── matrixgame2_training_pipeline.py # Matrix-Game 2.0 training
 │   ├── attention/             # Attention backends
 │   ├── distributed/           # Sequence/tensor parallel utilities
 │   ├── layers/                # Tensor-parallel layers
@@ -95,7 +95,7 @@ FastVideo-WorldModel/
 | Wan distillation (DMD) | `fastvideo/training/wan_distillation_pipeline.py` | `torchrun --nproc_per_node N` |
 | Self-forcing distill | `fastvideo/training/wan_self_forcing_distillation_pipeline.py` | `torchrun --nproc_per_node N` |
 | LTX-2 finetune | `fastvideo/training/ltx2_training_pipeline.py` | `torchrun --nproc_per_node N` |
-| MatrixGame | `fastvideo/training/matrixgame_training_pipeline.py` | `torchrun --nproc_per_node N` |
+| Matrix-Game 2.0 | `fastvideo/training/matrixgame2_training_pipeline.py` | `torchrun --nproc_per_node N` |
 
 ## W&B Integration
 

.agents/memory/dreamverse-integration/decisions-log.md

@@ -8,6 +8,8 @@ follow-up actions see [open-threads.md](open-threads.md).
 
 **Last updated:** 2026-05-06 (added D-21 — chunk-stutter root cause is software libx264 encoding consuming ~22% of segment wall-time, NOT a migration regression; verified by 3 parallel explore agents that NVFP4 + torch.compile coverage matches FastVideo-internal exactly; landed opt-in NVENC build path in install_native_ffmpeg.sh + `--nvenc`/`--no-nvenc` flag in dreamverse-deploy.sh + `apps/dreamverse/server/benchmarks/benchmark_av_streaming.py` regression test + memory dir update; default codec stays `libx264` for backward compat, opt-in via `--nvenc`. Earlier: added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle; added D-16 — streaming router polish round 2, second-pass review on top of D-15 covering bridge cancellation hygiene, registry state machine, httpx hard-fail, replica YAML parsing, and `websockets` dep; added D-17 — strategy reversal: abandon 6-PR split in favor of single mega-PR #1288 on `will/ltx2_sr_port`; added D-18 — Option B+ chosen: Dreamverse FE+product-server move into FastVideo as `apps/dreamverse/` subfolder while generic backend stays at `fastvideo.entrypoints.streaming.*`; integration-review.md deprecated, integration-plan.md is the executable migration plan; added D-19 — D-18 executed: 5 commits land on `will/dreamverse-monorepo`, fix-up commits corrected the integration-plan's invalid "delete generic-merged, import public substitutes" assumption — generic-merged files carried product-local instead, e2e passes against migrated code with /proc-verified evidence; added D-20 — segment-2 BrokenPipe root cause was a TWO-direction silent drop of LTX-2 audio kwargs in public `VideoGenerator`).
 
+**Update 2026-05:** Dreamverse frontend tooling migrated from standalone pnpm to standalone npm. `apps/dreamverse/web/package-lock.json` is authoritative; see PR #1385.
+
 ## Status legend
 
 - ✅ **Resolved** — decision made and implementation complete (or no implementation needed)
@@ -297,14 +299,14 @@ Playwright (8/8 PASS in 5.1s):
 - **Python ML library** stays at root: `fastvideo/`, `fastvideo-kernel/`.
 - **Generic backend** stays at `fastvideo.entrypoints.streaming.*` (already there per #1257/#1258/#1284/#1286/#1288).
 - **Dreamverse product** moves into `apps/dreamverse/{server,web,prompts,serve_configs,scripts}/`.
-- **Tooling**: uv workspace for Python (`[tool.uv.workspace] members = ["apps/dreamverse/server"]`), standalone pnpm for the FE (no root `package.json`), split CI workflows with path-filter triggers.
+- **Tooling**: uv workspace for Python (`[tool.uv.workspace] members = ["apps/dreamverse/server"]`), standalone npm for the FE (no root `package.json`), split CI workflows with path-filter triggers.
 
 **Rationale:**
 
 - Drops the cross-repo coordination overhead identified in the post-#1286 rebase cycle (D-17 handled by consolidating into mega-PR; D-18 prevents the next round of cross-repo coordination from happening).
 - Keeps the architectural separation Option D recommended (FastVideo owns reusable runtime; product owns product). The boundary is now `apps/dreamverse/` directory rather than two repos.
 - Single repo means atomic cross-cutting refactors (e.g. GpuPool API change + Dreamverse adoption) ship as one PR.
-- OSS precedents support the shape (chainlit uv-workspace + pnpm; open-webui Python + Svelte with paths-ignore CI). The librarian explicitly noted no precedent for "Python ML library + Next.js product merged into library namespace" — but this isn't that pattern. Dreamverse goes into a sibling directory, NOT into `fastvideo.entrypoints.dreamverse.*`. Library namespace stays clean.
+- OSS precedents support the shape (chainlit uv-workspace + frontend package manager; open-webui Python + Svelte with paths-ignore CI). The librarian explicitly noted no precedent for "Python ML library + Next.js product merged into library namespace" — but this isn't that pattern. Dreamverse goes into a sibling directory, NOT into `fastvideo.entrypoints.dreamverse.*`. Library namespace stays clean.
 
 **Why not Option D (separate repos):**
 

.agents/memory/dreamverse-integration/integration-plan.md

@@ -311,22 +311,22 @@ Notes:
   development and CI.
 - Product server release is Docker/deploy workflow, not PyPI.
 
-### Frontend build: standalone pnpm
+### Frontend build: standalone npm
 
 Do not add a root `package.json`.
 
 Keep all frontend tooling under:
 
 ```text
 apps/dreamverse/web/package.json
-apps/dreamverse/web/pnpm-lock.yaml
+apps/dreamverse/web/package-lock.json
 apps/dreamverse/web/playwright.config.*
 ```
 
 Rationale:
 
 - FastVideo remains primarily a Python ML library.
-- Python contributors should not need Node or pnpm for normal work.
+- Python contributors should not need Node or npm for normal work.
 - This intentionally diverges from chainlit's root JS workspace pattern and
   follows the simpler open-webui-style split.
 
@@ -455,31 +455,24 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      # IMPORTANT: pnpm/action-setup MUST run BEFORE setup-node when using
-      # cache: pnpm — setup-node otherwise can't find pnpm to populate cache.
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 9
-
       - name: Setup Node
         uses: actions/setup-node@v4
         with:
           node-version: '22'
-          cache: pnpm
-          cache-dependency-path: apps/dreamverse/web/pnpm-lock.yaml
+          cache: npm
+          cache-dependency-path: apps/dreamverse/web/package-lock.json
 
       - name: Install dependencies
-        run: pnpm install --frozen-lockfile
+        run: npm ci
 
       - name: Typecheck
-        run: pnpm run typecheck --if-present
+        run: npm run typecheck --if-present
 
       - name: Unit tests
-        run: pnpm run test --if-present
+        run: npm run test --if-present
 
       - name: Build
-        run: pnpm run build
+        run: npm run build
 
       # NOTE: Playwright tests require a running backend. Until Phase 4 lands
       # `/healthz`/`/readyz`/`/status`/`/prompt-system-config`/`/curated-presets`
@@ -489,13 +482,13 @@ jobs:
       # `fastvideo.entrypoints.streaming.build_app` and add product routes).
       #
       # - name: Install Playwright browsers
-      #   run: pnpm exec playwright install --with-deps chromium
+      #   run: npm exec -- playwright install --with-deps chromium
       #
       # - name: Playwright (re-enable in Phase 4)
-      #   run: pnpm exec playwright test
+      #   run: npm exec -- playwright test
 ```
 
-**Playwright config update needed** when moving FE: `Dreamverse/apps/web/playwright.config.ts` line 39 currently uses `npm run dev`; change to `pnpm run dev` post-move ([source](file:///home/william5lin/Dreamverse/apps/web/playwright.config.ts#L39)).
+**Playwright config update needed** when moving FE: keep `Dreamverse/apps/web/playwright.config.ts` line 39 on `npm run dev` after the move ([source](file:///home/william5lin/Dreamverse/apps/web/playwright.config.ts#L39)).
 
 #### `.github/workflows/ci-dreamverse-backend.yml`
 
@@ -604,7 +597,7 @@ monorepo CI path.
 | uv workspaces | https://docs.astral.sh/uv/concepts/workspaces/ | Authoritative Python workspace model. |
 | Hatch monorepo | https://hatch.pypa.io/latest/how-to/environment/workspace/ | Alternative workspace model; not selected. |
 | chainlit | https://github.com/Chainlit/chainlit | uv workspace precedent plus **per-language CI split** (separate `check-frontend.yaml` / `check-backend.yaml` workflows path-filtered by directory). Not a PR-level split — independent of D-17 single-mega-PR decision. |
-| open-webui | https://github.com/open-webui/open-webui | Frontend path filtering (`paths-ignore` on backend-only changes) and separate release tracks. **Note:** open-webui has a root `package.json`; we are choosing standalone-pnpm despite the precedent, to avoid forcing Python-only contributors to install Node. |
+| open-webui | https://github.com/open-webui/open-webui | Frontend path filtering (`paths-ignore` on backend-only changes) and separate release tracks. **Note:** open-webui has a root `package.json`; we are choosing standalone npm under `apps/dreamverse/web/` despite the precedent, to avoid forcing Python-only contributors to install Node. |
 | streamlit | https://github.com/streamlit/streamlit | Split Python and JS testing in one repo. |
 | gradio | https://github.com/gradio-app/gradio | Python package plus JS workspace precedent. |
 | full-stack-fastapi-template-nextjs | https://github.com/nemanjam/full-stack-fastapi-template-nextjs | Separate frontend/backend build and deploy workflows. |
@@ -623,8 +616,8 @@ class runs where, because not all tests can run on `ubuntu-latest` CI.
 | **Unit** | (none / `unit`) | `ci-dreamverse-backend.yml` (ubuntu-latest CI) + locally | Pure logic, no GPU, no live service. Mocked FastVideo backends, schema validation, helper functions. | `test_config.py`, `test_rewrite_prompt_payload.py`, `test_session_init_image.py`, the new `test_import_contract.py` |
 | **Integration (fakes)** | `integration` | `ci-dreamverse-backend.yml` + locally | FastAPI test client + in-process fakes/mocks for GPU pool. Validates routes, request/response shapes, session state machine. | `test_health_endpoints.py`, `test_mock_server.py`, `test_entrypoints.py`, `test_prompt_safety.py`, `test_batching.py` (deleted) |
 | **Live-service GPU** | `gpu` (skip-by-default in CI) | **Local GPU4 manual QA** + Buildkite-Modal (when added) | Real `fastvideo serve` process + real model weights + real WebSocket round-trips. Validates LTX-2 streaming, NVFP4 wiring, continuation state, frame emission. | `test_realtime_stress.py` (947 LOC), `test_session_logging.py` (1278 LOC) — these spin up real workers per their current shape |
-| **Frontend unit / build** | (n/a — pnpm) | `ci-dreamverse-frontend.yml` (ubuntu-latest, no GPU) | Vitest + tsc + Next.js build. No backend needed. | `apps/dreamverse/web/src/**/*.test.ts(x)` |
-| **Frontend Playwright E2E** | (n/a — pnpm) | **Local GPU4 manual QA** until Phase 4 lands public health routes; then `ci-dreamverse-frontend.yml` against a mock backend OR a deployed staging | Real browser → real backend WebSocket flow. Requires `/healthz`, `/readyz`, `/status`, `/prompt-system-config`, `/curated-presets`, `/v1/stream`. | `apps/dreamverse/web/e2e/{backend-health,frontend-shell,preset-prompt-generation}.spec.ts` |
+| **Frontend unit / build** | (n/a — npm) | `ci-dreamverse-frontend.yml` (ubuntu-latest, no GPU) | Vitest + tsc + Next.js build. No backend needed. | `apps/dreamverse/web/src/**/*.test.ts(x)` |
+| **Frontend Playwright E2E** | (n/a — npm) | **Local GPU4 manual QA** until Phase 4 lands public health routes; then `ci-dreamverse-frontend.yml` against a mock backend OR a deployed staging | Real browser → real backend WebSocket flow. Requires `/healthz`, `/readyz`, `/status`, `/prompt-system-config`, `/curated-presets`, `/v1/stream`. | `apps/dreamverse/web/e2e/{backend-health,frontend-shell,preset-prompt-generation}.spec.ts` |
 | **FastVideo public contract** | (none) | Existing FastVideo CI (`ci-precommit` + Buildkite for GPU) | Schema/shape guards that this migration must not break. | `fastvideo/tests/contract/test_dreamverse_shape.py`, `test_dynamo_shape.py`, `test_generate_async.py` |
 | **FastVideo SSIM regression** | (Buildkite path-filter) | Buildkite-Modal | Inference-quality gates for ported models. | `fastvideo/tests/ssim/test_*.py` |
 
@@ -709,7 +702,7 @@ CUDA_VISIBLE_DEVICES=4 uv run --locked --package dreamverse-server \
 # Smoke-test from another terminal
 curl -s http://localhost:8009/health | jq .
 curl -s http://localhost:8009/readyz | jq .   # Phase 4+ only
-# Drive the FE against it: cd apps/dreamverse/web && pnpm run dev
+# Drive the FE against it: cd apps/dreamverse/web && npm run dev
 ```
 
 This is the **manual QA gate** for any phase that touches the live-service
@@ -907,7 +900,7 @@ mass move. This should be a small PR.
 6. Add `apps/dreamverse/web/.*` to pre-commit global exclude.
 7. Add `docs/contributing/dreamverse-development.md` with local dev commands:
    backend `uv run --locked --package dreamverse-server --extra test pytest ...`;
-   frontend `cd apps/dreamverse/web && pnpm install && pnpm run build`.
+   frontend `cd apps/dreamverse/web && npm ci && npm run build`.
 8. Run `uv lock` to regenerate `uv.lock` with the new workspace member; commit
    the lock change in the same PR.
 9. Land D-12-A docstring caveat: mark `GpuPool` experimental/server-internal
@@ -990,7 +983,7 @@ These are **explicit shims** — Phase 4 is responsible for their promotion to p
 
 - [`config.py:13`](file:///home/william5lin/Dreamverse/server/config.py#L13): `_APP_ROOT / "apps" / "web"` → `_APP_ROOT / "web"` (since `_APP_ROOT` will resolve to `apps/dreamverse/` in the new layout).
 - [`apps/web/next.config.ts:11`](file:///home/william5lin/Dreamverse/apps/web/next.config.ts#L11): `outputFileTracingRoot: path.resolve(__dirname, "../..")` → `path.resolve(__dirname, "../../..")` (one extra `..` since the FE is one level deeper in the monorepo).
-- [`playwright.config.ts:39`](file:///home/william5lin/Dreamverse/apps/web/playwright.config.ts#L39): `command: "npm run dev"` → `command: "pnpm run dev"` (matches Phase 1 tooling decision).
+- [`playwright.config.ts:39`](file:///home/william5lin/Dreamverse/apps/web/playwright.config.ts#L39): keep `command: "npm run dev"` (matches Phase 1 tooling decision).
 - Any hardcoded `../FastVideo` paths in scripts/configs → make repo-root-relative since they now share a repo.
 
 **Steps:**
@@ -1084,17 +1077,17 @@ scripts, and product docs after backend tests are green.
 
 **Verification gate:**
 
-- `cd apps/dreamverse/web && pnpm install --frozen-lockfile` succeeds.
-- `cd apps/dreamverse/web && pnpm run build` succeeds.
-- `cd apps/dreamverse/web && pnpm run test --if-present` succeeds (Vitest + tsc).
+- `cd apps/dreamverse/web && npm ci` succeeds.
+- `cd apps/dreamverse/web && npm run build` succeeds.
+- `cd apps/dreamverse/web && npm run test --if-present` succeeds (Vitest + tsc).
 - **Frontend CI Playwright is intentionally DEFERRED to Phase 4** — at Phase 3
   the public `build_app` does not yet expose `/healthz`+`/readyz`+`/status`+
   `/prompt-system-config`+`/curated-presets`. The `ci-dreamverse-frontend.yml`
   scaffold from Phase 1 keeps Playwright steps commented out until Phase 4
   reactivates them. No PR note required.
 - **Manual GPU4 Playwright smoke** (recommended): on this dev node, run
   `apps/dreamverse/web/e2e/frontend-shell.spec.ts` against the GPU4-deployed
-  backend from Phase 2 manual QA + a `pnpm run dev` frontend at port 5274
+  backend from Phase 2 manual QA + a `npm run dev` frontend at port 5274
   to confirm shell hydration. `backend-health.spec.ts` and
   `preset-prompt-generation.spec.ts` will fail until Phase 4 — that is
   expected; document as deferred.
@@ -1249,7 +1242,7 @@ product-specific prompt orchestration.
 | CI cost increase | Medium | Medium | Add Dreamverse path-specific workflows; add `paths-ignore` to broad workflows; rely on Buildkite monorepo diff watch lists. | CI owner |
 | FastVideo PyPI release accidentally includes Dreamverse app | High | Low | Add `apps*` to `[tool.setuptools.packages.find]` and `[tool.wheel]` excludes; verify built wheel contents. | Release owner |
 | Release cadence coupling | Medium | Medium | Keep FastVideo PyPI version release unchanged; Dreamverse uses Docker/Vercel deploy from app paths. | Release owner |
-| Frontend tooling drift | Medium | Medium | Pin pnpm lockfile in `apps/dreamverse/web/`; no root JS workspace. | Frontend owner |
+| Frontend tooling drift | Medium | Medium | Pin npm lockfile in `apps/dreamverse/web/`; no root JS workspace. | Frontend owner |
 | Security surface enlargement | Medium | Medium | Product routes stay in `apps/dreamverse/server`; only generic health/streaming routes go into FastVideo. | Backend owner |
 | Product-specific API leakage into `fastvideo.*` | High | Medium | Enforce import/module boundary; keep curated presets and prompt UX product-local. | Architecture owner |
 | Migration regression | High | Medium | Phase gates; backend before frontend; can stop after any phase with Dreamverse repo still usable until Phase 7. | Migration owner |