diff --git a/README.md b/README.md index 12cb04f3f1..beca13419d 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,8 @@ FastVideo has the following features: - Support H100, A100, 4090 - Support Linux, Windows, MacOS - See this [page](https://hao-ai-lab.github.io/FastVideo/inference/support_matrix/) for full list of supported models, hardware assumptions, and optimization compatibility. +- Realtime video generation & editing + - [Dreamverse](apps/dreamverse/README.md): stream and "vibe direct" video in realtime ([live demo](https://dreamverse.fastvideo.org/)), deployable on local GPU, a self-hosted B200 server, Docker, or serverless Modal ## Getting Started @@ -69,6 +71,19 @@ See below for recipes and datasets: | [FastWan2.1-T2V-1.3B](https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers) | [Recipe](https://github.com/hao-ai-lab/FastVideo/tree/main/examples/distill/Wan2.1-T2V/Wan-Syn-Data-480P) | [FastVideo Synthetic Wan2.1 480P](https://huggingface.co/datasets/FastVideo/Wan-Syn_77x448x832_600k) | | [FastWan2.2-TI2V-5B](https://huggingface.co/FastVideo/FastWan2.2-TI2V-5B-Diffusers) | [Recipe](https://github.com/hao-ai-lab/FastVideo/tree/main/examples/distill/Wan2.2-TI2V-5B-Diffusers/Data-free) | [FastVideo Synthetic Wan2.2 720P](https://huggingface.co/datasets/FastVideo/Wan2.2-Syn-121x704x1280_32k) | +## Dreamverse — Realtime Video Generation & Editing + +[Dreamverse](apps/dreamverse/README.md) is FastVideo's realtime video generation +and editing platform — "vibe directing" a video as it streams. It lives in the +monorepo under [`apps/dreamverse/`](apps/dreamverse/) and ships its own backend +(`dreamverse-server`) plus a web UI. + +Try the [live demo](https://dreamverse.fastvideo.org/), read the +[blog](https://haoailab.com/blogs/dreamverse/), or run it yourself. Dreamverse +deploys on a local GPU, a self-hosted B200 server over SSH, Docker, or +serverless [Modal](apps/dreamverse/scripts/modal/README.md) — see the +[Dreamverse README](apps/dreamverse/README.md). + ## Inference ### Generating Your First Video diff --git a/apps/dreamverse/README.md b/apps/dreamverse/README.md index 33bf7cfb1b..6415d3eaf8 100644 --- a/apps/dreamverse/README.md +++ b/apps/dreamverse/README.md @@ -2,6 +2,8 @@ Dreamverse is the FastVideo realtime video generation & editing platform. It lives in this monorepo under `apps/dreamverse/`. +**Deploy on:** [local GPU](#quick-start-local-gpu) · [self-hosted B200 (SSH)](#server-b200-deployment-ssh) · [Docker](docker/README.md) · [Modal](scripts/modal/README.md) + ## Install Dreamverse You can install Dreamverse using one of the methods below. @@ -93,6 +95,14 @@ dreamverse-server --port 8009 dreamverse-mock-server --port 8009 ``` +> **Expect a slow first boot.** With `torch.compile` and startup warmup enabled +> (the default), the backend compiles the segment 1 and segment 2 inference +> paths before it reports ready — this can take **tens of minutes on a cold +> cache**, regardless of how you deploy (local, server, Docker, or Modal). +> `/healthz` responds as soon as the process is up; `/readyz` stays `503` until +> warmup finishes. For a faster, uncompiled startup while testing, set +> `FASTVIDEO_ENABLE_STARTUP_WARMUP=0` before starting the backend. + ## Frontend Setup Install the web dependencies once from the FastVideo checkout: @@ -163,6 +173,32 @@ BACKEND_HOST=localhost BACKEND_PORT=8009 npm run dev Open `http://localhost:5299`. +## Server B200 deployment (SSH) + +Deploying on a remote GPU host (for example a B200 box) is a local install run +over SSH, plus a few server-specific concerns. Two paths: + +### Option A: Native (source install) + +SSH in, then follow [Install → From source](#method-2-from-source) and +(recommended) [Building FFmpeg](#optional-building-ffmpeg-for-better-performance), +then start the backend as in [Quick Start: Local GPU](#quick-start-local-gpu). +For a remote host, a few things differ from localhost: + +- Bind all interfaces: `dreamverse-server --host 0.0.0.0 --port 8009`. +- Point the frontend/client at the host: `BACKEND_HOST= BACKEND_PORT=8009 npm run dev`. +- Keep the backend alive across SSH sessions (`tmux` / `systemd` / `nohup`). +- Expose / firewall port `8009`, or front it with a reverse proxy + auth. + +### Option B: Docker (on the server) + +SSH in, then follow [Install → Using Docker](#method-3-using-docker) and the run +steps in [`docker/README.md`](docker/README.md): + +```bash +CEREBRAS_API_KEY="" GROQ_API_KEY="" apps/dreamverse/docker/docker_run.sh +``` + ## Quick Start: Mock Backend (For UI development) The mock server emulates the Dreamverse backend protocol and streams a