|
1 | | -[](https://opensource.org/licenses/MIT) [](https://www.python.org/downloads/) [](https://github.com/comfyanonymous/ComfyUI) [](https://huggingface.co/lightx2v) [](https://deepwiki.com/GacLove/ComfyUI-Lightx2vWrapper) |
| 1 | +[](LICENSE) |
| 2 | +[](https://www.python.org/downloads/) |
| 3 | +[](https://github.com/comfyanonymous/ComfyUI) |
| 4 | +[](https://huggingface.co/lightx2v) |
2 | 5 |
|
3 | 6 | # ComfyUI-Lightx2vWrapper |
4 | 7 |
|
5 | | -[中文版](./README_CN.md) | English |
| 8 | +ComfyUI custom nodes for [LightX2V](https://github.com/ModelTC/LightX2V). The |
| 9 | +wrapper exposes LightX2V's modular configuration and runner APIs, together |
| 10 | +with ComfyUI-friendly file inputs and output previews. |
6 | 11 |
|
7 | | -A ComfyUI custom node wrapper for LightX2V, enabling modular video generation with advanced optimization features. |
| 12 | +[中文文档](README_CN.md) |
8 | 13 |
|
9 | | -## Features |
| 14 | +## Supported workflows |
10 | 15 |
|
11 | | -- **Modular Configuration System**: Separate nodes for each aspect of video generation |
12 | | -- **Text-to-Video (T2V) and Image-to-Video (I2V)**: Support for both generation modes |
13 | | -- **Advanced Optimizations**: |
14 | | - - TeaCache acceleration (up to 3x speedup) |
15 | | - - Quantization support (int8, fp8) |
16 | | - - Memory optimization with CPU offloading |
17 | | - - Lightweight VAE options |
18 | | -- **LoRA Support**: Chain multiple LoRA models for customization |
19 | | -- **Multiple Model Support**: wan2.1, hunyuan architectures |
| 16 | +The repository includes runnable workflow examples under |
| 17 | +[`example_workflows/`](example_workflows/): |
20 | 18 |
|
21 | | -## Installation |
| 19 | +- [Wan 2.2](example_workflows/lighx2v_wan2.2.json) text/image-to-video |
| 20 | +- [SekoTalk](example_workflows/lightx2v_seko_talk.json) and [static |
| 21 | + SekoTalk](example_workflows/lightx2v_seko_talk_static.json) |
| 22 | +- [Multi-person SekoTalk](example_workflows/lightx2v_seko_talk_multi_talk.json) |
| 23 | +- [SeedVR2](example_workflows/lightx2v_seedvr.json) video restoration |
| 24 | +- [SwiftVR image](example_workflows/lightx2v_swiftvr_image.json) and [SwiftVR |
| 25 | + video](example_workflows/lightx2v_swiftvr_video.json) restoration |
22 | 26 |
|
23 | | -1. Clone this repository with submodules into your ComfyUI's `custom_nodes` directory: |
| 27 | +The examples are versioned with the wrapper and are the best reference for |
| 28 | +current node names and connections. |
24 | 29 |
|
25 | | -```bash |
26 | | -cd ComfyUI/custom_nodes |
27 | | -git clone --recursive https://github.com/gaclove/ComfyUI-Lightx2vWrapper.git |
28 | | -``` |
| 30 | +## Installation |
29 | 31 |
|
30 | | -If you already cloned without submodules, initialize them: |
| 32 | +Clone the repository with its LightX2V submodule into ComfyUI's |
| 33 | +`custom_nodes` directory: |
31 | 34 |
|
32 | 35 | ```bash |
| 36 | +cd ComfyUI/custom_nodes |
| 37 | +git clone --recursive https://github.com/ModelTC/ComfyUI-Lightx2vWrapper.git |
33 | 38 | cd ComfyUI-Lightx2vWrapper |
34 | | -git submodule update --init --recursive |
| 39 | +python -m pip install -r lightx2v/requirements.txt |
| 40 | +python -m pip install -r requirements.txt |
35 | 41 | ``` |
36 | 42 |
|
37 | | -2. Install dependencies: |
| 43 | +For an existing checkout, update the submodule with: |
38 | 44 |
|
39 | 45 | ```bash |
40 | | -cd ComfyUI-Lightx2vWrapper |
41 | | -# Install lightx2v submodule dependencies |
42 | | -pip install -r lightx2v/requirements.txt |
43 | | -# Install ComfyUI wrapper dependencies |
44 | | -pip install -r requirements.txt |
| 46 | +git submodule update --init --recursive |
45 | 47 | ``` |
46 | 48 |
|
47 | | -3. Download models and place them in `ComfyUI/models/lightx2v/` directory |
48 | | - |
49 | | -## Node Overview |
50 | | - |
51 | | -### Configuration Nodes |
52 | | - |
53 | | -#### 1. LightX2V Inference Config |
54 | | - |
55 | | -Basic inference configuration for video generation. |
56 | | - |
57 | | -- **Inputs**: model, task_type, inference_steps, seed, cfg_scale, width, height, video_length, fps |
58 | | -- **Output**: Base configuration object |
59 | | - |
60 | | -#### 2. LightX2V TeaCache |
61 | | - |
62 | | -Feature caching acceleration configuration. |
63 | | - |
64 | | -- **Inputs**: enable, threshold (0.0-1.0), use_ret_steps |
65 | | -- **Output**: TeaCache configuration |
66 | | -- **Note**: Lower threshold = more speedup (0.1 ~2x, 0.2 ~3x) |
67 | | - |
68 | | -#### 3. LightX2V Quantization |
69 | | - |
70 | | -Model quantization settings for memory efficiency. |
71 | | - |
72 | | -- **Inputs**: dit_precision, t5_precision, clip_precision, backend, sensitive_layers_precision |
73 | | -- **Output**: Quantization configuration |
74 | | -- **Backends**: Auto-detected (vllm, sgl, q8f) |
75 | | - |
76 | | -#### 4. LightX2V Memory Optimization |
77 | | - |
78 | | -Memory management strategies. |
79 | | - |
80 | | -- **Inputs**: optimization_level, attention_type, enable_rotary_chunking, cpu_offload, unload_after_generate |
81 | | -- **Output**: Memory optimization configuration |
82 | | - |
83 | | -#### 5. LightX2V Lightweight VAE |
84 | | - |
85 | | -VAE optimization options. |
86 | | - |
87 | | -- **Inputs**: use_tiny_vae, use_tiling_vae |
88 | | -- **Output**: VAE configuration |
89 | | - |
90 | | -#### 6. LightX2V LoRA Loader |
91 | | - |
92 | | -Load and chain LoRA models. |
| 49 | +Download the model files from [LightX2V on Hugging Face](https://huggingface.co/lightx2v) |
| 50 | +and place them under `ComfyUI/models/lightx2v/`. |
93 | 51 |
|
94 | | -- **Inputs**: lora_name, strength (0.0-2.0), lora_chain (optional) |
95 | | -- **Output**: LoRA chain configuration |
| 52 | +## Node groups |
96 | 53 |
|
97 | | -### Combination Node |
| 54 | +- **Configuration:** `LightX2VInferenceConfig`, `LightX2VTeaCache`, |
| 55 | + `LightX2VQuantization`, and `LightX2VMemoryOptimization`. |
| 56 | +- **Composition:** `LightX2VConfigCombinerV2` and `LightX2VConfigCombinerV3`. |
| 57 | +- **Inference:** `LightX2VModularInferenceV2`. |
| 58 | +- **Talk inputs:** single-object, file-list, JSON, and multi-object talk |
| 59 | + combiners. |
| 60 | +- **Video inputs and restoration:** `LightX2VInputVideoPath`, SeedVR2 and |
| 61 | + SwiftVR loaders/samplers, plus `LightX2VOutputVideoPreview`. |
| 62 | +- **LoRA:** `LightX2VLoRALoader`. |
98 | 63 |
|
99 | | -#### 7. LightX2V Config Combiner |
| 64 | +The inference node currently accepts the model classes exposed by |
| 65 | +`model_utils.support_model_cls_list`: Wan 2.1/2.2 variants, SekoTalk, |
| 66 | +Qwen Image, and SeedVR2. The retired `*_distill` values remain selectable as |
| 67 | +compatibility aliases for saved ComfyUI workflows and are normalized before |
| 68 | +LightX2V's runner is built. |
100 | 69 |
|
101 | | -Combines all configuration modules into a single configuration. |
| 70 | +## Model files |
102 | 71 |
|
103 | | -- **Inputs**: All configuration types (optional) |
104 | | -- **Output**: Combined configuration object |
| 72 | +Place each model in its own directory under `ComfyUI/models/lightx2v/`. LoRA |
| 73 | +files can be placed in either `ComfyUI/models/lightx2v/loras/` or |
| 74 | +`ComfyUI/models/loras/`. |
105 | 75 |
|
106 | | -### Inference Node |
107 | | - |
108 | | -#### 8. LightX2V Modular Inference |
109 | | - |
110 | | -Main inference node for video generation. |
111 | | - |
112 | | -- **Inputs**: combined_config, prompt, negative_prompt, image (optional), audio (optional) |
113 | | -- **Outputs**: Generated video frames |
114 | | - |
115 | | -## Usage Examples |
116 | | - |
117 | | -### Basic T2V Workflow |
118 | | - |
119 | | -1. Create LightX2V Inference Config (task_type: "t2v") |
120 | | -2. Use LightX2V Config Combiner |
121 | | -3. Connect to LightX2V Modular Inference with text prompt |
122 | | -4. Save video output |
123 | | - |
124 | | -### I2V with Optimizations |
125 | | - |
126 | | -1. Load input image |
127 | | -2. Create LightX2V Inference Config (task_type: "i2v") |
128 | | -3. Add LightX2V TeaCache (threshold: 0.26) |
129 | | -4. Add LightX2V Memory Optimization |
130 | | -5. Combine configs with LightX2V Config Combiner |
131 | | -6. Run LightX2V Modular Inference |
132 | | - |
133 | | -### With LoRA |
134 | | - |
135 | | -1. Create base configuration |
136 | | -2. Load LoRA with LightX2V LoRA Loader |
137 | | -3. Chain multiple LoRAs if needed |
138 | | -4. Combine all configs |
139 | | -5. Run inference |
140 | | - |
141 | | -## Model Directory Structure |
142 | | - |
143 | | -Download models from: <https://huggingface.co/lightx2v> |
144 | | - |
145 | | -Models should be placed in: |
146 | | - |
147 | | -```txt |
| 76 | +```text |
148 | 77 | ComfyUI/models/lightx2v/ |
149 | | -├── Wan2.1-I2V-14B-720P-xxx/ # Main model checkpoints |
150 | | -├── Wan2.1-I2V-14B-480P-xxx/ # Main model checkpoints |
151 | | -├── loras/ # LoRA models |
| 78 | +├── Wan2.2-I2V-A14B-distill-fp8/ |
| 79 | +├── seedvr2_ema_3b_fp8/ |
| 80 | +└── loras/ |
152 | 81 | ``` |
153 | 82 |
|
154 | | -## Tips |
| 83 | +The exact directory names depend on the downloaded checkpoint. The model |
| 84 | +picker only lists directories that exist on the local ComfyUI installation. |
155 | 85 |
|
156 | | -- Start with default settings and adjust based on your hardware |
157 | | -- Use TeaCache with threshold 0.1-0.2 for significant speedup |
158 | | -- Enable memory optimization if running on limited VRAM |
159 | | -- Quantization can reduce memory usage but may affect quality |
160 | | -- Chain multiple LoRAs for complex style combinations |
| 86 | +## Development |
161 | 87 |
|
162 | | -## Troubleshooting |
163 | | - |
164 | | -- **Out of Memory**: Enable memory optimization or use quantization |
165 | | -- **Slow Generation**: Enable TeaCache or reduce inference steps |
166 | | -- **Model Not Found**: Check model paths in `ComfyUI/models/lightx2v/` |
167 | | - |
168 | | -## Example Workflows |
169 | | - |
170 | | -Example workflow JSON files are provided in the `examples/` directory: |
171 | | - |
172 | | -- `wan_i2v.json` - Basic image-to-video |
173 | | -- `wan_i2v_with_distill_lora.json` - I2V with distillation LoRA |
174 | | -- `wan_t2v_with_distill_lora.json` - T2V with distillation LoRA |
175 | | - |
176 | | -## Contributing Guidelines |
177 | | - |
178 | | -We welcome community contributions! Before submitting code, please ensure you follow these steps: |
179 | | - |
180 | | -### Install Development Dependencies |
181 | | - |
182 | | -```bash |
183 | | -pip install ruff pre-commit |
184 | | -``` |
185 | | - |
186 | | -### Code Quality Check |
187 | | - |
188 | | -Before committing code, run the following command: |
| 88 | +Install the development hook and run the same checks used by CI: |
189 | 89 |
|
190 | 90 | ```bash |
| 91 | +python -m pip install pre-commit |
| 92 | +pre-commit install |
191 | 93 | pre-commit run --all-files |
192 | 94 | ``` |
193 | 95 |
|
194 | | -This will automatically check code formatting, syntax errors, and other code quality issues. |
195 | | - |
196 | | -### Contribution Process |
197 | | - |
198 | | -1. Fork this repository |
199 | | -2. Create a feature branch (`git checkout -b feature/amazing-feature`) |
200 | | -3. Commit your changes (`git commit -m 'Add some amazing feature'`) |
201 | | -4. Push to the branch (`git push origin feature/amazing-feature`) |
202 | | -5. Create a Pull Request |
| 96 | +See [`CONTRIBUTING.md`](CONTRIBUTING.md) for pull request and workflow |
| 97 | +validation guidelines. |
0 commit comments