Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ on:
- "v*.*.*"

permissions:
contents: read
issues: write

jobs:
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Quality checks

on:
push:
pull_request:

permissions:
contents: read

jobs:
pre-commit:
name: Ruff and repository checks
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install pre-commit
run: python -m pip install pre-commit

- name: Run checks
run: pre-commit run --all-files
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -346,4 +346,4 @@ Temporary Items
CLAUDE.md
AGENTS.md
.gitnexus/
.claude/
.claude/
32 changes: 32 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Contributing

## Development setup

The wrapper follows the Python version supported by the bundled LightX2V
submodule (Python 3.10 or newer).

```bash
git clone --recursive https://github.com/ModelTC/ComfyUI-Lightx2vWrapper.git
cd ComfyUI-Lightx2vWrapper
python -m pip install -r lightx2v/requirements.txt
python -m pip install -r requirements.txt pre-commit
pre-commit install
```

## Before opening a pull request

Run the repository checks from the project root:

```bash
pre-commit run --all-files
```

For workflow or runner changes, validate the affected ComfyUI workflow with a
local LightX2V installation and include the workflow or reproduction details
in the pull request description. Keep changes to the `lightx2v` submodule in a
separate upstream contribution unless the wrapper specifically requires a
submodule revision update.

Pull requests should explain the user-visible behavior, list validation
performed, and avoid committing model weights, generated media, credentials,
or local debugging artifacts.
229 changes: 62 additions & 167 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,202 +1,97 @@
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT) [![Python](https://img.shields.io/badge/python-3.8%2B-blue)](https://www.python.org/downloads/) [![ComfyUI](https://img.shields.io/badge/ComfyUI-Custom%20Node-brightgreen)](https://github.com/comfyanonymous/ComfyUI) [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Models-lightx2v-yellow)](https://huggingface.co/lightx2v) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/GacLove/ComfyUI-Lightx2vWrapper)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
[![ComfyUI](https://img.shields.io/badge/ComfyUI-Custom%20Node-brightgreen)](https://github.com/comfyanonymous/ComfyUI)
[![Models](https://img.shields.io/badge/models-LightX2V-yellow)](https://huggingface.co/lightx2v)

# ComfyUI-Lightx2vWrapper

[中文版](./README_CN.md) | English
ComfyUI custom nodes for [LightX2V](https://github.com/ModelTC/LightX2V). The
wrapper exposes LightX2V's modular configuration and runner APIs, together
with ComfyUI-friendly file inputs and output previews.

A ComfyUI custom node wrapper for LightX2V, enabling modular video generation with advanced optimization features.
[中文文档](README_CN.md)

## Features
## Supported workflows

- **Modular Configuration System**: Separate nodes for each aspect of video generation
- **Text-to-Video (T2V) and Image-to-Video (I2V)**: Support for both generation modes
- **Advanced Optimizations**:
- TeaCache acceleration (up to 3x speedup)
- Quantization support (int8, fp8)
- Memory optimization with CPU offloading
- Lightweight VAE options
- **LoRA Support**: Chain multiple LoRA models for customization
- **Multiple Model Support**: wan2.1, hunyuan architectures
The repository includes runnable workflow examples under
[`example_workflows/`](example_workflows/):

## Installation
- [Wan 2.2](example_workflows/lighx2v_wan2.2.json) text/image-to-video
- [SekoTalk](example_workflows/lightx2v_seko_talk.json) and [static
SekoTalk](example_workflows/lightx2v_seko_talk_static.json)
- [Multi-person SekoTalk](example_workflows/lightx2v_seko_talk_multi_talk.json)
- [SeedVR2](example_workflows/lightx2v_seedvr.json) video restoration
- [SwiftVR image](example_workflows/lightx2v_swiftvr_image.json) and [SwiftVR
video](example_workflows/lightx2v_swiftvr_video.json) restoration

1. Clone this repository with submodules into your ComfyUI's `custom_nodes` directory:
The examples are versioned with the wrapper and are the best reference for
current node names and connections.

```bash
cd ComfyUI/custom_nodes
git clone --recursive https://github.com/gaclove/ComfyUI-Lightx2vWrapper.git
```
## Installation

If you already cloned without submodules, initialize them:
Clone the repository with its LightX2V submodule into ComfyUI's
`custom_nodes` directory:

```bash
cd ComfyUI/custom_nodes
git clone --recursive https://github.com/ModelTC/ComfyUI-Lightx2vWrapper.git
cd ComfyUI-Lightx2vWrapper
git submodule update --init --recursive
python -m pip install -r lightx2v/requirements.txt
python -m pip install -r requirements.txt
```

2. Install dependencies:
For an existing checkout, update the submodule with:

```bash
cd ComfyUI-Lightx2vWrapper
# Install lightx2v submodule dependencies
pip install -r lightx2v/requirements.txt
# Install ComfyUI wrapper dependencies
pip install -r requirements.txt
git submodule update --init --recursive
```

3. Download models and place them in `ComfyUI/models/lightx2v/` directory

## Node Overview

### Configuration Nodes

#### 1. LightX2V Inference Config

Basic inference configuration for video generation.

- **Inputs**: model, task_type, inference_steps, seed, cfg_scale, width, height, video_length, fps
- **Output**: Base configuration object

#### 2. LightX2V TeaCache

Feature caching acceleration configuration.

- **Inputs**: enable, threshold (0.0-1.0), use_ret_steps
- **Output**: TeaCache configuration
- **Note**: Lower threshold = more speedup (0.1 ~2x, 0.2 ~3x)

#### 3. LightX2V Quantization

Model quantization settings for memory efficiency.

- **Inputs**: dit_precision, t5_precision, clip_precision, backend, sensitive_layers_precision
- **Output**: Quantization configuration
- **Backends**: Auto-detected (vllm, sgl, q8f)

#### 4. LightX2V Memory Optimization

Memory management strategies.

- **Inputs**: optimization_level, attention_type, enable_rotary_chunking, cpu_offload, unload_after_generate
- **Output**: Memory optimization configuration

#### 5. LightX2V Lightweight VAE

VAE optimization options.

- **Inputs**: use_tiny_vae, use_tiling_vae
- **Output**: VAE configuration

#### 6. LightX2V LoRA Loader

Load and chain LoRA models.
Download the model files from [LightX2V on Hugging Face](https://huggingface.co/lightx2v)
and place them under `ComfyUI/models/lightx2v/`.

- **Inputs**: lora_name, strength (0.0-2.0), lora_chain (optional)
- **Output**: LoRA chain configuration
## Node groups

### Combination Node
- **Configuration:** `LightX2VInferenceConfig`, `LightX2VTeaCache`,
`LightX2VQuantization`, and `LightX2VMemoryOptimization`.
- **Composition:** `LightX2VConfigCombinerV2` and `LightX2VConfigCombinerV3`.
- **Inference:** `LightX2VModularInferenceV2`.
- **Talk inputs:** single-object, file-list, JSON, and multi-object talk
combiners.
- **Video inputs and restoration:** `LightX2VInputVideoPath`, SeedVR2 and
SwiftVR loaders/samplers, plus `LightX2VOutputVideoPreview`.
- **LoRA:** `LightX2VLoRALoader`.

#### 7. LightX2V Config Combiner
The inference node currently accepts the model classes exposed by
`model_utils.support_model_cls_list`: Wan 2.1/2.2 variants, SekoTalk,
Qwen Image, and SeedVR2. The retired `*_distill` values remain selectable as
compatibility aliases for saved ComfyUI workflows and are normalized before
LightX2V's runner is built.

Combines all configuration modules into a single configuration.
## Model files

- **Inputs**: All configuration types (optional)
- **Output**: Combined configuration object
Place each model in its own directory under `ComfyUI/models/lightx2v/`. LoRA
files can be placed in either `ComfyUI/models/lightx2v/loras/` or
`ComfyUI/models/loras/`.

### Inference Node

#### 8. LightX2V Modular Inference

Main inference node for video generation.

- **Inputs**: combined_config, prompt, negative_prompt, image (optional), audio (optional)
- **Outputs**: Generated video frames

## Usage Examples

### Basic T2V Workflow

1. Create LightX2V Inference Config (task_type: "t2v")
2. Use LightX2V Config Combiner
3. Connect to LightX2V Modular Inference with text prompt
4. Save video output

### I2V with Optimizations

1. Load input image
2. Create LightX2V Inference Config (task_type: "i2v")
3. Add LightX2V TeaCache (threshold: 0.26)
4. Add LightX2V Memory Optimization
5. Combine configs with LightX2V Config Combiner
6. Run LightX2V Modular Inference

### With LoRA

1. Create base configuration
2. Load LoRA with LightX2V LoRA Loader
3. Chain multiple LoRAs if needed
4. Combine all configs
5. Run inference

## Model Directory Structure

Download models from: <https://huggingface.co/lightx2v>

Models should be placed in:

```txt
```text
ComfyUI/models/lightx2v/
├── Wan2.1-I2V-14B-720P-xxx/ # Main model checkpoints
├── Wan2.1-I2V-14B-480P-xxx/ # Main model checkpoints
├── loras/ # LoRA models
├── Wan2.2-I2V-A14B-distill-fp8/
├── seedvr2_ema_3b_fp8/
└── loras/
```

## Tips
The exact directory names depend on the downloaded checkpoint. The model
picker only lists directories that exist on the local ComfyUI installation.

- Start with default settings and adjust based on your hardware
- Use TeaCache with threshold 0.1-0.2 for significant speedup
- Enable memory optimization if running on limited VRAM
- Quantization can reduce memory usage but may affect quality
- Chain multiple LoRAs for complex style combinations
## Development

## Troubleshooting

- **Out of Memory**: Enable memory optimization or use quantization
- **Slow Generation**: Enable TeaCache or reduce inference steps
- **Model Not Found**: Check model paths in `ComfyUI/models/lightx2v/`

## Example Workflows

Example workflow JSON files are provided in the `examples/` directory:

- `wan_i2v.json` - Basic image-to-video
- `wan_i2v_with_distill_lora.json` - I2V with distillation LoRA
- `wan_t2v_with_distill_lora.json` - T2V with distillation LoRA

## Contributing Guidelines

We welcome community contributions! Before submitting code, please ensure you follow these steps:

### Install Development Dependencies

```bash
pip install ruff pre-commit
```

### Code Quality Check

Before committing code, run the following command:
Install the development hook and run the same checks used by CI:

```bash
python -m pip install pre-commit
pre-commit install
pre-commit run --all-files
```

This will automatically check code formatting, syntax errors, and other code quality issues.

### Contribution Process

1. Fork this repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Create a Pull Request
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for pull request and workflow
validation guidelines.
Loading
Loading