diff --git a/README.md b/README.md index 9c60319..8e03cde 100644 --- a/README.md +++ b/README.md @@ -40,10 +40,10 @@ HunyuanVideo-1.5 is a video generation model that delivers top-tier quality with - +
- + @@ -67,7 +67,9 @@ HunyuanVideo-1.5 is a video generation model that delivers top-tier quality with If you develop/use HunyuanVideo-1.5 in your projects, welcome to let us know. -- **ComfyUI** - [ComfyUI](https://github.com/comfyanonymous/ComfyUI): A powerful and modular diffusion model GUI with a graph/nodes interface. ComfyUI supports HunyuanVideo-1.5 with various engineering optimizations for fast inference. +- **ComfyUI** - [ComfyUI](https://github.com/comfyanonymous/ComfyUI): A powerful and modular diffusion model GUI with a graph/nodes interface. ComfyUI supports HunyuanVideo-1.5 with various engineering optimizations for fast inference. We provide a [ComfyUI Usage Guide](./ComfyUI/README.md) for HunyuanVideo-1.5. + +- **Community-implemented ComfyUI Plugin** - [comfyui_hunyuanvideo_1.5_plugin](https://github.com/yuanyuan-spec/comfyui_hunyuanvideo_1.5_plugin): A community-implemented ComfyUI plugin for HunyuanVideo-1.5, offering both simplified and complete node sets for quick usage or deep workflow customization, with built-in automatic model download support. - **LightX2V** - [LightX2V](https://github.com/ModelTC/LightX2V): A lightweight and efficient video generation framework that integrates HunyuanVideo-1.5, supporting multiple engineering acceleration techniques for fast inference. @@ -95,6 +97,7 @@ If you develop/use HunyuanVideo-1.5 in your projects, welcome to let us know. - [Text to Video](#text-to-video) - [Image to Video](#image-to-video) - [Command Line Arguments](#command-line-arguments) + - [Optimal Inference Configurations](#optimal-inference-configurations) - [🧱 Models Cards](#-models-cards) - [🎬 More Examples](#-more-examples) - [📊 Evaluation](#-evaluation) @@ -157,8 +160,8 @@ pip install -i https://mirrors.tencent.com/pypi/simple/ --upgrade tencentcloud-s ### Step 3: Install Attention Libraries * Flash Attention: - It's recommended to install Flash Attention for faster inference and reduced GPU memory consumption. -Detailed installation instructions are available at [Flash Attention](https://github.com/Dao-AILab/flash-attention). + Install Flash Attention for faster inference and reduced GPU memory consumption. + Detailed installation instructions are available at [Flash Attention](https://github.com/Dao-AILab/flash-attention). * Flex-Block-Attention: flex-block-attn is only required for sparse attention to achieve faster inference and can be installed by the following command: @@ -169,6 +172,8 @@ Detailed installation instructions are available at [Flash Attention](https://gi ``` * SageAttention: + To enable SageAttention for faster inference, you need to install it by the following command: + > **Note**: Enabling SageAttention will automatically disable Flex-Block-Attention. ```bash git clone https://github.com/cooper1637/SageAttention.git cd SageAttention @@ -223,10 +228,11 @@ OUTPUT_PATH=./outputs/output.mp4 # Configuration N_INFERENCE_GPU=8 # Parallel inference GPU count CFG_DISTILLED=true # Inference with CFG distilled model, 2x speedup -SPARSE_ATTN=true # Inference with sparse attention +SPARSE_ATTN=false # Inference with sparse attention (only 720p models are equipped with sparse attention). Please ensure flex-block-attn is installed SAGE_ATTN=false # Inference with SageAttention +REWRITE=true # Enable prompt rewriting. Please ensure rewrite vLLM server is deployed and configured. +OVERLAP_GROUP_OFFLOADING=true # Only valid when group offloading is enabled, significantly increases CPU memory usage but speeds up inference MODEL_PATH=ckpts # Path to pretrained model -REWRITE=true # Enable prompt rewriting torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ --prompt "$PROMPT" \ @@ -239,10 +245,18 @@ torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ --use_sageattn $SAGE_ATTN \ --rewrite $REWRITE \ --output_path $OUTPUT_PATH \ + --overlap_group_offloading $OVERLAP_GROUP_OFFLOADING \ --save_pre_sr_video \ --model_path $MODEL_PATH ``` +> **Tips:** If your GPU memory is > 14GB but you encounter OOM (Out of Memory) errors during generation, you can try setting the following environment variable before running: +> ```bash +> export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:128 +> ``` + + + ### Command Line Arguments | Argument | Type | Required | Default | Description | @@ -264,6 +278,7 @@ torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ | `--sparse_attn` | bool | No | `false` | Enable sparse attention for faster inference (~1.5-2x speedup, requires H-series GPUs, auto-enables CFG distilled, use `--sparse_attn` or `--sparse_attn true` to enable) | | `--offloading` | bool | No | `true` | Enable CPU offloading (use `--offloading false` or `--offloading 0` to disable for faster inference if GPU memory allows) | | `--group_offloading` | bool | No | `None` | Enable group offloading (default: None, automatically enabled if offloading is enabled. Use `--group_offloading` or `--group_offloading true/1` to enable, `--group_offloading false/0` to disable) | +| `--overlap_group_offloading` | bool | No | `true` | Enable overlap group offloading (default: true). Significantly increases CPU memory usage but speeds up inference. Use `--overlap_group_offloading` or `--overlap_group_offloading true/1` to enable, `--overlap_group_offloading false/0` to disable | | `--dtype` | str | No | `bf16` | Data type for transformer: `bf16` (faster, lower memory) or `fp32` (better quality, slower, higher memory) | | `--use_sageattn` | bool | No | `false` | Enable SageAttention (use `--use_sageattn` or `--use_sageattn true/1` to enable, `--use_sageattn false/0` to disable) | | `--sage_blocks_range` | str | No | `0-53` | SageAttention blocks range (e.g., `0-5` or `0,1,2,3,4,5`) | @@ -271,22 +286,43 @@ torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ **Note:** Use `--nproc_per_node` to specify the number of GPUs. For example, `--nproc_per_node=8` uses 8 GPUs. +### Optimal Inference Configurations + +The following table provides the optimal inference configurations (CFG scale, embedded CFG scale, flow shift, and inference steps) for each model to achieve the best generation quality: + +| Model | CFG Scale | Embedded CFG Scale | Flow Shift | Inference Steps | +|-------|-----------|-------------------|------------|-----------------| +| 480p T2V | 6 | None | 5 | 50 | +| 480p I2V | 6 | None | 5 | 50 | +| 720p T2V | 6 | None | 9 | 50 | +| 720p I2V | 6 | None | 7 | 50 | +| 480p T2V CFG Distilled | 1 | None | 5 | 50 | +| 480p I2V CFG Distilled | 1 | None | 5 | 50 | +| 720p T2V CFG Distilled | 1 | None | 9 | 50 | +| 720p I2V CFG Distilled | 1 | None | 7 | 50 | +| 720p T2V CFG Distilled Sparse | 1 | None | 7 | 50 | +| 720p I2V CFG Distilled Sparse | 1 | None | 9 | 50 | +| 480→720 SR Step Distilled | 1 | None | 2 | 6 | +| 720→1080 SR Step Distilled | 1 | None | 2 | 8 | + +**Please note that the cfg distilled model we provided, must use 50 steps to generate correct results.** + ## 🧱 Models Cards |ModelName| Download | |-|---------------------------| |HunyuanVideo-1.5-480P-T2V|[480P-T2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_t2v) | |HunyuanVideo-1.5-480P-I2V |[480P-I2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_i2v) | -|HunyuanVideo-1.5-480P-T2V-distill | [480P-T2V-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_t2v_distilled) | -|HunyuanVideo-1.5-480P-I2V-distill |[480P-I2V-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_i2v_distilled) | +|HunyuanVideo-1.5-480P-T2V-cfg-distill | [480P-T2V-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_t2v_distilled) | +|HunyuanVideo-1.5-480P-I2V-cfg-distill |[480P-I2V-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_i2v_distilled) | |HunyuanVideo-1.5-720P-T2V|[720P-T2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_t2v) | |HunyuanVideo-1.5-720P-I2V |[720P-I2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v) | -|HunyuanVideo-1.5-720P-T2V-distiill| Comming soon | -|HunyuanVideo-1.5-720P-I2V-distiill |[720P-I2V-distiill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled) | -|HunyuanVideo-1.5-720P-T2V-sparse-distiill| Comming soon | -|HunyuanVideo-1.5-720P-I2V-sparse-distiill |[720P-I2V-sparse-distiill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled_sparse) | -|HunyuanVideo-1.5-720P-sr |[720P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_sr_distilled) | -|HunyuanVideo-1.5-1080P-sr |[1080P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/1080p_sr_distilled) | +|HunyuanVideo-1.5-720P-T2V-cfg-distill| Comming soon | +|HunyuanVideo-1.5-720P-I2V-cfg-distill |[720P-I2V-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled) | +|HunyuanVideo-1.5-720P-T2V-sparse-cfg-distill| Comming soon | +|HunyuanVideo-1.5-720P-I2V-sparse-cfg-distill |[720P-I2V-sparse-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled_sparse) | +|HunyuanVideo-1.5-720P-sr-step-distill |[720P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_sr_distilled) | +|HunyuanVideo-1.5-1080P-sr-step-distill |[1080P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/1080p_sr_distilled) | diff --git a/README_CN.md b/README_CN.md index 96aebf6..b17e1b0 100644 --- a/README_CN.md +++ b/README_CN.md @@ -24,10 +24,10 @@ HunyuanVideo-1.5作为一款轻量级视频生成模型，仅需83亿参数即 - +
- + @@ -51,6 +51,8 @@ HunyuanVideo-1.5作为一款轻量级视频生成模型，仅需83亿参数即 如果您在项目中使用或开发了 HunyuanVideo-1.5，欢迎告知我们。 - **ComfyUI** - [ComfyUI](https://github.com/comfyanonymous/ComfyUI): 一个强大且模块化的扩散模型图形界面，采用节点式工作流。ComfyUI 支持 HunyuanVideo-1.5，并提供多种工程加速优化以实现快速推理。 +我们提供了一个 [ComfyUI 使用指南](./ComfyUI/README.md) 用于 HunyuanVideo-1.5。 +- **社区实现的 ComfyUI 插件** - [comfyui_hunyuanvideo_1.5_plugin](https://github.com/yuanyuan-spec/comfyui_hunyuanvideo_1.5_plugin): 社区实现的 HunyuanVideo-1.5 ComfyUI 插件，提供简化版和完整版节点集，支持快速使用或深度工作流定制，内置自动模型下载功能。 - **LightX2V** - [LightX2V](https://github.com/ModelTC/LightX2V): 一个轻量级高效的视频生成框架，集成了 HunyuanVideo-1.5，支持多种工程加速技术以实现快速推理。 @@ -77,6 +79,7 @@ HunyuanVideo-1.5作为一款轻量级视频生成模型，仅需83亿参数即 - [🔑 使用方法](#-使用方法) - [视频生成](#视频生成) - [命令行参数](#命令行参数) + - [最优推理配置](#最优推理配置) - [🧱 模型卡片](#-模型卡片) - [🎬 更多示例](#-更多示例) - [📊 性能评估](#-性能评估) @@ -140,7 +143,7 @@ pip install -i https://mirrors.tencent.com/pypi/simple/ --upgrade tencentcloud-s ### 步骤 3：安装注意力库 * Flash Attention: - 建议安装 Flash Attention 以实现更快的推理速度和更低的 GPU 内存消耗。 + 安装 Flash Attention 以实现更快的推理速度和更低的 GPU 内存消耗。 详细安装说明请参考 [Flash Attention](https://github.com/Dao-AILab/flash-attention)。 * Flex-Block-Attention: @@ -152,7 +155,8 @@ pip install -i https://mirrors.tencent.com/pypi/simple/ --upgrade tencentcloud-s ``` * SageAttention: - + 要启用 SageAttention 以实现更快的推理，您需要通过以下命令安装： + > **注意**: 启用 SageAttention 将自动禁用 Flex-Block-Attention。 ```bash git clone https://github.com/cooper1637/SageAttention.git cd SageAttention @@ -211,10 +215,11 @@ OUTPUT_PATH=./outputs/output.mp4 # 配置 N_INFERENCE_GPU=8 # 并行推理 GPU 数量 CFG_DISTILLED=true # 使用 CFG 蒸馏模型进行推理，2倍加速 -SPARSE_ATTN=true # 使用稀疏注意力进行推理 +SPARSE_ATTN=false # 使用稀疏注意力进行推理（仅 720p 模型配备了稀疏注意力）。请确保 flex-block-attn 已安装 SAGE_ATTN=false # 使用 SageAttention 进行推理 +REWRITE=true # 启用提示词重写。请确保 rewrite vLLM server 已部署和配置。 +OVERLAP_GROUP_OFFLOADING=true # 仅在组卸载启用时有效，会显著增加 CPU 内存占用，但能够提速 MODEL_PATH=ckpts # 预训练模型路径 -REWRITE=true # 启用提示词重写 torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ --prompt "$PROMPT" \ @@ -227,10 +232,16 @@ torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ --use_sageattn $SAGE_ATTN \ --rewrite $REWRITE \ --output_path $OUTPUT_PATH \ + --overlap_group_offloading $OVERLAP_GROUP_OFFLOADING \ --save_pre_sr_video \ --model_path $MODEL_PATH ``` +> **Tips:** 如果您的 GPU 内存 > 14GB 但您在生成过程中遇到 OOM (Out of Memory) 错误，可以尝试在运行前设置以下环境变量： +> ```bash +> export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:128 +> ``` + ### 命令行参数 | 参数 | 类型 | 是否必需 | 默认值 | 描述 | @@ -252,6 +263,7 @@ torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ | `--sparse_attn` | bool | 否 | `false` | 启用稀疏注意力以加速推理（约 1.5-2 倍加速，需要 H 系列 GPU，会自动启用 CFG 蒸馏，使用 `--sparse_attn` 或 `--sparse_attn true` 来启用） | | `--offloading` | bool | 否 | `true` | 启用 CPU 卸载（使用 `--offloading false` 或 `--offloading 0` 来禁用，如果 GPU 内存允许，禁用后速度会更快） | | `--group_offloading` | bool | 否 | `None` | 启用组卸载（默认：None，如果启用了 offloading 则自动启用。使用 `--group_offloading` 或 `--group_offloading true/1` 来启用，`--group_offloading false/0` 来禁用） | +| `--overlap_group_offloading` | bool | 否 | `true` | 启用重叠组卸载（默认：true）。会显著增加 CPU 内存占用，但能够提速。使用 `--overlap_group_offloading` 或 `--overlap_group_offloading true/1` 来启用，`--overlap_group_offloading false/0` 来禁用 | | `--dtype` | str | 否 | `bf16` | Transformer 的数据类型：`bf16`（更快，内存占用更低）或 `fp32`（质量更好，速度更慢，内存占用更高） | | `--use_sageattn` | bool | 否 | `false` | 启用 SageAttention（使用 `--use_sageattn` 或 `--use_sageattn true/1` 来启用，`--use_sageattn false/0` 来禁用） | | `--sage_blocks_range` | str | 否 | `0-53` | SageAttention 块范围（例如：`0-5` 或 `0,1,2,3,4,5`） | @@ -259,22 +271,43 @@ torchrun --nproc_per_node=$N_INFERENCE_GPU generate.py \ **注意：** 使用 `--nproc_per_node` 指定使用的 GPU 数量。例如，`--nproc_per_node=8` 表示使用 8 个 GPU。 +### 最优推理配置 + +下表提供了每个模型的最优推理配置（CFG 缩放、嵌入 CFG 缩放、流偏移和推理步数），以获得最佳生成质量： + +| 模型 | CFG 缩放 | 嵌入 CFG 缩放 | 流偏移 | 推理步数 | +|-------|-----------|-------------------|------------|-----------------| +| 480p T2V | 6 | None | 5 | 50 | +| 480p I2V | 6 | None | 5 | 50 | +| 720p T2V | 6 | None | 9 | 50 | +| 720p I2V | 6 | None | 7 | 50 | +| 480p T2V cfg 蒸馏 | 1 | None | 5 | 50 | +| 480p I2V cfg 蒸馏 | 1 | None | 5 | 50 | +| 720p T2V cfg 蒸馏 | 1 | None | 9 | 50 | +| 720p I2V cfg 蒸馏 | 1 | None | 7 | 50 | +| 720p T2V cfg 蒸馏稀疏 | 1 | None | 7 | 50 | +| 720p I2V cfg 蒸馏稀疏 | 1 | None | 9 | 50 | +| 480→720 超分 步数蒸馏 | 1 | None | 2 | 6 | +| 720→1080 超分 步数蒸馏 | 1 | None | 2 | 8 | + +**请注意我们提供的cfg蒸馏模型，需要50步的推理步数来获得正确的结果.** + ## 🧱 模型卡片 |模型名称| 下载链接 | |-|---------------------------| |HunyuanVideo-1.5-480P-T2V|[480P-T2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_t2v) | |HunyuanVideo-1.5-480P-I2V |[480P-I2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_i2v) | -|HunyuanVideo-1.5-480P-T2V-distill | [480P-T2V-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_t2v_distilled) | -|HunyuanVideo-1.5-480P-I2V-distill |[480P-I2V-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_i2v_distilled) | +|HunyuanVideo-1.5-480P-T2V-cfg-distill | [480P-T2V-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_t2v_distilled) | +|HunyuanVideo-1.5-480P-I2V-cfg-distill |[480P-I2V-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/480p_i2v_distilled) | |HunyuanVideo-1.5-720P-T2V|[720P-T2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_t2v) | |HunyuanVideo-1.5-720P-I2V |[720P-I2V](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v) | -|HunyuanVideo-1.5-720P-T2V-distiill| Comming soon | -|HunyuanVideo-1.5-720P-I2V-distiill |[720P-I2V-distiill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled) | -|HunyuanVideo-1.5-720P-T2V-sparse-distiill| Comming soon | -|HunyuanVideo-1.5-720P-I2V-sparse-distiill |[720P-I2V-sparse-distiill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled_sparse) | -|HunyuanVideo-1.5-720P-sr |[720P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_sr_distilled) | -|HunyuanVideo-1.5-1080P-sr |[1080P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/1080p_sr_distilled) | +|HunyuanVideo-1.5-720P-T2V-cfg-distill| Comming soon | +|HunyuanVideo-1.5-720P-I2V-cfg-distill |[720P-I2V-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled) | +|HunyuanVideo-1.5-720P-T2V-sparse-cfg-distill| Comming soon | +|HunyuanVideo-1.5-720P-I2V-sparse-cfg-distill |[720P-I2V-sparse-cfg-distill](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_i2v_distilled_sparse) | +|HunyuanVideo-1.5-720P-sr-step-distill |[720P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/720p_sr_distilled) | +|HunyuanVideo-1.5-1080P-sr-step-distill |[1080P-sr](https://huggingface.co/tencent/HunyuanVideo-1.5/tree/main/transformer/1080p_sr_distilled) |