-
Notifications
You must be signed in to change notification settings - Fork 233
Support HunyuanImage-image3.0 for t2i #1250
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 9 commits
91400aa
c461254
23af5bb
911d90c
e98dfcb
14298d3
d47007f
b8d4601
6c5b537
efdfb62
b5da810
5b75701
e88143a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| { | ||
| "infer_steps": 50, | ||
| "sample_guide_scale": 5.0, | ||
| "flow_shift": 1.0, | ||
| "target_height": 1024, | ||
| "target_width": 1024, | ||
| "feature_caching": "NoCaching", | ||
| "enable_kv_cache": true, | ||
| "enable_text_kv_cache": true, | ||
| "use_taylor_cache": false, | ||
| "taylor_cache_interval": 5, | ||
| "taylor_cache_order": 2, | ||
| "taylor_cache_enable_first_enhance": false, | ||
| "taylor_cache_first_enhance_steps": 3, | ||
| "taylor_cache_enable_tailing_enhance": false, | ||
| "taylor_cache_tailing_enhance_steps": 1, | ||
| "taylor_cache_low_freqs_order": 2, | ||
| "taylor_cache_high_freqs_order": 2, | ||
| "pipeline_parallel": true, | ||
| "enable_cfg": true, | ||
| "hunyuan_cfg_mode": "parallel", | ||
| "parallel": { | ||
| "cfg_p_size": 2 | ||
| }, | ||
| "bot_task": "think_recaption", | ||
| "use_system_prompt": "en_unified", | ||
| "max_new_tokens": 2048, | ||
| "text_do_sample": true, | ||
| "text_top_k": 1024, | ||
| "text_top_p": 0.95, | ||
| "text_temperature": 0.6 | ||
| } | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,30 @@ | ||||||||||||||||||
| { | ||||||||||||||||||
| "infer_steps": 50, | ||||||||||||||||||
| "sample_guide_scale": 5.0, | ||||||||||||||||||
| "flow_shift": 1.0, | ||||||||||||||||||
| "target_height": 1024, | ||||||||||||||||||
| "target_width": 1024, | ||||||||||||||||||
| "feature_caching": "NoCaching", | ||||||||||||||||||
| "enable_kv_cache": true, | ||||||||||||||||||
| "enable_text_kv_cache": true, | ||||||||||||||||||
| "use_taylor_cache": false, | ||||||||||||||||||
| "pipeline_parallel": true, | ||||||||||||||||||
| "hunyuan_image3_pipeline_layout": "interleaved", | ||||||||||||||||||
| "enable_cfg": true, | ||||||||||||||||||
|
Comment on lines
+11
to
+13
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This configuration is intended for hybrid CFG and Sequence Parallel inference (using
Suggested change
|
||||||||||||||||||
| "hunyuan_cfg_mode": "parallel", | ||||||||||||||||||
| "moe_impl": "flashinfer", | ||||||||||||||||||
| "attn_impl": "flash_attention_2", | ||||||||||||||||||
| "flashinfer_autotune_mode": "tune", | ||||||||||||||||||
| "parallel": { | ||||||||||||||||||
| "cfg_p_size": 2, | ||||||||||||||||||
| "seq_p_size": 2, | ||||||||||||||||||
| "seq_p_attn_type": "kv_all_gather" | ||||||||||||||||||
| }, | ||||||||||||||||||
| "bot_task": "think_recaption", | ||||||||||||||||||
| "use_system_prompt": "en_unified", | ||||||||||||||||||
| "max_new_tokens": 2048, | ||||||||||||||||||
| "text_do_sample": true, | ||||||||||||||||||
| "text_top_k": 1024, | ||||||||||||||||||
| "text_top_p": 0.95, | ||||||||||||||||||
| "text_temperature": 0.6 | ||||||||||||||||||
| } | ||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,30 @@ | ||||||||||||||||||
| { | ||||||||||||||||||
| "infer_steps": 50, | ||||||||||||||||||
| "sample_guide_scale": 5.0, | ||||||||||||||||||
| "flow_shift": 1.0, | ||||||||||||||||||
| "target_height": 1024, | ||||||||||||||||||
| "target_width": 1024, | ||||||||||||||||||
| "feature_caching": "NoCaching", | ||||||||||||||||||
| "enable_kv_cache": true, | ||||||||||||||||||
| "enable_text_kv_cache": true, | ||||||||||||||||||
| "use_taylor_cache": false, | ||||||||||||||||||
| "pipeline_parallel": true, | ||||||||||||||||||
| "hunyuan_image3_pipeline_layout": "interleaved", | ||||||||||||||||||
| "enable_cfg": true, | ||||||||||||||||||
|
Comment on lines
+11
to
+13
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This configuration is intended for hybrid CFG and Sequence Parallel inference (using
Suggested change
|
||||||||||||||||||
| "hunyuan_cfg_mode": "parallel", | ||||||||||||||||||
| "moe_impl": "flashinfer", | ||||||||||||||||||
| "attn_impl": "sdpa", | ||||||||||||||||||
| "flashinfer_autotune_mode": "tune", | ||||||||||||||||||
| "parallel": { | ||||||||||||||||||
| "cfg_p_size": 2, | ||||||||||||||||||
| "seq_p_size": 2, | ||||||||||||||||||
| "seq_p_attn_type": "ulysses" | ||||||||||||||||||
| }, | ||||||||||||||||||
| "bot_task": "think_recaption", | ||||||||||||||||||
| "use_system_prompt": "en_unified", | ||||||||||||||||||
| "max_new_tokens": 2048, | ||||||||||||||||||
| "text_do_sample": true, | ||||||||||||||||||
| "text_top_k": 1024, | ||||||||||||||||||
| "text_top_p": 0.95, | ||||||||||||||||||
| "text_temperature": 0.6 | ||||||||||||||||||
| } | ||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,29 @@ | ||||||||||||
| { | ||||||||||||
| "infer_steps": 50, | ||||||||||||
| "sample_guide_scale": 5.0, | ||||||||||||
| "flow_shift": 1.0, | ||||||||||||
| "target_height": 1024, | ||||||||||||
| "target_width": 1024, | ||||||||||||
| "feature_caching": "NoCaching", | ||||||||||||
| "enable_kv_cache": true, | ||||||||||||
| "enable_text_kv_cache": true, | ||||||||||||
| "use_taylor_cache": false, | ||||||||||||
| "pipeline_parallel": true, | ||||||||||||
| "enable_cfg": true, | ||||||||||||
|
Comment on lines
+11
to
+12
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This configuration is intended for Sequence Parallel inference (using
Suggested change
|
||||||||||||
| "hunyuan_cfg_mode": "serial", | ||||||||||||
| "moe_impl": "flashinfer", | ||||||||||||
| "attn_impl": "sdpa", | ||||||||||||
| "flashinfer_autotune_mode": "tune", | ||||||||||||
| "parallel": { | ||||||||||||
| "cfg_p_size": 1, | ||||||||||||
| "seq_p_size": 2, | ||||||||||||
| "seq_p_attn_type": "kv_all_gather" | ||||||||||||
| }, | ||||||||||||
| "bot_task": "think_recaption", | ||||||||||||
| "use_system_prompt": "en_unified", | ||||||||||||
| "max_new_tokens": 2048, | ||||||||||||
| "text_do_sample": true, | ||||||||||||
| "text_top_k": 1024, | ||||||||||||
| "text_top_p": 0.95, | ||||||||||||
| "text_temperature": 0.6 | ||||||||||||
| } | ||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,29 @@ | ||||||||||||
| { | ||||||||||||
| "infer_steps": 50, | ||||||||||||
| "sample_guide_scale": 5.0, | ||||||||||||
| "flow_shift": 1.0, | ||||||||||||
| "target_height": 1024, | ||||||||||||
| "target_width": 1024, | ||||||||||||
| "feature_caching": "NoCaching", | ||||||||||||
| "enable_kv_cache": true, | ||||||||||||
| "enable_text_kv_cache": true, | ||||||||||||
| "use_taylor_cache": false, | ||||||||||||
| "pipeline_parallel": true, | ||||||||||||
| "enable_cfg": true, | ||||||||||||
|
Comment on lines
+11
to
+12
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This configuration is intended for Sequence Parallel inference (using
Suggested change
|
||||||||||||
| "hunyuan_cfg_mode": "serial", | ||||||||||||
| "moe_impl": "flashinfer", | ||||||||||||
| "attn_impl": "sdpa", | ||||||||||||
| "flashinfer_autotune_mode": "tune", | ||||||||||||
| "parallel": { | ||||||||||||
| "cfg_p_size": 1, | ||||||||||||
| "seq_p_size": 2, | ||||||||||||
| "seq_p_attn_type": "ulysses" | ||||||||||||
| }, | ||||||||||||
| "bot_task": "think_recaption", | ||||||||||||
| "use_system_prompt": "en_unified", | ||||||||||||
| "max_new_tokens": 2048, | ||||||||||||
| "text_do_sample": true, | ||||||||||||
| "text_top_k": 1024, | ||||||||||||
| "text_top_p": 0.95, | ||||||||||||
| "text_temperature": 0.6 | ||||||||||||
| } | ||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| { | ||
| "infer_steps": 50, | ||
| "sample_guide_scale": 5.0, | ||
| "flow_shift": 1.0, | ||
| "image_size": "auto", | ||
| "feature_caching": "NoCaching", | ||
| "enable_kv_cache": true, | ||
| "enable_text_kv_cache": true, | ||
| "use_taylor_cache": false, | ||
| "taylor_cache_interval": 5, | ||
| "taylor_cache_order": 2, | ||
| "taylor_cache_enable_first_enhance": false, | ||
| "taylor_cache_first_enhance_steps": 3, | ||
| "taylor_cache_enable_tailing_enhance": false, | ||
| "taylor_cache_tailing_enhance_steps": 1, | ||
| "taylor_cache_low_freqs_order": 2, | ||
| "taylor_cache_high_freqs_order": 2, | ||
| "pipeline_parallel": true, | ||
| "enable_cfg": true, | ||
| "infer_align_image_size": true, | ||
| "bot_task": "think_recaption", | ||
| "use_system_prompt": "en_unified", | ||
| "max_new_tokens": 2048, | ||
| "text_do_sample": true, | ||
| "text_top_k": 1024, | ||
| "text_top_p": 0.95, | ||
| "text_temperature": 0.6 | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| { | ||
| "infer_steps": 50, | ||
| "sample_guide_scale": 5.0, | ||
| "flow_shift": 1.0, | ||
| "target_height": 1024, | ||
| "target_width": 1024, | ||
| "feature_caching": "NoCaching", | ||
| "enable_kv_cache": true, | ||
| "enable_text_kv_cache": true, | ||
| "use_taylor_cache": false, | ||
| "taylor_cache_interval": 5, | ||
| "taylor_cache_order": 2, | ||
| "taylor_cache_enable_first_enhance": false, | ||
| "taylor_cache_first_enhance_steps": 3, | ||
| "taylor_cache_enable_tailing_enhance": false, | ||
| "taylor_cache_tailing_enhance_steps": 1, | ||
| "taylor_cache_low_freqs_order": 2, | ||
| "taylor_cache_high_freqs_order": 2, | ||
| "pipeline_parallel": true, | ||
| "enable_cfg": true, | ||
| "bot_task": "think_recaption", | ||
| "use_system_prompt": "en_unified", | ||
| "max_new_tokens": 2048, | ||
| "text_do_sample": true, | ||
| "text_top_k": 1024, | ||
| "text_top_p": 0.95, | ||
| "text_temperature": 0.6 | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Hunyuan Image3 Think Recaption Implementation Plan | ||
|
|
||
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. | ||
|
|
||
| **Goal:** Add a native LightX2V text-generation pre-stage for Hunyuan Image3 `recaption` and `think_recaption`, then feed generated COT text into the existing native image generation path. | ||
|
|
||
| **Architecture:** Keep the DiT path native. Add small runner-level helpers for text input preparation, autoregressive token generation without KV cache, stage transitions from `</think>` to `<recaption>`, and generated COT injection via tokenizer `batch_cot_text` for `gen_image`. | ||
|
|
||
| **Tech Stack:** Python, PyTorch, Hunyuan tokenizer/image processor helpers, LightX2V native HunyuanImage3 model/infer/weights. | ||
|
|
||
| --- | ||
|
|
||
| ### Task 1: Regression Tests | ||
|
|
||
| **Files:** | ||
| - Modify: `tests/test_hunyuan_image3_integration.py` | ||
|
|
||
| - [ ] Add tests for resolving bot task, generating COT before image inputs, and stage transition token forcing. | ||
| - [ ] Run targeted pytest and verify failures before implementation. | ||
|
|
||
| ### Task 2: Runner Text Generation Helpers | ||
|
|
||
| **Files:** | ||
| - Modify: `lightx2v/models/runners/hunyuan_image3/hunyuan_image3_runner.py` | ||
|
|
||
| - [ ] Add helpers for system prompt resolution, text template creation, next-token selection, and COT text generation. | ||
| - [ ] Preserve existing `bot_task=image` behavior. | ||
| - [ ] For `think_recaption`, force `<recaption>` after `</think>` and stop after `</recaption>`. | ||
|
|
||
| ### Task 3: Feed COT Into Image Generation | ||
|
|
||
| **Files:** | ||
| - Modify: `lightx2v/models/runners/hunyuan_image3/hunyuan_image3_runner.py` | ||
| - Modify: `configs/hunyuan_image3/hunyuan_image3_t2i.json` | ||
|
|
||
| - [ ] Pass `batch_cot_text` into `apply_chat_template(..., mode="gen_image")`. | ||
| - [ ] Add config keys for `bot_task`, `use_system_prompt`, `max_new_tokens`, and deterministic text generation. | ||
|
|
||
| ### Task 4: Verification | ||
|
|
||
| **Files:** | ||
| - Test: `tests/test_hunyuan_image3_integration.py` | ||
|
|
||
| - [ ] Run full Hunyuan integration tests. | ||
| - [ ] Run ruff and py_compile on touched files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The configuration specifies
"hunyuan_cfg_mode": "parallel"and a"cfg_p_size"of 2, but"cfg_parallel": trueis missing. Without this flag, CFG parallel inference will not be enabled.