Skip to content

feat(qwen-image): add a tiling option to the Qwen-Image VAE nodes - #9427

Open
Pfannkuchensack wants to merge 4 commits into
invoke-ai:mainfrom
Pfannkuchensack:feat/qwen_image_i2l_tiling
Open

feat(qwen-image): add a tiling option to the Qwen-Image VAE nodes#9427
Pfannkuchensack wants to merge 4 commits into
invoke-ai:mainfrom
Pfannkuchensack:feat/qwen_image_i2l_tiling

Conversation

@Pfannkuchensack

Copy link
Copy Markdown
Collaborator

Summary

Both Qwen-Image VAE nodes reserve working memory for a full-frame operation, which at high resolutions is more than a 24 GB card has — so the cache evicts everything else to honour it. On CUDA at 2560x1440:

Node reserved, untiled reserved, tiled (256px)
qwen_image_l2i (decode) 19.91 GiB 0.46 GiB
qwen_image_i2l (encode) 10.99 GiB 0.26 GiB

Tiling is the intended escape hatch, but today it does not work on either node:

  • i2l hardcoded vae.disable_tiling() — there was no way to enable it at all.
  • l2i honours the global force_tiled_decode, but computes its working-memory estimate before and independently of that flag. So tiling bounds the VAE while the cache still reserves the full-frame figure — the memory is never actually freed for anything else. Effectively inert.

This PR adds tiled / tile_size input fields to both nodes, following the SD/SDXL i2l/l2i nodes, OR'd with the global force_tiled_decode. Off by default, so behaviour is unchanged unless enabled.

estimate_vae_working_memory_qwen_image gains a matching tile_size parameter, and both nodes resolve tile_size=0 to the VAE's default (256px) before estimating. Tiled, it budgets one tile plus 25% overlap plus the resident RGB image, mirroring estimate_vae_working_memory_wan. Without this the change would be cosmetic on i2l and remain inert on l2i.

Measured end-to-end through the i2l node at 2560x1440: 10.99 → 0.26 GiB reserved, 9.26 → 0.17 GiB actual peak, identical latent shape. Tiled latents differ by ~1.4% relative L2 on noise input (worst case for tile blending; real images blend far better), which is why this stays opt-in.

Related Issues / Discussions

None filed — found while debugging VRAM exhaustion in a latents → image → upscale → image → latents workflow at 2560x1440 on a 24 GB card, where a ~12 GB transformer stays resident across the VAE round-trip.

QA Instructions

Reproducing the limit (no code needed): run a Qwen-Image / Krea-2 img2img round-trip at ~2560x1440 while a large transformer is resident. The VAE nodes request ~20 GB (decode) and ~11 GB (encode) of working memory, forcing the cache to evict the transformer; on a 24 GB card this surfaces as Loading 0.0 MB into VRAM, but only -N MB were requested and models loading at <100%.

With this PR:

  1. Enable tiled on the Latents to Image and/or Image to Latents - Qwen Image nodes (or set force_tiled_decode: true in invokeai.yaml for both).
  2. Re-run the workflow. The nodes should complete without evicting the transformer, and the VRAM warning should not appear.
  3. Compare against a tiled: false run. Images should be visually equivalent — tile seams are the failure mode to look for; none were observed on photographic content.
  4. Leave tiled off and confirm behaviour is identical to main. This is the default path and the most important check.
  5. tile_size: 0 uses the VAE default (256px). Larger values trade memory for fewer seams; reserved memory scales with tile_size² (512 → ~1 GiB, 256 → ~0.26 GiB at any resolution).

Automated: pytest tests/app/invocations/test_qwen_image_working_memory.py — includes a new case asserting the resolved tile size reaches the estimator, which is what makes the difference between working and inert tiling.

Not measured: the decode's runtime peak was not benchmarked separately — only its reservation, and the encode end-to-end. The decode constants themselves are unchanged from the existing calibration.

Merge Plan

Nothing special. Both node versions bumped 1.0.01.1.0; all new fields have defaults, so saved workflows load unchanged and keep current (untiled) behaviour. The new fields change the generated OpenAPI schema, so schema.ts needs regenerating as usual.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration — n/a, backend only
  • Documentation added / updated (if applicable) — n/a, field descriptions are inline
  • Updated What's New copy (if doing a release after this PR)

The Qwen-Image i2l node hardcoded vae.disable_tiling(), so a full-frame encode
was the only option. At 2560x1440 that peaks at 9.26 GiB — on top of a resident
multi-GB transformer, which is what makes an upscale round-trip run out of
headroom exactly at this node while every other node fits.

Adds `tiled` / `tile_size` input fields following the SD/SDXL i2l node, OR'd
with the global force_tiled_decode setting. Off by default, so behaviour is
unchanged unless enabled.

estimate_vae_working_memory_qwen_image gains a matching tile_size parameter.
Without it the change would be inert: the cache would keep reserving the
full-frame figure (10.99 GiB at 2560x1440) and evict models to honour it, no
matter what the VAE actually does. Tiled, it budgets one tile plus 25% overlap
plus the resident RGB image, mirroring estimate_vae_working_memory_wan.

Measured through the node at 2560x1440: 10.99 -> 0.26 GiB reserved, 9.26 -> 0.17
GiB actual peak, identical latent shape. Tiled latents differ by ~1.4% relative
L2 on noise input (worst case for tile blending; real images blend far better),
which is why this stays opt-in.
Both nodes reserve working memory for a full-frame operation, which at high
resolutions exceeds a 24 GB card, so the model cache evicts everything else to
honour it. On CUDA at 2560x1440: 19.91 GiB for the decode and 10.99 GiB for the
encode.

Tiling is the intended escape hatch, but it did not work on either node:

- qwen_image_i2l hardcoded vae.disable_tiling(), so it could not be enabled.
- qwen_image_l2i honoured the global force_tiled_decode, but computed its
  working-memory estimate before and independently of that flag. Tiling bounded
  the VAE while the cache still reserved the full-frame figure, so the memory was
  never freed for anything else — effectively inert.

Adds `tiled` / `tile_size` input fields to both nodes following the SD/SDXL
i2l/l2i nodes, OR'd with force_tiled_decode. Off by default; behaviour is
unchanged unless enabled.

estimate_vae_working_memory_qwen_image gains a matching tile_size parameter, and
both nodes resolve tile_size=0 to the VAE default (256px) before estimating.
Tiled it budgets one tile plus 25% overlap plus the resident RGB image,
mirroring estimate_vae_working_memory_wan. Without this the change would be
cosmetic on i2l and remain inert on l2i.

Measured through the i2l node at 2560x1440: 10.99 -> 0.26 GiB reserved,
9.26 -> 0.17 GiB actual peak, identical latent shape. Verified across eight
resolutions that tiled and untiled encodes produce the same latent dimensions.
Tiled latents differ by ~1.4% relative L2 on noise input (worst case for tile
blending), which is why this stays opt-in.

Also fixes a crash in qwen_image_i2l: `width`/`height` are `int | None`, but the
workflow UI sends 0 for an unset number input, and `0 is not None` reached
`image.resize((0, 0))` -> "height and width must be > 0". Non-positive values are
now treated as unset, matching how tile_size uses 0.
@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files frontend PRs that change frontend files python-tests PRs that change python tests labels Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14 Nice-to-Have 6.14.1 backend PRs that change backend files frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant