Feat: USP sequence parallelism (Ulysses x Ring) via SequenceParallelPlan; Wan enabled#21

Open

zhihengy wants to merge 24 commits into

mainfrom

feat/wan_usp

Commits on Jul 10, 2026

diffusion: sequence-parallel config and Option B parallel state

zhihengy
and
claude
committed
diffusion: USP sequence parallelism for Wan training

zhihengy
and
claude
committed
diffusion: align engine grouping span with rollout, assert full coverage

zhihengy
and
claude
committed
diffusion: SP test suite (mesh, init smoke, parity, weight sync, import guard)

zhihengy
and
claude
committed
diffusion: SP guardrail runner + fp32 mode for grad-sync parity

zhihengy
and
claude
committed
diffusion: guardrail runner cleanup preamble + expert-selection override

zhihengy
and
claude
committed
diffusion: drive SP shard/gather hooks from the model's diffusers _cp_plan

zhihengy
and
claude
committed
diffusion: guardrail runner supports rollout-data save/replay for same-data band comparison

zhihengy
and
claude
committed
diffusion: own the differentiable USP operators, drop the sglang training dependency

zhihengy
and
claude
committed
diffusion: first-principles SP cleanup

zhihengy
and
claude
committed
diffusion: fsdp_shard_mode dp_sp — shard parameters over the dp x sp mesh

zhihengy
and
claude
committed
diffusion: materialize clip_grad_norm's DTensor before logging

zhihengy
and
claude
committed
diffusion: SequenceParallelPlan — per-family SP declaration behind ModelBackend

zhihengy
and
claude
committed
style: pre-commit formatting across sp files

zhihengy
and
claude
committed
diffusion: fail-fast SP validation — driver-side support check, wildcard and key-resolution guards

zhihengy
and
claude
committed
tests(sp): determinism smoke — SP attention path bitwise under deterministic mode

zhihengy
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feat: USP sequence parallelism (Ulysses x Ring) via SequenceParallelPlan; Wan enabled#21

Feat: USP sequence parallelism (Ulysses x Ring) via SequenceParallelPlan; Wan enabled#21
zhihengy wants to merge 24 commits into
mainfrom
feat/wan_usp

Commits on Jul 10, 2026

diffusion: sequence-parallel config and Option B parallel state

diffusion: USP sequence parallelism for Wan training

diffusion: align engine grouping span with rollout, assert full coverage

diffusion: SP test suite (mesh, init smoke, parity, weight sync, import guard)

diffusion: SP guardrail runner + fp32 mode for grad-sync parity

diffusion: guardrail runner cleanup preamble + expert-selection override

diffusion: drive SP shard/gather hooks from the model's diffusers _cp_plan

diffusion: guardrail runner supports rollout-data save/replay for same-data band comparison

diffusion: own the differentiable USP operators, drop the sglang training dependency

diffusion: first-principles SP cleanup

diffusion: fsdp_shard_mode dp_sp — shard parameters over the dp x sp mesh

diffusion: materialize clip_grad_norm's DTensor before logging

diffusion: SequenceParallelPlan — per-family SP declaration behind ModelBackend

style: pre-commit formatting across sp files

diffusion: fail-fast SP validation — driver-side support check, wildcard and key-resolution guards

tests(sp): determinism smoke — SP attention path bitwise under deterministic mode

Commits on Jul 13, 2026

diffusion: parameter placement is Option A unconditionally — Option B demoted to a test anchor

diffusion: ring_degree is derived, not configured — drop it from every input surface

docs(sp_mesh): note the sp_subgroups <-> locate_rank inverse relation

diffusion: default SP attention intercepts at diffusers' dispatch_attention_fn

Revert "diffusion: align engine grouping span with rollout, assert full coverage"

Revert the clip_grad_norm logging materialization (split to #35)

tests: keep the SP suite local — it lands in a separate test-infra PR

refactor: sp_plan.py owns the plan contract; sp_attention.py is dispatch-only