[models] Upgrade Qwen3.5 implementation to Qwen3.8

[wwwjn]

Summary

• replace the qwen3_5 model package with a single qwen3_8 package
• support the released Qwen3.8-27B multimodal model and Qwen3.8-2.4T-A95B text-only MoE model
• remove all legacy Qwen3.5 model flavors and recipe configurations
• support both Hugging Face checkpoint layouts: model.language_model.* for the multimodal checkpoint and model.* for the text-only checkpoint
• make the vision encoder optional for the text-only MoE model
• rename Qwen tests, numerical utilities, integration recipes, and RL entry points
• explicitly exclude Qwen3.8-Flash-Next because it uses the separate qwen4_exp architecture

Hugging Face retains the Qwen3_5* implementation class and model-type names for these Qwen3.8 checkpoints, so those internal class names remain where they describe the upstream architecture contract.

Qwen3.8-27B numerical parity

Tested against Qwen/Qwen3.8-27B with Transformers 5.15 on an NVIDIA H100. The released 55.6 GB safetensor checkpoint was converted through the TorchTitan state-dict adapter to DCP without missing-key or shape errors.

FP16 multimodal, three deterministic image-text samples

• average KL divergence: 1.6405e-6
• cosine similarity: approximately 0.99997
• top-1 match: 100%
• top-5 match: 100%
• image preprocessing: all 589824 reconstructed pixel values matched, maximum difference 1.19e-7

Per-sample KL: 2.1510e-6, 4.8427e-7, 2.2861e-6.

Text-only FP16 isolation

• KL divergence: 4.3627e-7
• cosine similarity: 0.999966
• top-1/top-5 match: 100%

This isolates and validates the decoder architecture and HF weight conversion independently of the vision encoder.

BF16 multimodal, three deterministic samples

• average KL divergence: 1.9280e-4
• top-1/top-5 match: 100%

Tests

• uvx pre-commit run --all-files
  • all formatting, lint, docstring, spelling, and link hooks pass
  • repository-wide Pyrefly reports three pre-existing environment/API errors outside this change
• scoped Pyrefly check for the changed Qwen3.8 implementation and parity utilities: pass
• pytest -q tests/unit_tests/cpu/test_qwen3_8.py tests/unit_tests/cpu/test_no_new_cli_options.py::TestCliOptionsFrozen::test_every_model_is_guarded tests/unit_tests/cpu/components/data/test_qwen_multimodal_data.py::test_qwen38_recipe_geometry_matches_dataset_processor
  • 13 passed

Some existing varlen-attention tests require the separately installed FlashAttention-3 flash_attn_interface module.

[shuhuayu]

do we want to retire qwen 3.5 now or we can in addition support qwen 3.8 in the same folder? what are the major differences?

[tianyu-l]

it seems small models are gone -- what are best small models on the market, both dense and moe?

[wwwjn]

do we want to retire qwen 3.5 now or we can in addition support qwen 3.8 in the same folder? what are the major differences?

My current plan is to retire all qwen3.5 and just keep Qwen 3.8 model. However qwen3.8 only have 27B model as the smallest model.

Because the model architecture is the same, we can keep these small models in the config_regitry.py as well?

[tianyu-l]

Because the model architecture is the same, we can keep these small models in the config_regitry.py as well?

I think we should keep two folders one for 3.5 and one for 3.8.

• Most model code can be in one of them (e.g. we can put in 3.8 so that later we can remove 3.5 first)
• __init__.py file has the model_registry we need to keep in both.