Repository navigation
支持stablelm - #4482
支持stablelm#4482learncat163 wants to merge 9 commits into
Conversation
Codecov Report❌ Patch coverage is Please upload reports for the commit 7aced95 to get more accurate results. Additional details and impacted files@@ Coverage Diff @@
## develop #4482 +/- ##
==========================================
Coverage ? 46.66%
==========================================
Files ? 478
Lines ? 90750
Branches ? 0
==========================================
Hits ? 42345
Misses ? 48405
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PaddleFormers Log Analysis
日志分析报告
失败的测试case: 根本原因分析: 修复建议:
🔄 每次 Re-run 后自动更新 |
|
Thanks for your contribution! |
1 similar comment
|
Thanks for your contribution! |
|
/re-run all-failed |
|
/re-run all-failed |
|
/re-run all-failed |
liuhao2638
left a comment
liuhao2638
left a comment
There was a problem hiding this comment.
已完成代码 review，发现 StableLM 在多卡 tensor parallel / sequence parallel 路径上还有需要合入前修复的问题，具体已放在行级评论中。
当前 unittest-gpu-ci 失败停在 Change python version 阶段，未进入测试执行，建议 rerun 后再确认完整 CI。
| self.num_heads = config.num_attention_heads | ||
| self.num_key_value_heads = config.num_key_value_heads | ||
| self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads | ||
| self.head_dim = config.hidden_size // config.num_attention_heads | ||
|
|
||
| self.rotary_ndims = int(self.head_dim * config.rope_parameters["partial_rotary_factor"]) | ||
| self.scaling = self.head_dim**-0.5 | ||
| self.attention_dropout = config.attention_dropout | ||
|
|
||
| q_hidden_size = self.num_heads * self.head_dim | ||
| kv_hidden_size = self.num_key_value_heads * self.head_dim |
There was a problem hiding this comment.
开启 tensor_model_parallel_size > 1 时这里会用全局 head 数 reshape 已经被 ColumnParallelLinear 切分过的 q/k/v 输出。GeneralLinear.create(..., tp_plan="colwise") 在 TP 下默认 gather_output=False，每个 rank 只拿到 out_features / tp，但当前 self.num_heads/self.num_key_value_heads 仍是全局值，所以第一层 forward 会在 reshape([bsz, q_len, self.num_heads, self.head_dim]) 处形状不匹配；同类 Llama/Phi3 实现会先用全局维度建线性层，再把本地 head 数除以 TP degree。请同步这个本地 head 处理，或显式禁用 StableLM 的 TP。
| self.num_heads = config.num_attention_heads | |
| self.num_key_value_heads = config.num_key_value_heads | |
| self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads | |
| self.head_dim = config.hidden_size // config.num_attention_heads | |
| self.rotary_ndims = int(self.head_dim * config.rope_parameters["partial_rotary_factor"]) | |
| self.scaling = self.head_dim**-0.5 | |
| self.attention_dropout = config.attention_dropout | |
| q_hidden_size = self.num_heads * self.head_dim | |
| kv_hidden_size = self.num_key_value_heads * self.head_dim | |
| self.num_heads = config.num_attention_heads | |
| self.num_key_value_heads = config.num_key_value_heads | |
| self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads | |
| self.head_dim = config.hidden_size // config.num_attention_heads | |
| q_hidden_size = config.num_attention_heads * self.head_dim | |
| kv_hidden_size = config.num_key_value_heads * self.head_dim | |
| if config.tensor_model_parallel_size > 1: | |
| assert self.num_heads % config.tensor_model_parallel_size == 0, ( | |
| f"num_heads: {self.num_heads}, tensor_model_parallel_size: {config.tensor_model_parallel_size}" | |
| ) | |
| assert self.num_key_value_heads % config.tensor_model_parallel_size == 0, ( | |
| f"num_key_value_heads: {self.num_key_value_heads}, tensor_model_parallel_size: {config.tensor_model_parallel_size}" | |
| ) | |
| self.num_heads = self.num_heads // config.tensor_model_parallel_size | |
| self.num_key_value_heads = self.num_key_value_heads // config.tensor_model_parallel_size | |
| self.rotary_ndims = int(self.head_dim * config.rope_parameters["partial_rotary_factor"]) | |
| self.scaling = self.head_dim**-0.5 | |
| self.attention_dropout = config.attention_dropout |
| } | ||
| causal_mask, attn_mask_startend_row_indices = create_causal_mask_and_row_indices(**mask_kwargs) | ||
|
|
||
| hidden_states = inputs_embeds |
There was a problem hiding this comment.
StableLmDecoderLayer 已把 LayerNorm/Linear 标成 input_is_parallel=config.sequence_parallel，但 StableLmModel.forward 没有像 Llama/Qwen/Phi3 那样在 embedding 后执行 ScatterOp.apply。因此用户配置 sequence_parallel=True 时，后续 ColumnSequenceParallelLinear/RowSequenceParallelLinear 会按已切分的序列输入工作，而这里仍传入完整 [bsz, seq, hidden]，多卡训练/推理会得到错误的张量布局或通信形状错误。请在设置 hidden_states 前按现有模型模式 scatter，并同步 attention 中 sequence-parallel 的 batch/seq reshape。
可按下面的形状处理补齐（还需要在文件顶部引入 ScatterOp，并在 attention 输出后按 config.sequence_parallel reshape）：
from paddle.distributed.fleet.utils.sequence_parallel_utils import ScatterOp
...
if self.config.sequence_parallel:
inputs_embeds = inputs_embeds.reshape([-1, inputs_embeds.shape[-1]])
inputs_embeds = ScatterOp.apply(inputs_embeds)
hidden_states = inputs_embeds
|
/re-run all-failed |
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查新提交。StableLM 的 tensor parallel head 拆分和 sequence parallel scatter/reshape 已按现有模型模式补齐；当前已展示的检查为通过状态，未发现需要阻塞合入的问题。本次没有新增行级评论。
|
/re-run all-failed |
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查最新提交。此前 StableLM 的 TP/SP 代码问题在当前代码中仍保持已修复状态；这次新增阻塞点是当前 Lint 检查失败，black 会重写 paddleformers/transformers/stablelm/modeling.py。具体处理要求已放在行级评论中，请格式化后提交新的 commit。
|
|
||
| attention_factor = 1.0 | ||
|
|
||
| inv_freq = 1.0 / (base ** (paddle.arange(0, dim, 2, dtype=paddle.int64).astype(dtype=paddle.float32) / dim)) |
There was a problem hiding this comment.
处理要求：请针对该评论修复并提交新的 commit。
当前 Lint 检查失败，CI 日志显示 black 会重写 paddleformers/transformers/stablelm/modeling.py（reformatted paddleformers/transformers/stablelm/modeling.py）。这会阻塞合入；请按仓库的 Black 配置格式化该文件并提交格式化结果。
可直接执行：
python -m black paddleformers/transformers/stablelm/modeling.py
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查新的格式化提交。此前 P1 格式化问题已通过新 commit 修复，当前 Lint 检查已通过；本轮未发现新的阻塞问题。仍在运行中的 CI 请以最终结果为准。
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查最新提交。此次改动移除了会在普通 CI 中在线拉取 ModelScope tokenizer 的非 slow 用例，之前的 P1 格式化问题仍保持修复；当前未发现新的阻塞问题。仍在运行中的 CI 请以最终结果为准。
|
This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。 |
…ort-stablelm # Conflicts: # paddleformers/cli/utils/llm_utils.py # paddleformers/transformers/__init__.py # paddleformers/transformers/auto/configuration.py # paddleformers/transformers/auto/modeling.py
Paddle-Bot Review Board （review完成)
Powered by Nyanpasu claude with Opus 5 默认推理级别, please check the suggestions carefully. |
| self.norms = nn.LayerList([nn.LayerNorm(dim, epsilon=eps, bias_attr=bias) for _ in range(self.num_heads)]) | ||
|
|
||
| def forward(self, hidden_states: paddle.Tensor): | ||
| states_per_heads = paddle.split(hidden_states, 1, axis=1) |
There was a problem hiding this comment.
paddle.split 的第二个参数是「切分份数」，不是 torch.split 的「每份大小」。这里传 1 返回的是只含一个元素的列表（整个张量），于是 zip(self.norms, states_per_heads) 只迭代一次：所有 head 共用 self.norms[0] 做归一化，norms[1:] 完全不参与计算。由于输出 shape 仍为 [bsz, num_heads, seq_len, head_dim]，不会抛错，只会静默算错。
本地实测（Paddle 3.x）：
x = paddle.randn([2, 4, 3, 8])
len(paddle.split(x, 1, axis=1)) # 1
len(paddle.split(x, 4, axis=1)) # 4以 num_heads=4 构造该层并给每个 norm 设置不同 weight，当前实现与逐 head 归一化参考结果的最大绝对误差为 6.30。
该路径仅在 qk_layernorm=True 时启用（stablelm-2 系列即为该配置），而新增单测固定 qk_layernorm=False（tests/transformers/stablelm/test_modeling.py:92），因此前向/生成对齐测试都覆盖不到。建议按 head 份数切分：
| states_per_heads = paddle.split(hidden_states, 1, axis=1) | |
| states_per_heads = paddle.split(hidden_states, self.num_heads, axis=1) |
There was a problem hiding this comment.
已在当前 head 验证修复：paddle.split(hidden_states, self.num_heads, axis=1) 的切分份数已改为 head 数，CPU 实测 per-head 输出与逐 head 手工归一化参考结果一致（heads=1/2/4、bias=False/True 四组最大绝对误差 1.19e-07 ~ 9.54e-07）。额外新增的 qk_layernorm + 张量并行 ValueError 守卫也已覆盖该组合暂不支持的场景。该问题关闭。
| use_qkv_bias=False, | ||
| qk_layernorm=False, | ||
| use_parallel_residual=False, |
There was a problem hiding this comment.
get_config() 把 use_qkv_bias / qk_layernorm / use_parallel_residual 全部固定为 False，但本 PR 新增的三条条件分支——q/k/v bias、per-head QK LayerNorm（modeling.py:177-181）、parallel residual（modeling.py:304-307）以及 _gen_aoa_config 中对应的权重映射分支——只在这些开关为 True 时生效，当前没有任何用例进入。stablelm-2 系列正是 qk_layernorm=True，modeling.py:105 的静默算错也正因此未被发现。
建议补充参数化用例，至少覆盖 qk_layernorm=True 和 use_parallel_residual=True；并且不要只断言 shape——per-head LayerNorm 出错时输出 shape 不变，需要对数值做校验（例如与逐 head 手工归一化的参考结果对比，或 save_pretrained/from_pretrained 往返后前向一致性）。
There was a problem hiding this comment.
已在当前 head 验证修复：新增用例以 itertools.product 覆盖 use_qkv_bias/qk_layernorm/use_parallel_residual 全部 8 种组合，并包含 save_pretrained/from_pretrained 往返后的 state_dict 与 logits 数值一致性校验；另有 per-head LayerNorm 数值对比、两种 residual 公式、loss_mask 与 legacy rope_scaling 的专项用例。不再是仅断言 shape。该问题关闭。
| ) | ||
|
|
||
|
|
||
| class StableLmForCausalLM(StableLmPretrainedModel): |
There was a problem hiding this comment.
需要支持PP，增加StableLmForCausalLMPipe
|
|
||
|
|
||
| @register_base_model | ||
| class StableLmModel(StableLmPretrainedModel): |
| attentions=outputs.attentions, | ||
| ) | ||
|
|
||
| def auto_dist_config(self, prefix=""): |
| ("molmo", "MolmoConfig"), | ||
| ("qwen3_5", "Qwen3_5Config"), | ||
| ("qwen3_5_moe", "Qwen3_5MoEConfig"), | ||
| ("stablelm", "StableLmConfig"), |
There was a problem hiding this comment.
下面的MODEL_NAMES_MAPPING也需要注册对应内容
同时也需要在docs/zh/model_capability.md和仓库README补一下该模型相关的内容
| if past_key_values is not None: | ||
| key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx) | ||
|
|
||
| attention_interface = ALL_ATTENTION_FUNCTIONS["sdpa"] |
| hidden_states = inputs_embeds | ||
| position_embeddings = self.rotary_emb(hidden_states, position_ids) | ||
|
|
||
| all_hidden_states = () if output_hidden_states else None |
There was a problem hiding this comment.
建议参考其他模型，从config取值。下面CausalLM同理
| self.criterion = CriterionLayer(config) | ||
| self.tie_weights() | ||
|
|
||
| def forward( |
There was a problem hiding this comment.
建议补一下attention_mask 的 bool cast、attn_mask_start_row_indices 旧名兼容、两者同时传入时的 warning。可以参考olmo2
|
|
||
| class StableLmForCausalLM(StableLmPretrainedModel): | ||
| base_model_class = StableLmModel | ||
| _keys_to_ignore_on_load_missing = [r"lm_head.weight"] |
There was a problem hiding this comment.
官方权重tie_word_embeddings=False，这里最好不要忽略
| aoa_statements.extend( | ||
| [ | ||
| f"model.layers.$LAYER_ID.self_attn.q_layernorm.norms.*.weight -> {model_prefix}layers.$LAYER_ID.self_attn.q_layernorm.norms.*.weight", | ||
| f"model.layers.$LAYER_ID.self_attn.q_layernorm.norms.*.bias -> {model_prefix}layers.$LAYER_ID.self_attn.q_layernorm.norms.*.bias", |
There was a problem hiding this comment.
qk norm官方权重没有bias，可以删掉。下面同理
同样的test里面StableLmLayerNormPerHeadTest也不用增加bias为true的用例
| standardize_rope_params(self, rope_theta=self.rope_theta) | ||
| if self.partial_rotary_factor is not None and isinstance(self.rope_parameters, dict): | ||
| self.rope_parameters["partial_rotary_factor"] = self.partial_rotary_factor | ||
| rope_config_validation(self) |
There was a problem hiding this comment.
这里可以参考其他模型，加一个ignore_keys={"partial_rotary_factor"}
| ) -> tuple["paddle.Tensor", float]: | ||
| base = config.rope_parameters["rope_theta"] | ||
| partial_rotary_factor = config.rope_parameters.get("partial_rotary_factor", 1.0) | ||
| head_dim = getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads |
There was a problem hiding this comment.
这里优先用了config里面的head_dim，下面attention又变成了手动计算，可能会有不一致的结果。建议统一，或者参考其他模型在config里面给定
| self.config = config | ||
| self.layer_idx = layer_idx | ||
| if config.qk_layernorm and config.tensor_model_parallel_size > 1: | ||
| raise ValueError("StableLM per-head QK LayerNorm does not support tensor parallelism yet") |
There was a problem hiding this comment.
qk norm也可以支持TP，需要改一下AOA 的分片语义，可以参考cohere
| scaling=self.scaling, | ||
| ) | ||
|
|
||
| attn_output = attn_output.reshape([bsz, q_len, -1]) |
There was a problem hiding this comment.
attention_interface出来就已经是对应形状了吧，这个reshape还需要吗
|
|
||
| self.rope_theta = rope_theta | ||
| self.partial_rotary_factor = partial_rotary_factor | ||
| if rope_parameters is not None and not isinstance(rope_parameters, dict): |
There was a problem hiding this comment.
standardize_rope_params会检查dict，这里不需要了
| raise ValueError("rope_parameters and rope_scaling must be dicts or None") | ||
| self.rope_parameters = dict(rope_parameters) if rope_parameters is not None else None | ||
| standardize_rope_params(self, rope_theta=self.rope_theta) | ||
| if self.partial_rotary_factor is not None and isinstance(self.rope_parameters, dict): |
There was a problem hiding this comment.
standardize_rope_params会注入partial_rotary_factor，这里不需要了
| ], | ||
| "glm_ocr.processor": ["Glm46VProcessor"], | ||
| "glm_ocr.image_processor": ["Glm46VImageProcessor"], | ||
| "stablelm.configuration": ["StableLmConfig"], |
There was a problem hiding this comment.
上面可以再补一个"stablelm": []，风格对齐
support StableLM
1. 模型概述
StableLM 是 Stability AI 发布的开源语言模型系列。stable-code-3b 是专注于代码生成的 3B 参数模型。
本次 PR 将 StableLM 模型从 HuggingFace transformers 迁移到 PaddleFormers。
2. 权重信息
支持直接加载 HuggingFace 权重（从 HF cache 加载）。
也支持paddle版本的权重加载，权重地址：https://aistudio.baidu.com/modelsdetail/47974/
3. 对齐测试
3.1 前向对齐概要
使用 float32 精度对齐验证，所有指标满足 1e-2 阈值：
float32 下 diff < 1e-5，证明代码实现基本正确。bf16 下的 0.05 diff 来自累计误差，但是输出token是一致的
配套有对应的单元测试
tests/transformers/stablelm/test_modeling.py的test_diff_first_token_logits和test_diff_generated_tokens方法3.2 Token 生成一致性
使用原版代码加载原版权重，和Paddle代码加载原版权重，分别进行推理测试，进行输出token的对比。可以看到前30个token的结构都是完全一致的。
3 个 prompt，每一个的前30 个 token 全部一致：
3.3 Logits 精度
受限于模型大小和积累误差，float32 精度下可以实现对齐：
在 float32 模式下均满足精度对齐要求。
4. CE验证
目前没有多卡环境，只能运行单卡测试。全参数训练因 OOM (32GB 不足) 未完成，使用 LoRA 训练方式。
4.1 SFT LoRA训练
4.2 热启动测试
从 checkpoint-10 恢复训练到 step 12:
可以正常继续之前的训练并输出，loss 连续。
4.3 2轮loss对比
4.4 全参微调 OOM
受模型限制，32G的显存不足以全参微调，ms-swift和paddleformers-cli都无法运行。
4.5 paddle训练曲线（200步）
4.6 PaddleFormers 和 ms-swift LoRA Loss 对比
使用相同配置 (LoRA rank=8, alpha=16, lr=1e-4, batch=1, bf16, max_steps=200, GSM8K) 在两个框架下分别训练，前40步逐步对比（全参对比会OOM）：