Skip to content

[DSV4] Fix Muon slice config empty layer offset#4758

Merged
huangjiyi merged 1 commit into
developfrom
codex/dsv4-empty-layer-muon-slice
Jul 10, 2026
Merged

[DSV4] Fix Muon slice config empty layer offset#4758
huangjiyi merged 1 commit into
developfrom
codex/dsv4-empty-layer-muon-slice

Conversation

@huangjiyi

@huangjiyi huangjiyi commented Jul 9, 2026

Copy link
Copy Markdown
Member

Description

  • 修复 DSV4 Muon slice config 的层号偏移:生成 model.layers.* 配置项时计入 num_empty_layers_add_in_head
  • compress_ratios 及其他逐层配置仍使用逻辑层号查询,避免空层偏移影响配置读取。

启用头部空层后,checkpoint 中的物理层号与 DSV4 逻辑层号不再一致。原实现生成的 slice config 未计入该偏移,可能导致 checkpoint 加载后使用错误的 tensor split metadata。

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已完成初轮审查。代码语义上新增的 empty layer 偏移和 DSV4 checkpoint 映射方向一致;当前需要阻塞的是 Lint 已失败,具体修复点已放在行级评论中。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment on lines +145 to +147
num_empty_layers_add_in_head = getattr(
config, "num_empty_layers_add_in_head", 0
) or 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 优先级:P0

问题: 当前新增的换行格式没有通过项目的 black 校验,PR 的 Lint check 已经失败,合入前必须修复。

证据/影响: CI 日志显示 blackpaddleformers/transformers/deepseek_v4/modeling.py 执行了 reformat;我用 .pre-commit-config.yaml 中固定的 black 22.8.0 复核,唯一 diff 是这里三行需要折叠为一行。若不修复,必需 CI 会继续失败。

期望处理: 请按 black 输出格式更新该赋值行,然后重新推送 commit。

处理要求:请针对该评论修复并提交新的 commit。

Suggested change
num_empty_layers_add_in_head = getattr(
config, "num_empty_layers_add_in_head", 0
) or 0
num_empty_layers_add_in_head = getattr(config, "num_empty_layers_add_in_head", 0) or 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 优先级:P0

当前提交还没有完全解决这个阻塞项:black 22.8.0 仍会重排这里,新的 Lint job 也仍然因为 paddleformers/transformers/deepseek_v4/modeling.py 被 black 修改而失败。

当前 head 的格式应改成下面这样再提交新的 commit:

        num_empty_layers_add_in_head = getattr(config, "num_empty_layers_add_in_head", 0)

处理要求:请针对该评论修复并提交新的 commit。

@Paddle-CI-Bot

Copy link
Copy Markdown

PaddleFormers Log Analysis

Run #29015350463 · Attempt 1

日志分析报告

流水线名称 问题标签 修复建议 日志片段
upload-coverage (job#86104644888) GitHub API 限流(429) Rerun 该 job,或升级 runner 上 git 至 ≥2.18 以避免走 REST API 下载仓库 报错代码

失败的测试case:

无测试用例失败。unittest-gpu-ci 本体已全部通过:
4239 passed, 319 skipped in 1468.18s

根本原因分析:

upload-coverage job 在 Checkout Code 步骤触发时,runner(ernie-cpu-06 / yq01-bfs-cs52)使用的 git 版本为 1.8.3.1(低于 2.18),actions/checkout@v5 降级为通过 GitHub REST API 下载仓库压缩包;连续三次下载均收到 429 Too Many Requests,达到重试上限后 checkout 失败,导致后续 Codecov 上传步骤全部被跳过,job 整体标记为 failure。

修复建议:

  1. 直接 rerun:在 GitHub Actions 页面对 upload-coverage job 单独 rerun,限流窗口过后通常可恢复。
  2. 根治:升级 ernie-cpu-06 runner 机器上的 git 到 ≥2.18,使 actions/checkout 走本地 git clone 而非 REST API,彻底规避 429 限流。
  3. 注意unittest-gpu-ci 本体测试全部通过,本 PR 代码无问题,无需修改代码。

🔍 准确性记录:请点击评论底部 😊 图标,选择 👍(准确)或 👎(有误),将自动记录到 CI 监控系统

🔄 每次 Re-run 后自动更新

@huangjiyi huangjiyi force-pushed the codex/dsv4-empty-layer-muon-slice branch from eaa39fe to ae303a5 Compare July 10, 2026 02:40

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查新的提交。上一轮的格式阻塞项仍未完全解决,当前 head 仍会被 black 重排且 Lint 仍失败;我已在原行级线程里补充了当前代码需要的修复形态。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Offset generated DSV4 layer keys by num_empty_layers_add_in_head so Muon slice configs target the physical checkpoint layer ids.

Co-authored-by: Codex <codex@openai.com>
@huangjiyi huangjiyi force-pushed the codex/dsv4-empty-layer-muon-slice branch from ae303a5 to 3a8c78a Compare July 10, 2026 03:19

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查当前提交,上一轮的格式阻塞项已经修复:black 22.8.0 本地校验通过,CI Lint 也已通过。当前改动看起来可以接受,剩余仍在运行的 CI 请以 GitHub Checks 结果为准。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@From00 From00 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@huangjiyi huangjiyi merged commit 463f1b2 into develop Jul 10, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants