[Cherry-Pick][Optimization] merge matmul and add(#6986) #7184
[Cherry-Pick][Optimization] merge matmul and add(#6986) #7184zoooo0820 merged 2 commits intoPaddlePaddle:release/2.5from
Conversation
|
Thanks for your contribution! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## release/2.5 #7184 +/- ##
==============================================
Coverage ? 69.19%
==============================================
Files ? 390
Lines ? 54384
Branches ? 8574
==============================================
Hits ? 37631
Misses ? 14041
Partials ? 2712
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
fastdeploy-bot
left a comment
fastdeploy-bot
left a comment
There was a problem hiding this comment.
🤖 AI Code Review |
2026-04-03 17:30 CST
📋 Review 摘要
PR 概述：将 UnquantizedLinearMethod.apply() 中的 matmul + add 替换为 paddle.nn.functional.linear，优化带 bias 场景的计算性能
变更范围：model_executor/layers/linear.py（核心优化）、tests/e2e/utils/（测试 baseline 路径更新）
影响面 Tag：OP
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🟡 建议 | rollout_routing_replay_test_utils.py:160 |
baseline 路径硬编码日期 0403，建议语义化命名 |
总体评价
核心优化逻辑正确，新增的 bias 维度 assert 检查增强了代码健壮性。测试文件的 baseline 路径变更建议改用更语义化的版本命名方式。
| if model_path: | ||
| baseline_path = os.path.join(model_path, f"R3_BaseLine_25_uint8/routing_replay_output_baseline_{model_name}") | ||
| baseline_path = os.path.join( | ||
| model_path, f"R3_BaseLine_dev_uint8_0403/routing_replay_output_baseline_{model_name}" |
There was a problem hiding this comment.
🟡 建议 baseline 路径中硬编码了日期 0403。
这种命名方式可能导致未来维护困难：
- 每次更新 baseline 都需要修改代码中的路径
- 日期含义不够清晰，
dev_uint8_0403中的0403指的是什么不明确
建议：
- 使用更语义化的版本命名，如
R3_BaseLine_v2.5_uint8 - 或者将 baseline 路径配置化，通过环境变量或配置文件管理
EmmonsCurse
changed the title
fastdeploy-bot
left a comment
fastdeploy-bot
left a comment
There was a problem hiding this comment.
🤖 AI Code Review |
2026-04-08 16:57 CST
📋 Review 摘要
PR 概述：将 UnquantizedLinearMethod 中的 matmul + add 融合为 paddle.nn.functional.linear 调用，提升推理性能。
变更范围：model_executor/layers/linear.py、测试文件
影响面 Tag：[Optimization] [OP]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🔴 Bug | linear.py:87 |
assert 语句在每次 forward 时执行，引入额外性能开销 |
总体评价
PR 的性能优化方向正确，使用 paddle.nn.functional.linear 融合 matmul 和 add 是合理的优化策略。但在热路径（apply 方法）中添加 assert 语句会在每次推理时执行验证，这违背了性能优化的初衷。建议将形状验证移至初始化或权重加载阶段。测试文件中的预期输出更新和 baseline 路径变更属于正常的维护更新。
| linear_out = paddle.add(linear_out, layer.bias) | ||
| return linear_out | ||
| bias = layer.bias | ||
| assert bias.dim() == 1 and bias.shape[-1] == layer.weight.shape[-1], ( |
There was a problem hiding this comment.
🔴 Bug 在 apply 方法中每次 forward 都执行 assert 语句检查 bias 形状，这会引入不必要的性能开销，违背了性能优化的初衷。
影响分析：
assert语句在每次调用forward_cuda时都会执行，增加了 Python 层的开销- bias 的形状在初始化时就已经确定，无需在每次调用时验证
- 这会部分抵消使用
paddle.nn.functional.linear带来的性能收益
建议修复方式：
将形状检查移到权重加载阶段，例如在 process_loaded_weights 中验证：
def process_loaded_weights(self, layer, weights) -> None:
if layer.weight.dtype != weights.dtype:
weights = weights.cast(layer.weight.dtype)
layer.weight.set_value(weights)
# 在这里验证 bias 和 weight 的形状匹配（仅在权重加载时执行一次）
if layer.with_bias:
assert layer.bias.dim() == 1 and layer.bias.shape[0] == layer.weight.shape[-1], (
f"bias must be 1D with size equal to the last dim of weight, "
f"but got bias.shape={layer.bias.shape}, weight.shape[-1]={layer.weight.shape[-1]}"
)这样验证只在模型加载时执行一次，而非每次推理调用。
4 participants
Motivation
性能优化
Modifications
将UnquantizedLinearMethod中的matmul和add用linear替换。
带bias情况基本上有加速，不带bias情况小shape下性能有下降（主要是python层if等调度开销，linear内部实现也是matmul）。
Usage or Command
无
Accuracy Tests
精度保持一致
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.