[Cherry-Pick][Feature] support decode unified attention(#7688)#7850
Conversation
|
Thanks for your contribution! |
lizhenyun01
force-pushed
the
dec_attn_0415
branch
from
727d30a to
b621b28
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-05-20 10:29:59
📋 Review 摘要
PR 概述：新增 Decode Unified Attention 支持（C16/动态/静态C8），适用于集中式（USE_DECODE_UNIFIED_ATTENTION=1）与 PD 分离（FD_ATTENTION_BACKEND=DECODE_UNIFIED_ATTN）两种场景。
变更范围：custom_ops/gpu_ops/（新增 CUDA kernels）、fastdeploy/model_executor/layers/attention/（新增 backend）、fastdeploy/worker/gpu_model_runner.py、fastdeploy/spec_decode/mtp.py、fastdeploy/envs.py、fastdeploy/platforms/cuda.py
影响面 Tag：[OP] [Feature] [Speculative Decoding]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🟡 建议 | custom_ops/setup_ops.py |
decode_unified_attention 仅在 cc>=90 下编译，但 PR 描述未说明硬件限制，A100 用户启用后会运行时报错 |
| 🟡 建议 | fastdeploy/model_executor/layers/attention/decode_unified_attention_backend.py |
DecodeUnifiedAttentionMetadata dataclass 中 _dtype 字段重复定义两次，应删除冗余定义 |
| ❓ 疑问 | fastdeploy/envs.py |
USE_DECODE_UNIFIED_ATTENTION 命名缺 FD_ 前缀；PR 描述中提到用此变量，但代码实际通过 FD_ATTENTION_BACKEND 控制，两者联动逻辑不明确 |
| ❓ 疑问 | fastdeploy/platforms/cuda.py |
ValueError 错误信息未包含 DECODE_UNIFIED_ATTN，会误导用户 |
| 📝 PR 规范 | PR 描述 | Modifications、Usage or Command、Accuracy Tests 三个 section 均为空占位符 |
📝 PR 规范检查
标题格式 [Cherry-Pick][Feature] support decode unified attention(#7688) 符合 Cherry-Pick 规范。但 PR 描述的 Modifications、Usage or Command、Accuracy Tests 三个 section 均为空（仅保留了注释占位符），checklist 各项均未勾选。
标题建议（合规，可直接使用）：
[Cherry-Pick][Feature] support decode unified attention(#7688)
PR 描述建议（可直接复制）：
## Motivation
C16/动态/静态C8 decode unified attention 支持，解决 decode 阶段 attention 计算的统一化问题。
- 集中式：flash_attn 开启情况下 `export USE_DECODE_UNIFIED_ATTENTION=1`（待确认与 `FD_ATTENTION_BACKEND` 的联动关系）
- PD 分离：D 节点 `export FD_ATTENTION_BACKEND=DECODE_UNIFIED_ATTN`
- 注意：当前仅支持 SM90+（Hopper，如 H100）及以上架构，最低需要 CUDA 12.0
## Modifications
- `custom_ops/gpu_ops/decode_unified_attention/`：新增 decode unified attention CUDA kernels，支持 C16（FP16/BF16）和 C8（FP8/INT8/动态量化）
- `custom_ops/gpu_ops/decoder_write_cache_with_rope.cu`：新增 decoder write cache with RoPE kernel
- `custom_ops/gpu_ops/decode_unified_attention/config_for_attention.cu`：新增 block/chunk 配置计算 kernel
- `custom_ops/setup_ops.py`：将上述 kernel 注册进编译流程（cc>=90 分支）
- `fastdeploy/model_executor/layers/attention/decode_unified_attention_backend.py`：新增 `DecodeUnifiedAttentionBackend` attention backend 实现
- `fastdeploy/model_executor/layers/attention/ops/`：新增对应 Python wrapper（decode_unified_attention、decoder_write_cache_with_rope、config_for_attention）
- `fastdeploy/worker/gpu_model_runner.py`：在 `FD_ATTENTION_BACKEND=DECODE_UNIFIED_ATTN` 时分配对应 buffer
- `fastdeploy/spec_decode/mtp.py`：适配 MTP 投机解码路径，复用 target_model 的 workspace tensor
- `fastdeploy/platforms/cuda.py`：注册新 backend 类型
- `fastdeploy/envs.py`：新增 `USE_DECODE_UNIFIED_ATTENTION` 环境变量
## Usage or Command
```bash
# 集中式（需开启 flash_attn）
export USE_DECODE_UNIFIED_ATTENTION=1
# PD 分离部署（D 节点）
export FD_ATTENTION_BACKEND=DECODE_UNIFIED_ATTN
```
## Accuracy Tests
N/A（Cherry-Pick 自 #7688，精度验证见原 PR）
## Checklist
- [x] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [x] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [x] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.总体评价
本 PR 实现了 Decode Unified Attention 功能（C16/C8），新增了约 2500 行 CUDA kernel 代码和对应的 Python 接入层，测试文件也有补充。主要建议：明确声明 SM90+ 硬件要求、厘清 USE_DECODE_UNIFIED_ATTENTION 与 FD_ATTENTION_BACKEND 的关系、补全 PR 描述中缺失的 section。
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览存在 3 个 required 任务失败，阻塞合并，需优先处理。
2 任务状态汇总2.1 Required任务 : 5/8 通过
2.2 可选任务 — 22/24 通过
3 失败详情（仅 required）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — 测试失败（置信度: 高）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage
失败用例:
根因详情: 关键日志: 修复建议:
修复建议摘要: 更新测试mock，添加use_fused_cast和fd_config字段 关联变更: 链接: 查看日志 xpu_4cards_case_test / run_xpu_4cards_cases — 测试失败（置信度: 中）xpu_4cards_case_test / run_xpu_4cards_cases
失败用例:
根因详情: 关键日志: 修复建议:
修复建议摘要: 检查XPU decode attention兼容性，下载artifact查服务日志 关联变更: 链接: 查看日志 xpu_8cards_case_test / run_xpu_8cards_cases — 测试失败（置信度: 中）xpu_8cards_case_test / run_xpu_8cards_cases
失败用例:
根因详情: 修复建议:
修复建议摘要: 检查gpu_model_runner中XPU PD分离attention初始化逻辑 关联变更: 链接: 查看日志 |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/online/20260415 #7850 +/- ##
==========================================================
Coverage ? 73.82%
==========================================================
Files ? 387
Lines ? 54014
Branches ? 8467
==========================================================
Hits ? 39875
Misses ? 11415
Partials ? 2724
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
freeliuzc
merged commit 15aed4f
into
PaddlePaddle:release/online/20260415
4 participants
Motivation
C16/动态/静态C8 decode unified attention支持，使用方式：
集中式：flash_attn开启情况下export USE_DECODE_UNIFIED_ATTENTION=1
PD分离：D节点export FD_ATTENTION_BACKEND=DECODE_UNIFIED_ATTN
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.