reduce sleep time in loops and cancel schedule threashold for prefill instance#7871
Conversation
|
Thanks for your contribution! |
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览当前 Required 失败 2 个（1 个覆盖率阈值失败、1 个等待 Approval），Required 无运行中/等待中任务；需处理覆盖率并完成人工审批后再合并。
2 任务状态汇总日志列说明：失败任务直接使用预生成 Job 链接；运行中任务如无 URL 则展示任务名。 2.1 Required任务 : 8/10 通过
2.2 可选任务 — 22/26 通过
3 失败详情（仅 required）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — 覆盖率不足（置信度: 高）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage
失败用例: 无。单元测试通过（ 根因详情: 关键日志: 修复建议:
修复建议摘要: 补测未覆盖行或申请覆盖率豁免 关联变更: |
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-05-21 11:22:00
📋 Review 摘要
PR 概述：针对 PD 分离场景调整各循环 sleep 时长、为 Prefill 实例取消调度块阈值、修复 block_num 负数问题、调整默认预填充准备线程数。
变更范围：engine/、engine/sched/、envs.py、output/、splitwise/
影响面 Tag：[PD Disaggregation] [Scheduler] [Engine]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| ❓ 疑问 | fastdeploy/engine/common_engine_prepare_mixin.py:251 |
PR 标题称"reduce sleep time"，但实际所有 sleep 均为增大（0.002→0.02 / 0.002→0.005 / 0.001→0.005） |
| 📝 PR 规范 | — | 目标分支为 release，标题缺少 [Cherry-Pick] 前缀及原 PR 号；标题无官方 Tag；所有描述 section 均为空 |
📝 PR 规范检查
PR 存在以下规范问题：
- 目标分支为
release/2.6，按规范标题须用[Cherry-Pick][Tag] 标题(#原PR号)格式，但当前标题既无[Cherry-Pick]也无官方 Tag。 - PR body 中 Motivation / Modifications / Usage or Command / Accuracy Tests 均为空（仅保留模板注释）。
标题建议（可直接复制）：
[Cherry-Pick][PD Disaggregation] Adjust sleep intervals and cancel schedule threshold for prefill instance(#<原PR号>)
PR 描述建议（可直接复制）：
## Motivation
针对 PD 分离（Prefill/Decode 解耦）场景的性能与稳定性调优：
1. 将各忙等循环的 sleep 时间适当调大，降低 CPU 无效轮询；
2. 为 Prefill 实例移除 decode 阶段调度块阈值限制，避免 running 请求被错误阻塞；
3. 修复 `get_new_block_nums` 中 `block_num` 可能返回负数的潜在 bug；
4. 将 `FD_PREFILL_PREPARE_REQ_THREAD_NUM` 默认线程数从 5 调整为 3，减少不必要线程开销。
## Modifications
- `fastdeploy/engine/common_engine_prepare_mixin.py`：`_fetch_loop` 中正常/异常路径 sleep 从 `0.002s` 调整为 `0.02s`
- `fastdeploy/engine/sched/resource_manager_v1.py`：
- `get_new_block_nums` 增加 `block_num = max(block_num, 0)` 防负数保护
- `_allocate_decode_and_extend` 中 Prefill 实例（`splitwise_role == "prefill"`）直接设 `can_schedule_block_num_threshold = 0`，跳过阈值计算
- `fastdeploy/envs.py`：`FD_PREFILL_PREPARE_REQ_THREAD_NUM` 默认值从 `5` 改为 `3`
- `fastdeploy/output/token_processor.py`：`_recycle_resources` 等待 KV cache 发送的 sleep 从 `0.002s` 调整为 `0.005s`
- `fastdeploy/splitwise/splitwise_connector.py`：`check_decode_allocated` 轮询 sleep 从 `0.001s` 调整为 `0.005s`
## Usage or Command
N/A
## Accuracy Tests
N/A
## Checklist
- [x] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [x] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [x] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.总体评价
代码逻辑改动合理：block_num 负数防护是有效的防御性修复，Prefill 实例跳过调度块阈值符合 PD 分离语义。需要作者确认 sleep 调整方向与 PR 标题是否一致（标题写"reduce"但实际均为增大），并补全 PR 描述以便后续追溯。
| self._pause_cond.wait_for(lambda: not self.is_paused) | ||
| fetch_fn() | ||
| time.sleep(0.002) | ||
| time.sleep(0.02) |
There was a problem hiding this comment.
❓ 疑问 PR 标题为 "reduce sleep time in loops"，但此处实际是将 0.002s 增大到 0.02s（10 倍）。token_processor.py（0.002→0.005）和 splitwise_connector.py（0.001→0.005）同样是增大。
请确认：标题是否有笔误（应为 "adjust/increase sleep time"）？还是存在本 PR 未体现的减少操作？
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/2.6 #7871 +/- ##
==============================================
Coverage ? 72.20%
==============================================
Files ? 381
Lines ? 54225
Branches ? 8474
==============================================
Hits ? 39153
Misses ? 12302
Partials ? 2770
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
3 participants
Motivation
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.