Commit 8dea20a
committed
[KVCache][Scheduler] disable write_cache_to_storage* calls under cache manager v1
## Motivation
在 cache manager v1 下,KV cache 的存储回写由 v1 内部的 RadixTree 机制处理,
resource_manager_v1 中的 write_cache_to_storage / write_cache_to_storage_decode
调用属于冗余,应跳过。
## Modifications
- resource_manager_v1.py:preemption 路径的两处存储回写调用(decode/非decode)加上
`and not self.enable_cache_manager_v1` 条件,v1 下不再触发
- cache_manager/v1/cache_manager.py:prefix caching 未启用时,补充初始化
`request._match_result = MatchResult()`,避免后续访问空属性
## Usage or Command
启动服务时设置 `--enable-cache-manager-v1` 即可复现修复效果:
```bash
python -m fastdeploy.entrypoints.openai.api_server \
--enable-cache-manager-v1 \
...
```1 parent b50b6da commit 8dea20a
2 files changed
Lines changed: 3 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
503 | 503 | | |
504 | 504 | | |
505 | 505 | | |
| 506 | + | |
506 | 507 | | |
507 | 508 | | |
508 | 509 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
482 | 482 | | |
483 | 483 | | |
484 | 484 | | |
485 | | - | |
| 485 | + | |
486 | 486 | | |
487 | 487 | | |
488 | 488 | | |
489 | 489 | | |
490 | 490 | | |
491 | | - | |
| 491 | + | |
492 | 492 | | |
493 | 493 | | |
494 | 494 | | |
| |||
0 commit comments