Skip to content

Commit 935668f

Browse files
sufubaoclaude
andcommitted
fix: restore EMA update for output length prediction
During rebase, the RouterStatics.update() call was lost. This is needed by the scheduler for predicting request output length via EMA calculation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 7daa4e9 commit 935668f

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

lightllm/server/router/manager.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -352,6 +352,9 @@ def _filter_reqs_from_running_batch(self):
352352
cache_len=req.prompt_cache_len,
353353
mtp_accepted=req.mtp_accepted_token_num,
354354
)
355+
# Update EMA for output length prediction (used by scheduler)
356+
if not req.is_aborted:
357+
self.router_statics.update(req.candetoken_out_len)
355358
self.running_batch.filter_out_finished_req(self.shm_req_manager)
356359
if self.running_batch.is_clear():
357360
self.running_batch = None

0 commit comments

Comments
 (0)