Commit b8b6670
committed
[None][fix] Guard changeBeamWidth on mInflightReqIds.empty() in error handler
In pipeline-parallel multi-micro-batch mode, requests from other micro-batches
may still be tracked in mInflightReqIds after the error handler erases only the
current activeRequests. changeBeamWidth asserts mInflightReqIds.empty(), so
calling it unconditionally would throw (caught by the inner try-catch) and skip
the intended buffer reset.
Add an explicit guard so the reset is skipped when other micro-batches are still
in-flight. The next successful forwardAsync iteration will perform the reset via
the normal changeBeamWidth call in verifyRequests once the set is clear. This
makes the skip explicit rather than relying on TLLM_CHECK as control flow.
Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>1 parent 1cfdc82 commit b8b6670
1 file changed
Lines changed: 5 additions & 2 deletions
Lines changed: 5 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1223 | 1223 | | |
1224 | 1224 | | |
1225 | 1225 | | |
1226 | | - | |
1227 | | - | |
| 1226 | + | |
| 1227 | + | |
| 1228 | + | |
| 1229 | + | |
| 1230 | + | |
1228 | 1231 | | |
1229 | 1232 | | |
1230 | 1233 | | |
| |||
0 commit comments