You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/user_guide/ensemble_models.md
-1Lines changed: 0 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -242,7 +242,6 @@ The optimal value depends on your specific deployment, including batch size, req
242
242
no synchronization overhead is incurred.
243
243
***Minimal overhead when enabled**: Uses a blocking/wakeup mechanism per ensemble step, where upstream models are paused ("blocked") when the inflight requests limit is reached and resumed ("woken up") as downstream models complete processing them. This synchronization ensures memory usage stays within bounds, though it may increase latency.
244
244
245
-
246
245
**Note**: This blocking does not cancel or internally time out intermediate requests, but clients may experience increased end-to-end latency.
0 commit comments