Skip to content

Commit 4f4ff8e

Browse files
chenshi5012dancingactor
authored andcommitted
[serve] Fix AttributeError when request_router is None in update_deployment_config (ray-project#63180)
## Description `AsyncioRouter.update_deployment_config()` unconditionally evaluates `len(self.request_router.curr_replicas)` at the end of the method. The `request_router` property performs lazy initialisation and returns `None` when `_request_router_class` is `None` and `_request_router` has not yet been initialised. In that state the call raises: ``` AttributeError: 'NoneType' object has no attribute 'curr_replicas' ``` **Trigger conditions** 1. `AsyncioRouter` is constructed without a `request_router_class` (the parameter is `Optional`), and the Controller's first long-poll config push arrives before any replica is assigned — the lazy-init path is never entered, so the property returns `None`. 2. A live deployment is hot-updated to remove a custom `request_router_class`; `update_deployment_config()` overwrites `self._request_router_class` with the new value before the guard `if self._request_router:` is evaluated, leaving both fields `None`. 3. Race condition: the Controller pushes a new config between construction and the first call to `assign_request()` that would trigger lazy init. **Fix** Cache the property result in a local variable and fall back to `0` when it is `None`. Zero is semantically correct: no replicas are active yet, so `MetricsManager` should not trigger a scaled-to-zero optimised push. ```python # before curr_num_replicas=len(self.request_router.curr_replicas), # after _router = self.request_router curr_num_replicas = len(_router.curr_replicas) if _router is not None else 0 ``` ## Related issues None — this is a self-contained bug fix. ## Checks - [x] I've signed off every commit with `Signed-off-by` - [x] I've run `scripts/format.sh` to lint the changes in this PR - [x] I've included any doc changes needed for this PR Signed-off-by: chenshi5012 <chenshi5012@163.com>
1 parent 4c94023 commit 4f4ff8e

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

python/ray/serve/_private/router.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -796,9 +796,15 @@ def update_deployment_config(self, deployment_config: DeploymentConfig):
796796
max_backoff_s=self._max_backoff_s,
797797
)
798798

799+
# Guard against the case where request_router is None (e.g., when
800+
# request_router_class is None and lazy initialization has not yet
801+
# occurred). In that scenario there are no active replicas yet, so
802+
# passing 0 is the correct and safe value.
803+
_router = self.request_router
804+
curr_num_replicas = len(_router.curr_replicas) if _router is not None else 0
799805
self._metrics_manager.update_deployment_config(
800806
deployment_config,
801-
curr_num_replicas=len(self.request_router.curr_replicas),
807+
curr_num_replicas=curr_num_replicas,
802808
)
803809

804810
async def _resolve_request_arguments(

0 commit comments

Comments
 (0)