Skip to content

Commit 316a64b

Browse files
authored
Re-pin deployment desired_worker to current model_version on serve (#1024)
* Re-pin deployment desired_worker to current model_version on serve When `clarifai model serve` reuses an existing deployment, the patch silently failed for two reasons: 1. PatchDeployments only accepts action='overwrite' — 'merge' is rejected by the backend ("Invalid action: Unrecognized action 'merge'. Supported actions: overwrite"). The CLI used 'merge', and the failure was swallowed by `except Exception: pass`, so even the visibility patch this code claimed to do was never landing. 2. `Deployment.worker` is the *observed* state (read-only on input); writes to it are silently ignored by the backend. The writable input is `Deployment.desired_worker`. With deploy_latest_version=False (set in #1022 for serve deployments), the deployment is pinned to whatever desired_worker.model.model_version.id it has. When the method-signatures hash changes a new model_version and runner are created, but the deployment's desired_worker stayed on the old version_id — leaving API calls routed to a version with no live runner ("Model is still deploying..."). Switch action to 'overwrite', patch desired_worker (not worker), and surface non-success responses + exceptions as warnings instead of swallowing them. Idempotent when the version is unchanged; heals the binding when it changed. * linter
1 parent 4493886 commit 316a64b

1 file changed

Lines changed: 36 additions & 6 deletions

File tree

clarifai/cli/model.py

Lines changed: 36 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1854,6 +1854,7 @@ def serve_cmd(
18541854
return
18551855

18561856
from clarifai_grpc.grpc.api import resources_pb2, service_pb2
1857+
from clarifai_grpc.grpc.api.status import status_code_pb2
18571858

18581859
from clarifai.client.user import User
18591860
from clarifai.runners.models import deploy_output as out
@@ -2224,25 +2225,54 @@ def _get_saved(key):
22242225
click.echo("done")
22252226

22262227
# 8. Deployment — reuse if exists, create if not.
2227-
# deploy_latest_version=True means it auto-routes to new versions.
2228+
#
2229+
# Three things to know about PatchDeployments:
2230+
# 1. The backend only accepts action='overwrite' — 'merge' returns
2231+
# "Invalid action: Unrecognized action 'merge'".
2232+
# 2. `worker` is the *observed* state (read-only on input). Writes to
2233+
# it are silently ignored. The writable input is `desired_worker`.
2234+
# 3. With `deploy_latest_version=False` (set in #1022 for serve
2235+
# deployments), the deployment is pinned to whatever
2236+
# `desired_worker.model.model_version.id` it has.
2237+
#
2238+
# When the method-signatures hash changes, a new model_version is
2239+
# created and the runner is rebound — but unless we re-patch
2240+
# `desired_worker` here, the deployment stays pinned to the old
2241+
# version_id and API calls route to a version with no live runner
2242+
# ("Model is still deploying..."). Re-patching is a no-op when the
2243+
# version is unchanged and the correct heal action when it changed.
22282244
deployment_exists = False
22292245
try:
22302246
nodepool.deployment(deployment_id)
2231-
# Patch visibility to match --public flag
22322247
try:
22332248
patch_req = service_pb2.PatchDeploymentsRequest(
22342249
user_app_id=nodepool.user_app_id,
22352250
deployments=[
22362251
resources_pb2.Deployment(
22372252
id=deployment_id,
22382253
visibility=app_visibility,
2254+
desired_worker=resources_pb2.Worker(
2255+
model=resources_pb2.Model(
2256+
id=model_id,
2257+
user_id=user_id,
2258+
app_id=app_id,
2259+
model_version=resources_pb2.ModelVersion(
2260+
id=version_id,
2261+
),
2262+
)
2263+
),
22392264
)
22402265
],
2241-
action='merge',
2266+
action='overwrite',
22422267
)
2243-
nodepool._grpc_request(nodepool.STUB.PatchDeployments, patch_req)
2244-
except Exception:
2245-
pass
2268+
resp = nodepool._grpc_request(nodepool.STUB.PatchDeployments, patch_req)
2269+
if resp.status.code != status_code_pb2.SUCCESS:
2270+
out.warning(
2271+
f"Could not update deployment binding "
2272+
f"({resp.status.description}): {resp.status.details}"
2273+
)
2274+
except Exception as exc:
2275+
out.warning(f"Failed to patch deployment {deployment_id}: {exc}")
22462276
deployment_exists = True
22472277
out.status(f"Deployment ready ({deployment_id})")
22482278
except Exception:

0 commit comments

Comments
 (0)