Skip to content

[Bug] Controller manager v0.1.13 crashes on startup: flag provided but not defined: -kube-client-qps #855

@zson-two

Description

@zson-two

Environment

  • Deployment Manifests (Helm/YAML): v0.1.10 (or older)
  • Controller Manager Image: v0.1.13
  • Client SDK: v0.1.8

Bug Description

After upgrading the OpenSandbox server image to v0.1.13, the controller-manager pod fails to start and enters a CrashLoopBackOff state. As a result, all API requests to the sandbox service fail immediately, resulting in TCP connection drops.

Clients using the Python SDK receive the following exception:
httpx.RemoteProtocolError: Server disconnected without sending a response

Raw curl commands return:
curl: (52) Empty reply from server

Checking the pod logs reveals that the binary is crashing instantly due to an unrecognized command-line flag:

$ kubectl logs opensandbox-controller-manager-58b457c97b-bm7hk -n opensandbox-system
flag provided but not defined: -kube-client-qps
Usage of /workspace/server:
  -enable-file-log
        Enable log output to file
  -enable-http2
        If set, HTTP/2 will be enabled for the metrics and webhook servers
  ...

Root Cause Analysis

The Go binary in the v0.1.13 image no longer defines or accepts the -kube-client-qps command-line flag. However, because our deployment manifests (Helm chart or Kubernetes Deployment YAML) are still from an older version (e.g., v0.1.10), they are still passing -kube-client-qps in the container's args.

When the v0.1.13 binary starts and parses the arguments, the flag package throws an error for the undefined flag, prints the usage text, and exits with a fatal error. This brings down the entire controller-manager pod, leaving the service without an active backend.

Expected Behavior

  1. Backward Compatibility: If a flag is removed, it is usually better to keep it defined but ignored (with a deprecation warning log) for a few minor releases. This ensures backward compatibility during rolling upgrades or when users mix slightly older Helm charts with newer images.
  2. Documentation: Breaking changes like the removal of command-line flags (e.g., -kube-client-qps) should be prominently documented in the Release Notes so users are aware that they must update their Helm charts/Deployment manifests simultaneously with the image.

Steps to Reproduce

  1. Deploy OpenSandbox using older manifests (e.g., v0.1.10) which include the -kube-client-qps flag in the deployment container args.
  2. Update the container image tag of the opensandbox-controller-manager deployment to v0.1.13.
  3. Check the pod logs to observe the flag provided but not defined: -kube-client-qps crash.
  4. Attempt to hit the service endpoint and observe the immediate TCP connection drop.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions