From b991126ed1ed408d88030f650a1eb9871ec2a400 Mon Sep 17 00:00:00 2001 From: root Date: Tue, 3 Mar 2026 20:33:37 +0800 Subject: [PATCH 01/14] feature: add port allocation kep --- keps/171-pod-port-allocation/README.md | 230 +++++++++++++++++++++++++ keps/171-pod-port-allocation/kep.yaml | 16 ++ 2 files changed, 246 insertions(+) create mode 100644 keps/171-pod-port-allocation/README.md create mode 100644 keps/171-pod-port-allocation/kep.yaml diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md new file mode 100644 index 00000000..cfdafc12 --- /dev/null +++ b/keps/171-pod-port-allocation/README.md @@ -0,0 +1,230 @@ +# KEP-171 Pod Port Allocation + +- [Motivation](#motivation) +- [Goals](#goals) +- [Proposal](#proposal) + - [User Stories](#user-stories) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [API](#api) + - [Port Allocator Interface](#port-allocator-interface) + - [Implementation](#implementation) + - [Test Plan](#test-plan) + - [Unit Tests](#unit-tests) + - [Integration tests](#integration-tests) + - [End to End Tests](#end-to-end-tests) + + +## Motivation +When deploying LLM services using the PD (Prefill-Decode) disaggregated approach, containers often need to use `hostNetwork: true` to leverage RDMA networks. In the hostNetwork scenario, if two replicas of the same Role are scheduled to the same node, port conflicts will occur because both containers are configured with the same port, resulting in only one replica being able to start successfully. + +Therefore, RBG needs to provide a dynamic port allocation solution that can automatically assign different port numbers to replicas during service deployment, thereby improving the deployment density of inference service Pods. + +## Goals +- RBG applications should be able to automatically allocate service ports for containers on-demand during deployment, avoiding port conflicts in hostNetwork scenarios. +- The dynamically assigned ports for each Pod can be injected into the generated Pod through "container environment variables" and "Pod annotations", allowing containers to be aware of their assigned port values. +- It should be possible to obtain the port values allocated to other Pods within the current Role replicas on-demand, for Pod service discovery purposes. + +## Proposal + +### User Stories + +#### Story 1 +> Deploying inference services in a cluster + +As a cluster administrator, I want to deploy my model service using RBG in the cluster. +The service requires hostNetwork to be enabled for both the Prefill and Decoder roles. +However, since there are existing services also using hostNetwork, I need to avoid port conflicts. + + +#### Story 2 +> Scale out Role replicas + +As a LLM service operator, I noticed that the Decoder Role's load is very high recently and I need to scale out more replicas for the Decoder Role. +If each replica can use different ports, I can complete the deployment with fewer nodes when resources are sufficient. + +### Risks and Mitigations + +- The dynamic port allocation feature is only supported in *InstanceSet* mode. + +## Design Details +When the Role of an RBG resource is InstanceSet, the resource directly associated with the Pod is the Instance resource. +Therefore, the current design requires dynamic port allocation to be completed within the scope of the Instance. + +### API +If users want to configure dynamic ports for a Pod generated by an RBG object, they need to add the following annotation in the corresponding Pod Template of the RBG object: +```yaml +rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[ + { + "shareName": "", + "startPort": 30000, + "portRange": 10000, + "annotationKey": "sglang.ai/bootstrap-port", + "env": "BOOTSTRAP_PORT", + "scope": "pod", + "portRef": "" + }, + { + "shareName": "discorvery", + "startPort": 30000, + "portRange": 10000, + "annotationKey": "sglang.ai/service-discovery-port", + "env": "DISCOVERY_PORT", + "scope": "component", + "portRef": "" + } +]' +``` +The JSON field definitions in the annotation are as follows: +```go +type PortScope string + +const ( + // Port is only valid for the current Pod + Pod PortScope = "pod" + // Port is valid for all Pod replicas in the current component + Component PortScope = "component" +) + +type PortAllocatorConfig struct { + // The name of the allocated port, which can be referenced by other Pods + ShareName string `json:"shareName"` + // Reference port name, used in conjunction with the ShareName of other Pods, for injecting ports allocated to other Pods + PortRef string `json:"portRef"` + // Not empty + // Default is 30000 or setup parameter + // The starting port number for port allocation + StartPort int32 `json:"startPort"` + // Not empty + // Default is 10000 or setup parameter + // Port allocation range + PortRange int32 `json:"portRange"` + // The annotation key for port injection into the Pod + Annotation string `json:"annotationKey"` + // Not empty + // The environment variable name for port injection into the Pod + Env string `json:"env"` + // Not empty + // Default is "pod" + // The scope of port injection into the Pod + Scope PortScope `json:"scope"` +} +``` +For example, when using the following configuration: +- The leader Pod will be allocated one port + - Port 1: Injected into the container as the `LEADER_PORT` environment variable. +- Each worker Pod will be allocated three ports, where Port 1 has different values across the two Pods, Port 2 has the same value across the two Pods, and Port 3 is the same as the leader Pod's Port 1 + - Port 1: Injected into the container as the `WORKER_PORT1` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. + - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. + - Port 3: Injected into the container as the `LEADER_PORT_REF` environment variable. +```yaml +apiVersion: workloads.x-k8s.io/v1alpha1 +kind: Instance +metadata: + name: test + namespace: default +spec: + readyPolicy: AllPodReady + components: + - name: leader + size: 1 + template: + metadata: + annotations: + rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ + "shareName": "leaderPort", + "startPort": 30000, + "portRange": 10000, + "env": "LEADER_PORT", + "scope": "pod", + }]' + spec: + containers: + - name: nginx + image: nginx:1.28.0 + - name: worker + size: 2 + template: + metadata: + annotations: + rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ + "startPort": 30000, + "portRange": 10000, + "annotationKey": "test/worker-port1", + "env": "WORKER_PORT1", + "scope": "pod", + },{ + "startPort": 30000, + "portRange": 10000, + "annotationKey": "test/worker-port2", + "env": "WORKER_PORT2", + "scope": "component", + },{ + "env": "LEADER_PORT_REF", + "portRef": "leaderPort", + }]' + spec: + containers: + - name: nginx + image: nginx:1.28.0 +``` +### Port Allocator Interface + +To facilitate future extensibility, an interface can be defined for dynamic port allocation, with different port allocation strategies implemented. The port allocation strategy interface is defined as follows: + +```go +// AllocatePolicy program startup parameters +type AllocatePolicy string + +type PortAllocator struct { + policy AllocatePolicy + pa PortAllocatorInterface + client client.Client +} + +type PortAllocatorInterface interface { + // Start use to initialize the port allocator when the program starts + Start(client client.Client) error + // Allocate is used to inject requested ports into the pod + Allocate(instance *v1alpha1.Instance, pod *v1.Pod) error + // Release is used to release allocated ports + Release(instance *v1alpha1.Instance, pod *v1.Pod) error +} + +// Singleton pattern, created at program startup based on the port allocation strategy +var portAllocator *PortAllocator + +// GetPortAllocator +func GetPortAllocator() PortAllocatorInterface { + return portAllocator.pa +} +``` +### Implementation +- At program startup, add the following steps: + - Parse and validate the port allocator parameters. + - Create the corresponding port allocation strategy instance based on the configured port allocation strategy. + +- When creating pods managed by Instance, add the following steps: + - Call the port allocator's `Allocate` method to inject the required ports into the pod. + +- When updating pods managed by Instance, add the following steps: + - Call the port allocator's `Allocate` method to inject the required ports into the pod. + - Call the port allocator's `Release` method to release ports that are no longer needed by the pod. + +- When deleting pods managed by Instance, add the following steps: + - Call the port allocator's `Release` method to release ports occupied by the pod. + +### Test Plan + +#### Unit Tests + + +#### Integration tests + + +#### End to End Tests + + + diff --git a/keps/171-pod-port-allocation/kep.yaml b/keps/171-pod-port-allocation/kep.yaml new file mode 100644 index 00000000..bdbef5ca --- /dev/null +++ b/keps/171-pod-port-allocation/kep.yaml @@ -0,0 +1,16 @@ +title: Pod Port Allocation +kep-number: 171 +authors: + - "@shangsimou" +status: provisional +creation-date: 2026-03-03 +reviewers: + - "@cheyang" +approvers: + +stage: alpha + +latest-milestone: "v0.7.0" + +milestone: + alpha: "v0.7.0" From a9f27c29cc4bfb8e2a81c459fc65adcdd6107255 Mon Sep 17 00:00:00 2001 From: shangsmo <52484897+shangsmo@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:45:23 +0800 Subject: [PATCH 02/14] Update keps/171-pod-port-allocation/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index cfdafc12..b87f2778 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -67,7 +67,7 @@ rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[ "portRef": "" }, { - "shareName": "discorvery", + "shareName": "discovery", "startPort": 30000, "portRange": 10000, "annotationKey": "sglang.ai/service-discovery-port", From b038fef17277dbd920cdca8a69625e2e572a73b5 Mon Sep 17 00:00:00 2001 From: shangsmo <52484897+shangsmo@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:45:56 +0800 Subject: [PATCH 03/14] Update keps/171-pod-port-allocation/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index b87f2778..b39bbdad 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -117,7 +117,7 @@ For example, when using the following configuration: - Port 1: Injected into the container as the `LEADER_PORT` environment variable. - Each worker Pod will be allocated three ports, where Port 1 has different values across the two Pods, Port 2 has the same value across the two Pods, and Port 3 is the same as the leader Pod's Port 1 - Port 1: Injected into the container as the `WORKER_PORT1` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. - - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. + - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port2`. - Port 3: Injected into the container as the `LEADER_PORT_REF` environment variable. ```yaml apiVersion: workloads.x-k8s.io/v1alpha1 From b95e17b473ca50d1a6ba32c249d529f3f266f7be Mon Sep 17 00:00:00 2001 From: shangsmo <52484897+shangsmo@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:46:09 +0800 Subject: [PATCH 04/14] Update keps/171-pod-port-allocation/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index b39bbdad..232f0e2c 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -185,7 +185,7 @@ type PortAllocator struct { } type PortAllocatorInterface interface { - // Start use to initialize the port allocator when the program starts + // Start is used to initialize the port allocator when the program starts Start(client client.Client) error // Allocate is used to inject requested ports into the pod Allocate(instance *v1alpha1.Instance, pod *v1.Pod) error From b34d447fa7a91413834fd342fe597de02d81afd1 Mon Sep 17 00:00:00 2001 From: root Date: Tue, 3 Mar 2026 20:51:18 +0800 Subject: [PATCH 05/14] Update keps/171-pod-port-allocation/README.md --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index 232f0e2c..9d9c0f8a 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -102,7 +102,7 @@ type PortAllocatorConfig struct { // Port allocation range PortRange int32 `json:"portRange"` // The annotation key for port injection into the Pod - Annotation string `json:"annotationKey"` + AnnotationKey string `json:"annotationKey"` // Not empty // The environment variable name for port injection into the Pod Env string `json:"env"` From 1de8e4397a8ea5a5dd8dbb31c0283f852a68a4ab Mon Sep 17 00:00:00 2001 From: Super User Date: Wed, 4 Mar 2026 15:18:48 +0800 Subject: [PATCH 06/14] Update keps/171-pod-port-allocation/README.md --- keps/171-pod-port-allocation/README.md | 166 ++++++++++++++----------- 1 file changed, 91 insertions(+), 75 deletions(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index 9d9c0f8a..5e11b55a 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -56,69 +56,73 @@ Therefore, the current design requires dynamic port allocation to be completed w ### API If users want to configure dynamic ports for a Pod generated by an RBG object, they need to add the following annotation in the corresponding Pod Template of the RBG object: ```yaml -rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[ - { - "shareName": "", - "startPort": 30000, - "portRange": 10000, - "annotationKey": "sglang.ai/bootstrap-port", - "env": "BOOTSTRAP_PORT", - "scope": "pod", - "portRef": "" - }, - { - "shareName": "discovery", - "startPort": 30000, - "portRange": 10000, - "annotationKey": "sglang.ai/service-discovery-port", - "env": "DISCOVERY_PORT", - "scope": "component", - "portRef": "" - } -]' +rolebasedgroup.workloads.x-k8s.io/port-allocator: | + { + "allocations": [ + { + "name": "grpc", // Logical name + "env": "GRPC_PORT", // Env var to inject + "annotationKey": "test/grpc-port", // annotationKey to inject + "policy": "Dynamic" // Dynamic (per-pod) or Static (per-role) + } + ], + "references": [ + { + "env": "LEADER_ADDR_PORT", // Env var to inject + "from": "leader.grpc" // Format: "." + } + ] + } ``` The JSON field definitions in the annotation are as follows: ```go -type PortScope string +type PortPolicy string const ( // Port is only valid for the current Pod - Pod PortScope = "pod" - // Port is valid for all Pod replicas in the current component - Component PortScope = "component" + Dynamic PortPolicy = "Dynamic" + // Port is valid for all Pod replicas in the current role + Static PortPolicy = "Static" ) type PortAllocatorConfig struct { - // The name of the allocated port, which can be referenced by other Pods - ShareName string `json:"shareName"` - // Reference port name, used in conjunction with the ShareName of other Pods, for injecting ports allocated to other Pods - PortRef string `json:"portRef"` - // Not empty - // Default is 30000 or setup parameter - // The starting port number for port allocation - StartPort int32 `json:"startPort"` - // Not empty - // Default is 10000 or setup parameter - // Port allocation range - PortRange int32 `json:"portRange"` - // The annotation key for port injection into the Pod + // Allocations specifies the ports to be allocated + Allocations []PortAllocation `json:"allocations"` + // References specifies the ports to be referenced from other pod + References []PortReference `json:"references"` +} + +type PortAllocation struct { + // Not Empty + // Name specifies the name of the port + Name string `json:"name"` + // Not Empty + // Env specifies the name of the environment variable to be injected into the container + Env string `json:"env"` + // AnnotationKey specifies the key of the annotation to be injected into the Pod AnnotationKey string `json:"annotationKey"` - // Not empty - // The environment variable name for port injection into the Pod + // Not Empty + // Default is Dynamic + // Policy specifies the scope of the port + Policy PortPolicy `json:"policy"` +} + +type PortReference struct { + // Not Empty + // Env specifies the name of the environment variable to be injected into the container Env string `json:"env"` - // Not empty - // Default is "pod" - // The scope of port injection into the Pod - Scope PortScope `json:"scope"` + // Not Empty + // From specifies the name of the port to be referenced + From string `json:"from"` } ``` For example, when using the following configuration: - The leader Pod will be allocated one port - Port 1: Injected into the container as the `LEADER_PORT` environment variable. -- Each worker Pod will be allocated three ports, where Port 1 has different values across the two Pods, Port 2 has the same value across the two Pods, and Port 3 is the same as the leader Pod's Port 1 - - Port 1: Injected into the container as the `WORKER_PORT1` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. - - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port2`. - - Port 3: Injected into the container as the `LEADER_PORT_REF` environment variable. +- Each worker Pod will be allocated two ports and reference one from the leader Pod: + - **Dynamic Port** (`WORKER_PORT1`): Unique per Pod, annotated as `test/worker-port1` + - **Static Port** (`WORKER_PORT2`): Shared across Pods, annotated as `test/worker-port2` + - **Reference Port** (`LEADER_PORT_REF`): Points to leader's Port 1 ```yaml apiVersion: workloads.x-k8s.io/v1alpha1 kind: Instance @@ -133,13 +137,17 @@ spec: template: metadata: annotations: - rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ - "shareName": "leaderPort", - "startPort": 30000, - "portRange": 10000, - "env": "LEADER_PORT", - "scope": "pod", - }]' + rolebasedgroup.workloads.x-k8s.io/port-allocator: | + { + "allocations": [ + { + "name": "leader-port", + "env": "LEADER_PORT", + "annotationKey": "test/grpc-port", + "policy": "Dynamic" + } + ] + } spec: containers: - name: nginx @@ -149,22 +157,29 @@ spec: template: metadata: annotations: - rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ - "startPort": 30000, - "portRange": 10000, - "annotationKey": "test/worker-port1", - "env": "WORKER_PORT1", - "scope": "pod", - },{ - "startPort": 30000, - "portRange": 10000, - "annotationKey": "test/worker-port2", - "env": "WORKER_PORT2", - "scope": "component", - },{ - "env": "LEADER_PORT_REF", - "portRef": "leaderPort", - }]' + rolebasedgroup.workloads.x-k8s.io/port-allocator: | + { + "allocations": [ + { + "name": "worker-port1", + "env": "WORKER_PORT1", + "annotationKey": "test/worker-port1", + "policy": "Dynamic" + }, + { + "name": "worker-port2", + "env": "WORKER_PORT2", + "annotationKey": "test/worker-port2", + "policy": "Static" + } + ], + "references": [ + { + "env": "LEADER_ADDR_PORT", + "from": "leader.leader-port" + } + ] + } spec: containers: - name: nginx @@ -175,11 +190,11 @@ spec: To facilitate future extensibility, an interface can be defined for dynamic port allocation, with different port allocation strategies implemented. The port allocation strategy interface is defined as follows: ```go -// AllocatePolicy program startup parameters -type AllocatePolicy string +// AllocateStrategy program startup flags +type AllocateStrategy string type PortAllocator struct { - policy AllocatePolicy + strategy AllocateStrategy pa PortAllocatorInterface client client.Client } @@ -202,8 +217,8 @@ func GetPortAllocator() PortAllocatorInterface { } ``` ### Implementation -- At program startup, add the following steps: - - Parse and validate the port allocator parameters. +- At controller startup, add the following steps: + - Parse and validate port allocator flags, such as `AllocateStrategy`, `StartPort`, and `PortRange`. - Create the corresponding port allocation strategy instance based on the configured port allocation strategy. - When creating pods managed by Instance, add the following steps: @@ -228,3 +243,4 @@ func GetPortAllocator() PortAllocatorInterface { + From 818be1607149dbdcd3994dda74ff0422c885095a Mon Sep 17 00:00:00 2001 From: root Date: Tue, 3 Mar 2026 20:33:37 +0800 Subject: [PATCH 07/14] feature: add port allocation kep --- keps/171-pod-port-allocation/README.md | 230 +++++++++++++++++++++++++ keps/171-pod-port-allocation/kep.yaml | 16 ++ 2 files changed, 246 insertions(+) create mode 100644 keps/171-pod-port-allocation/README.md create mode 100644 keps/171-pod-port-allocation/kep.yaml diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md new file mode 100644 index 00000000..cfdafc12 --- /dev/null +++ b/keps/171-pod-port-allocation/README.md @@ -0,0 +1,230 @@ +# KEP-171 Pod Port Allocation + +- [Motivation](#motivation) +- [Goals](#goals) +- [Proposal](#proposal) + - [User Stories](#user-stories) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [API](#api) + - [Port Allocator Interface](#port-allocator-interface) + - [Implementation](#implementation) + - [Test Plan](#test-plan) + - [Unit Tests](#unit-tests) + - [Integration tests](#integration-tests) + - [End to End Tests](#end-to-end-tests) + + +## Motivation +When deploying LLM services using the PD (Prefill-Decode) disaggregated approach, containers often need to use `hostNetwork: true` to leverage RDMA networks. In the hostNetwork scenario, if two replicas of the same Role are scheduled to the same node, port conflicts will occur because both containers are configured with the same port, resulting in only one replica being able to start successfully. + +Therefore, RBG needs to provide a dynamic port allocation solution that can automatically assign different port numbers to replicas during service deployment, thereby improving the deployment density of inference service Pods. + +## Goals +- RBG applications should be able to automatically allocate service ports for containers on-demand during deployment, avoiding port conflicts in hostNetwork scenarios. +- The dynamically assigned ports for each Pod can be injected into the generated Pod through "container environment variables" and "Pod annotations", allowing containers to be aware of their assigned port values. +- It should be possible to obtain the port values allocated to other Pods within the current Role replicas on-demand, for Pod service discovery purposes. + +## Proposal + +### User Stories + +#### Story 1 +> Deploying inference services in a cluster + +As a cluster administrator, I want to deploy my model service using RBG in the cluster. +The service requires hostNetwork to be enabled for both the Prefill and Decoder roles. +However, since there are existing services also using hostNetwork, I need to avoid port conflicts. + + +#### Story 2 +> Scale out Role replicas + +As a LLM service operator, I noticed that the Decoder Role's load is very high recently and I need to scale out more replicas for the Decoder Role. +If each replica can use different ports, I can complete the deployment with fewer nodes when resources are sufficient. + +### Risks and Mitigations + +- The dynamic port allocation feature is only supported in *InstanceSet* mode. + +## Design Details +When the Role of an RBG resource is InstanceSet, the resource directly associated with the Pod is the Instance resource. +Therefore, the current design requires dynamic port allocation to be completed within the scope of the Instance. + +### API +If users want to configure dynamic ports for a Pod generated by an RBG object, they need to add the following annotation in the corresponding Pod Template of the RBG object: +```yaml +rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[ + { + "shareName": "", + "startPort": 30000, + "portRange": 10000, + "annotationKey": "sglang.ai/bootstrap-port", + "env": "BOOTSTRAP_PORT", + "scope": "pod", + "portRef": "" + }, + { + "shareName": "discorvery", + "startPort": 30000, + "portRange": 10000, + "annotationKey": "sglang.ai/service-discovery-port", + "env": "DISCOVERY_PORT", + "scope": "component", + "portRef": "" + } +]' +``` +The JSON field definitions in the annotation are as follows: +```go +type PortScope string + +const ( + // Port is only valid for the current Pod + Pod PortScope = "pod" + // Port is valid for all Pod replicas in the current component + Component PortScope = "component" +) + +type PortAllocatorConfig struct { + // The name of the allocated port, which can be referenced by other Pods + ShareName string `json:"shareName"` + // Reference port name, used in conjunction with the ShareName of other Pods, for injecting ports allocated to other Pods + PortRef string `json:"portRef"` + // Not empty + // Default is 30000 or setup parameter + // The starting port number for port allocation + StartPort int32 `json:"startPort"` + // Not empty + // Default is 10000 or setup parameter + // Port allocation range + PortRange int32 `json:"portRange"` + // The annotation key for port injection into the Pod + Annotation string `json:"annotationKey"` + // Not empty + // The environment variable name for port injection into the Pod + Env string `json:"env"` + // Not empty + // Default is "pod" + // The scope of port injection into the Pod + Scope PortScope `json:"scope"` +} +``` +For example, when using the following configuration: +- The leader Pod will be allocated one port + - Port 1: Injected into the container as the `LEADER_PORT` environment variable. +- Each worker Pod will be allocated three ports, where Port 1 has different values across the two Pods, Port 2 has the same value across the two Pods, and Port 3 is the same as the leader Pod's Port 1 + - Port 1: Injected into the container as the `WORKER_PORT1` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. + - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. + - Port 3: Injected into the container as the `LEADER_PORT_REF` environment variable. +```yaml +apiVersion: workloads.x-k8s.io/v1alpha1 +kind: Instance +metadata: + name: test + namespace: default +spec: + readyPolicy: AllPodReady + components: + - name: leader + size: 1 + template: + metadata: + annotations: + rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ + "shareName": "leaderPort", + "startPort": 30000, + "portRange": 10000, + "env": "LEADER_PORT", + "scope": "pod", + }]' + spec: + containers: + - name: nginx + image: nginx:1.28.0 + - name: worker + size: 2 + template: + metadata: + annotations: + rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ + "startPort": 30000, + "portRange": 10000, + "annotationKey": "test/worker-port1", + "env": "WORKER_PORT1", + "scope": "pod", + },{ + "startPort": 30000, + "portRange": 10000, + "annotationKey": "test/worker-port2", + "env": "WORKER_PORT2", + "scope": "component", + },{ + "env": "LEADER_PORT_REF", + "portRef": "leaderPort", + }]' + spec: + containers: + - name: nginx + image: nginx:1.28.0 +``` +### Port Allocator Interface + +To facilitate future extensibility, an interface can be defined for dynamic port allocation, with different port allocation strategies implemented. The port allocation strategy interface is defined as follows: + +```go +// AllocatePolicy program startup parameters +type AllocatePolicy string + +type PortAllocator struct { + policy AllocatePolicy + pa PortAllocatorInterface + client client.Client +} + +type PortAllocatorInterface interface { + // Start use to initialize the port allocator when the program starts + Start(client client.Client) error + // Allocate is used to inject requested ports into the pod + Allocate(instance *v1alpha1.Instance, pod *v1.Pod) error + // Release is used to release allocated ports + Release(instance *v1alpha1.Instance, pod *v1.Pod) error +} + +// Singleton pattern, created at program startup based on the port allocation strategy +var portAllocator *PortAllocator + +// GetPortAllocator +func GetPortAllocator() PortAllocatorInterface { + return portAllocator.pa +} +``` +### Implementation +- At program startup, add the following steps: + - Parse and validate the port allocator parameters. + - Create the corresponding port allocation strategy instance based on the configured port allocation strategy. + +- When creating pods managed by Instance, add the following steps: + - Call the port allocator's `Allocate` method to inject the required ports into the pod. + +- When updating pods managed by Instance, add the following steps: + - Call the port allocator's `Allocate` method to inject the required ports into the pod. + - Call the port allocator's `Release` method to release ports that are no longer needed by the pod. + +- When deleting pods managed by Instance, add the following steps: + - Call the port allocator's `Release` method to release ports occupied by the pod. + +### Test Plan + +#### Unit Tests + + +#### Integration tests + + +#### End to End Tests + + + diff --git a/keps/171-pod-port-allocation/kep.yaml b/keps/171-pod-port-allocation/kep.yaml new file mode 100644 index 00000000..bdbef5ca --- /dev/null +++ b/keps/171-pod-port-allocation/kep.yaml @@ -0,0 +1,16 @@ +title: Pod Port Allocation +kep-number: 171 +authors: + - "@shangsimou" +status: provisional +creation-date: 2026-03-03 +reviewers: + - "@cheyang" +approvers: + +stage: alpha + +latest-milestone: "v0.7.0" + +milestone: + alpha: "v0.7.0" From f06cdd4abfa1837efaa81a9048bad3a3b868dc1e Mon Sep 17 00:00:00 2001 From: shangsmo <52484897+shangsmo@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:45:23 +0800 Subject: [PATCH 08/14] Update keps/171-pod-port-allocation/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index cfdafc12..b87f2778 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -67,7 +67,7 @@ rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[ "portRef": "" }, { - "shareName": "discorvery", + "shareName": "discovery", "startPort": 30000, "portRange": 10000, "annotationKey": "sglang.ai/service-discovery-port", From 5c77a8f255c82b9156a0af11e4d869ee812f33e6 Mon Sep 17 00:00:00 2001 From: shangsmo <52484897+shangsmo@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:45:56 +0800 Subject: [PATCH 09/14] Update keps/171-pod-port-allocation/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index b87f2778..b39bbdad 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -117,7 +117,7 @@ For example, when using the following configuration: - Port 1: Injected into the container as the `LEADER_PORT` environment variable. - Each worker Pod will be allocated three ports, where Port 1 has different values across the two Pods, Port 2 has the same value across the two Pods, and Port 3 is the same as the leader Pod's Port 1 - Port 1: Injected into the container as the `WORKER_PORT1` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. - - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. + - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port2`. - Port 3: Injected into the container as the `LEADER_PORT_REF` environment variable. ```yaml apiVersion: workloads.x-k8s.io/v1alpha1 From a2371390cacd4d5f510f10fef685792e5ab080e5 Mon Sep 17 00:00:00 2001 From: shangsmo <52484897+shangsmo@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:46:09 +0800 Subject: [PATCH 10/14] Update keps/171-pod-port-allocation/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index b39bbdad..232f0e2c 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -185,7 +185,7 @@ type PortAllocator struct { } type PortAllocatorInterface interface { - // Start use to initialize the port allocator when the program starts + // Start is used to initialize the port allocator when the program starts Start(client client.Client) error // Allocate is used to inject requested ports into the pod Allocate(instance *v1alpha1.Instance, pod *v1.Pod) error From bb1662e0f5a6958097634614955f577b82168393 Mon Sep 17 00:00:00 2001 From: root Date: Tue, 3 Mar 2026 20:51:18 +0800 Subject: [PATCH 11/14] Update keps/171-pod-port-allocation/README.md --- keps/171-pod-port-allocation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index 232f0e2c..9d9c0f8a 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -102,7 +102,7 @@ type PortAllocatorConfig struct { // Port allocation range PortRange int32 `json:"portRange"` // The annotation key for port injection into the Pod - Annotation string `json:"annotationKey"` + AnnotationKey string `json:"annotationKey"` // Not empty // The environment variable name for port injection into the Pod Env string `json:"env"` From eb79bb3e5bb61fd79e5a073a699e66e9db29944b Mon Sep 17 00:00:00 2001 From: Super User Date: Wed, 4 Mar 2026 15:18:48 +0800 Subject: [PATCH 12/14] Update keps/171-pod-port-allocation/README.md --- keps/171-pod-port-allocation/README.md | 166 ++++++++++++++----------- 1 file changed, 91 insertions(+), 75 deletions(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index 9d9c0f8a..5e11b55a 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -56,69 +56,73 @@ Therefore, the current design requires dynamic port allocation to be completed w ### API If users want to configure dynamic ports for a Pod generated by an RBG object, they need to add the following annotation in the corresponding Pod Template of the RBG object: ```yaml -rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[ - { - "shareName": "", - "startPort": 30000, - "portRange": 10000, - "annotationKey": "sglang.ai/bootstrap-port", - "env": "BOOTSTRAP_PORT", - "scope": "pod", - "portRef": "" - }, - { - "shareName": "discovery", - "startPort": 30000, - "portRange": 10000, - "annotationKey": "sglang.ai/service-discovery-port", - "env": "DISCOVERY_PORT", - "scope": "component", - "portRef": "" - } -]' +rolebasedgroup.workloads.x-k8s.io/port-allocator: | + { + "allocations": [ + { + "name": "grpc", // Logical name + "env": "GRPC_PORT", // Env var to inject + "annotationKey": "test/grpc-port", // annotationKey to inject + "policy": "Dynamic" // Dynamic (per-pod) or Static (per-role) + } + ], + "references": [ + { + "env": "LEADER_ADDR_PORT", // Env var to inject + "from": "leader.grpc" // Format: "." + } + ] + } ``` The JSON field definitions in the annotation are as follows: ```go -type PortScope string +type PortPolicy string const ( // Port is only valid for the current Pod - Pod PortScope = "pod" - // Port is valid for all Pod replicas in the current component - Component PortScope = "component" + Dynamic PortPolicy = "Dynamic" + // Port is valid for all Pod replicas in the current role + Static PortPolicy = "Static" ) type PortAllocatorConfig struct { - // The name of the allocated port, which can be referenced by other Pods - ShareName string `json:"shareName"` - // Reference port name, used in conjunction with the ShareName of other Pods, for injecting ports allocated to other Pods - PortRef string `json:"portRef"` - // Not empty - // Default is 30000 or setup parameter - // The starting port number for port allocation - StartPort int32 `json:"startPort"` - // Not empty - // Default is 10000 or setup parameter - // Port allocation range - PortRange int32 `json:"portRange"` - // The annotation key for port injection into the Pod + // Allocations specifies the ports to be allocated + Allocations []PortAllocation `json:"allocations"` + // References specifies the ports to be referenced from other pod + References []PortReference `json:"references"` +} + +type PortAllocation struct { + // Not Empty + // Name specifies the name of the port + Name string `json:"name"` + // Not Empty + // Env specifies the name of the environment variable to be injected into the container + Env string `json:"env"` + // AnnotationKey specifies the key of the annotation to be injected into the Pod AnnotationKey string `json:"annotationKey"` - // Not empty - // The environment variable name for port injection into the Pod + // Not Empty + // Default is Dynamic + // Policy specifies the scope of the port + Policy PortPolicy `json:"policy"` +} + +type PortReference struct { + // Not Empty + // Env specifies the name of the environment variable to be injected into the container Env string `json:"env"` - // Not empty - // Default is "pod" - // The scope of port injection into the Pod - Scope PortScope `json:"scope"` + // Not Empty + // From specifies the name of the port to be referenced + From string `json:"from"` } ``` For example, when using the following configuration: - The leader Pod will be allocated one port - Port 1: Injected into the container as the `LEADER_PORT` environment variable. -- Each worker Pod will be allocated three ports, where Port 1 has different values across the two Pods, Port 2 has the same value across the two Pods, and Port 3 is the same as the leader Pod's Port 1 - - Port 1: Injected into the container as the `WORKER_PORT1` environment variable, and the corresponding Pod is annotated with `test/worker-port1`. - - Port 2: Injected into the container as the `WORKER_PORT2` environment variable, and the corresponding Pod is annotated with `test/worker-port2`. - - Port 3: Injected into the container as the `LEADER_PORT_REF` environment variable. +- Each worker Pod will be allocated two ports and reference one from the leader Pod: + - **Dynamic Port** (`WORKER_PORT1`): Unique per Pod, annotated as `test/worker-port1` + - **Static Port** (`WORKER_PORT2`): Shared across Pods, annotated as `test/worker-port2` + - **Reference Port** (`LEADER_PORT_REF`): Points to leader's Port 1 ```yaml apiVersion: workloads.x-k8s.io/v1alpha1 kind: Instance @@ -133,13 +137,17 @@ spec: template: metadata: annotations: - rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ - "shareName": "leaderPort", - "startPort": 30000, - "portRange": 10000, - "env": "LEADER_PORT", - "scope": "pod", - }]' + rolebasedgroup.workloads.x-k8s.io/port-allocator: | + { + "allocations": [ + { + "name": "leader-port", + "env": "LEADER_PORT", + "annotationKey": "test/grpc-port", + "policy": "Dynamic" + } + ] + } spec: containers: - name: nginx @@ -149,22 +157,29 @@ spec: template: metadata: annotations: - rolebasedgroup.workloads.x-k8s.io/port.allocator-config: '[{ - "startPort": 30000, - "portRange": 10000, - "annotationKey": "test/worker-port1", - "env": "WORKER_PORT1", - "scope": "pod", - },{ - "startPort": 30000, - "portRange": 10000, - "annotationKey": "test/worker-port2", - "env": "WORKER_PORT2", - "scope": "component", - },{ - "env": "LEADER_PORT_REF", - "portRef": "leaderPort", - }]' + rolebasedgroup.workloads.x-k8s.io/port-allocator: | + { + "allocations": [ + { + "name": "worker-port1", + "env": "WORKER_PORT1", + "annotationKey": "test/worker-port1", + "policy": "Dynamic" + }, + { + "name": "worker-port2", + "env": "WORKER_PORT2", + "annotationKey": "test/worker-port2", + "policy": "Static" + } + ], + "references": [ + { + "env": "LEADER_ADDR_PORT", + "from": "leader.leader-port" + } + ] + } spec: containers: - name: nginx @@ -175,11 +190,11 @@ spec: To facilitate future extensibility, an interface can be defined for dynamic port allocation, with different port allocation strategies implemented. The port allocation strategy interface is defined as follows: ```go -// AllocatePolicy program startup parameters -type AllocatePolicy string +// AllocateStrategy program startup flags +type AllocateStrategy string type PortAllocator struct { - policy AllocatePolicy + strategy AllocateStrategy pa PortAllocatorInterface client client.Client } @@ -202,8 +217,8 @@ func GetPortAllocator() PortAllocatorInterface { } ``` ### Implementation -- At program startup, add the following steps: - - Parse and validate the port allocator parameters. +- At controller startup, add the following steps: + - Parse and validate port allocator flags, such as `AllocateStrategy`, `StartPort`, and `PortRange`. - Create the corresponding port allocation strategy instance based on the configured port allocation strategy. - When creating pods managed by Instance, add the following steps: @@ -228,3 +243,4 @@ func GetPortAllocator() PortAllocatorInterface { + From 6110903cffde42e94b9cdc5e3bffe7d7e30f774f Mon Sep 17 00:00:00 2001 From: shangsmo Date: Sun, 8 Mar 2026 15:51:59 +0800 Subject: [PATCH 13/14] =?UTF-8?q?update=20kep-171=20pod=20port=20allocatio?= =?UTF-8?q?n=E2=80=98s=20README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- keps/171-pod-port-allocation/README.md | 83 ++++++++++++++++++++------ 1 file changed, 66 insertions(+), 17 deletions(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index 5e11b55a..71c7b223 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -202,10 +202,10 @@ type PortAllocator struct { type PortAllocatorInterface interface { // Start is used to initialize the port allocator when the program starts Start(client client.Client) error - // Allocate is used to inject requested ports into the pod - Allocate(instance *v1alpha1.Instance, pod *v1.Pod) error - // Release is used to release allocated ports - Release(instance *v1alpha1.Instance, pod *v1.Pod) error + // Release releases a port, input the port to release + Release(port int32) error + // AllocateBatch allocates multiple ports, input the number of ports to allocate, output the list of allocated port numbers + AllocateBatch(num int32) ([]int32, error) } // Singleton pattern, created at program startup based on the port allocation strategy @@ -217,20 +217,69 @@ func GetPortAllocator() PortAllocatorInterface { } ``` ### Implementation -- At controller startup, add the following steps: - - Parse and validate port allocator flags, such as `AllocateStrategy`, `StartPort`, and `PortRange`. - - Create the corresponding port allocation strategy instance based on the configured port allocation strategy. - -- When creating pods managed by Instance, add the following steps: - - Call the port allocator's `Allocate` method to inject the required ports into the pod. - -- When updating pods managed by Instance, add the following steps: - - Call the port allocator's `Allocate` method to inject the required ports into the pod. - - Call the port allocator's `Release` method to release ports that are no longer needed by the pod. - -- When deleting pods managed by Instance, add the following steps: - - Call the port allocator's `Release` method to release ports occupied by the pod. +#### Controller Startup Modifications + +- **Controller Initialization**: + - Parse and validate port allocator configuration flags, including `AllocateStrategy`, `StartPort`, and `PortRange` + - Instantiate the corresponding port allocator implementation based on the configured `AllocateStrategy` + - Initialize the port allocator by calling its `Start` method with the Kubernetes client + +#### Instance Reconciler Modifications for `Dynamic` Policy Ports + +For ports with `Dynamic` policy, the Instance reconciler manages port allocation on a per-Pod basis: + +- **Pod Creation**: + 1. Parse the port allocator annotation to extract `Dynamic` policy port configurations + 2. Call the port allocator's `AllocateBatch` method to obtain the required number of ports + 3. Create or update a ConfigMap named `instace--ports` to store allocated ports: + - Data keys follow the format: `.` (e.g., `worker-0.grpc-port: "30001"`) + - The ConfigMap is owned by the Instance resource for automatic cleanup + 4. Inject allocated ports into the Pod specification: + - Add environment variables to containers using `valueFrom.configMapKeyRef` to reference the ConfigMap as specified by the `env` field + - Add Pod annotations as specified by the `annotationKey` field with the port value from ConfigMap + - Mount the ConfigMap for cross-pod service discovery if references are configured + +- **Pod Update**: + 1. Detect changes in port allocation requirements by comparing current and updated pod templates + 2. Allocate new ports via `AllocateBatch` for additional or modified port requirements + 3. Update the ConfigMap with new port allocations + 4. Update the Pod spec to reflect the new ConfigMap references: + - Update environment variable references (`valueFrom.configMapKeyRef`) to point to the new keys in ConfigMap + - Update Pod annotations with new port values from ConfigMap + 5. Release obsolete ports via the `Release` method after the updated Pod becomes ready + 6. Clean up removed port entries from the ConfigMap + +- **Pod Deletion**: + 1. Retrieve allocated port information from the ConfigMap for the deleted Pod + 2. Release all ports associated with the deleted Pod via the `Release` method + 3. Remove corresponding entries from the ConfigMap + 4. Delete the ConfigMap if no ports remain (optional cleanup policy) + +#### InstanceSet Reconciler Modifications for `Static` Policy Ports + +For ports with `Static` policy, the InstanceSet reconciler manages port allocation at the Role level: + +- **Instance Creation**: + 1. Parse the port allocator annotation to extract `Static` policy port configurations + 2. Call the port allocator's `AllocateBatch` method to obtain one port per port definition (shared across all replicas) + 3. Create or update a ConfigMap named `instanceset--ports` to store static port allocations: + - Data keys follow the format: `.` (e.g., `test.worker-port2: "30002"`) + - The ConfigMap is owned by the InstanceSet resource for automatic cleanup + 4. Inject the ConfigMap reference into the Pod template: + - Add volume referencing the ConfigMap to the Pod spec + +- **Instance Update**: + 1. Detect changes in static port requirements between old and new pod templates + 2. Allocate new ports via `AllocateBatch` for new or modified port requirements + 3. Update the ConfigMap with new static port allocations + 4. Update the Pod template volume references to the ConfigMap + 5. Release obsolete ports via the `Release` method once all Pods using the old template are terminated + +- **InstanceSet Deletion** (when all instances are deleted): + 1. When the InstanceSet is being deleted or scaled to zero replicas, retrieve all static port allocations from the ConfigMap + 2. Release all ports occupied by the InstanceSet via the `Release` method + 3. Delete the ConfigMap containing static port allocations ### Test Plan #### Unit Tests From a907d9f74d2069109c762911abca78ac6dc42b1c Mon Sep 17 00:00:00 2001 From: shangsmo Date: Sun, 8 Mar 2026 16:12:13 +0800 Subject: [PATCH 14/14] =?UTF-8?q?update=20kep-171=20pod=20port=20allocatio?= =?UTF-8?q?n=E2=80=98s=20README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- keps/171-pod-port-allocation/README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/keps/171-pod-port-allocation/README.md b/keps/171-pod-port-allocation/README.md index 71c7b223..36770284 100644 --- a/keps/171-pod-port-allocation/README.md +++ b/keps/171-pod-port-allocation/README.md @@ -276,10 +276,9 @@ For ports with `Static` policy, the InstanceSet reconciler manages port allocati 4. Update the Pod template volume references to the ConfigMap 5. Release obsolete ports via the `Release` method once all Pods using the old template are terminated -- **InstanceSet Deletion** (when all instances are deleted): - 1. When the InstanceSet is being deleted or scaled to zero replicas, retrieve all static port allocations from the ConfigMap - 2. Release all ports occupied by the InstanceSet via the `Release` method - 3. Delete the ConfigMap containing static port allocations +- **Instance Deletion**: + 1. Remove the corresponding key from the ConfigMap for the deleted instance + 2. If all instances in the InstanceSet are deleted, release all ports occupied by the InstanceSet via the `Release` method and delete the ConfigMap ### Test Plan #### Unit Tests