feat(api): expose Type/LoadBalancerIP/ExternalTrafficPolicy on ServiceOverrides (closes #305)#791
Conversation
…eOverrides Resolves planetscale#305 — "Expose vtgate service by nodeport or load balancer". Adds three fields to the existing ServiceOverrides struct so operators can expose vtgate (and any other operator-managed Service that already uses ServiceOverrides — gateway, vtctld, vtadmin, etcd-lockserver, vttablet headless) via NodePort or LoadBalancer directly through the VitessCluster / VitessCell CR, without the layered-Service workaround mentioned in planetscale#305. Fields added: - Type: override Service.Spec.Type (default still ClusterIP). Applied at create time AND in-place; Kubernetes supports transitions between ClusterIP / NodePort / LoadBalancer. - LoadBalancerIP: pre-assigned LB IP when supported by the cloud / MetalLB. Create-time only (the field is immutable on existing Services). - ExternalTrafficPolicy: Cluster | Local. Mutable in-place. ClusterIP remains create-time-only (immutable on existing Services) as already documented. The same field placement is reused so all callers that already wire ServiceOverrides (cluster-aggregate vtgate, per-cell vtgate, vtctld, vtadmin, etcd-lockserver, vttablet) pick up the new fields automatically — no controller-side changes required. Includes a new pkg/operator/update/service_test.go covering: - nil overrides is a no-op - all fields applied at create time - empty overrides don't clobber existing Service spec - InPlaceServiceOverrides skips ClusterIP/LoadBalancerIP (immutable) but applies Type + ExternalTrafficPolicy + Annotations Regenerated CRD YAML via `make generate` (controller-gen). No DeepCopy changes needed — new fields are simple-type aliases handled by the existing *out = *in shallow copy. Signed-off-by: Akhilesh Agarwal <akhilesh.agarwal@gmail.com>
mattlord
left a comment
There was a problem hiding this comment.
In pkg/operator/update/service.go:43-50 I don't think that we should apply external Service fields to every ServiceOverrides target blindly. I say that as ServiceOverrides is reused by "headless" Services too, notably the global vttablet Service and the etcd peer Service. With this change, tabletService.type: LoadBalancer or peerService.type: NodePort leaves clusterIP: None in place and Kubernetes rejects the Service because headless Services cannot be NodePort/LoadBalancer. Separately, externalTrafficPolicy is applied even when the effective Service type is still ClusterIP, despite the API comment saying it is ignored; Kubernetes rejects that as only valid for externally-accessible Services. Unless I'm missing something? If not then I think we should probably split/gate these fields so they are only available for non-headless exposable Services, or validate/ignore invalid combinations before applying them.
You also should run make generate in your branch to update both the deploy/crds/* and the generated API docs as it looks like this PR currently only includes the CRD changes. That should then address the failing test.
Summary
Resolves #305 — Expose vtgate service by nodeport or load balancer.
Adds three fields to the existing
ServiceOverridesstruct so operators can expose vtgate (and any other operator-managed Service that already usesServiceOverrides— gateway, vtctld, vtadmin, etcd-lockserver, vttablet headless) via NodePort or LoadBalancer directly through theVitessCluster/VitessCellCR, eliminating the layered-Service workaround discussed in #305.API addition
Mutability matrix (matches Kubernetes' rules):
Example usage
Why this approach
The same field placement is reused so every caller that already wires
ServiceOverrides(cluster-aggregate vtgate, per-cell vtgate, vtctld, vtadmin, etcd-lockserver, vttablet headless) picks up the new fields automatically — no controller-side changes needed. The change is confined to:pkg/apis/planetscale/v2/vitesscluster_types.go— three new field declarations onServiceOverrides.pkg/operator/update/service.go— wire the new fields throughServiceOverrides()(all fields) andInPlaceServiceOverrides()(only the mutable ones).pkg/operator/update/service_test.go— new unit-tests covering nil, all-fields-applied, empty-fields-don't-clobber, in-place-skips-immutable.deploy/crds/*.yaml— regenerated bymake generate. NoDeepCopychanges — all new fields are simple-type aliases (string,corev1.ServiceType,corev1.ServiceExternalTrafficPolicy) handled by the existing shallow copy.Test plan
go build ./...passesgo test ./pkg/operator/update/ -run TestServiceOverrides -v— 4 new tests passmake generateproduces the expected CRD diff (5 CRDs updated;extraVolumeMounts-style placement at everyservice:/gatewayService:/tabletService:etc.)gatewayService.type: LoadBalancerto a live cluster with the patched operator produces a LoadBalancer Service (vs ClusterIP without the patch).Compatibility
if so.Foo != ""guards.+optionalwithomitemptytags, so existing tooling that doesn't know about them continues to validate.TypeandExternalTrafficPolicyare now updated on subsequent reconciles. Operators that previously relied on the operator-managed Service staying ClusterIP regardless of the spec will see the Service flip type if they add atype:override. This is the desired behaviour.Open questions for reviewers
LoadBalancerIPbe applied in-place even though k8s rejects it on existing Services? Current PR skips it to avoid noisy reconcile errors; happy to add a one-time create-detection if you'd prefer the apply attempt.ServiceOverrides. Want me to gate any of them (e.g., disallowTypeon the vttablet headless Service, since headless + LoadBalancer is a nonsense combination)?Signed-off-by
DCO:
Signed-off-by: Akhilesh Agarwal <akhilesh.agarwal@gmail.com>on the single commit.