Description

When deploying a second DOKS cluster with an ingress-nginx service using the same service.beta.kubernetes.io/do-loadbalancer-name annotation as an existing LB on a different cluster, the existing LB was destroyed and replaced with a new one in a different region. The original IP was permanently lost, causing a full production outage.

This is distinct from #556 where the LB "moved" between clusters but retained the same IP. In our case, the LB was deleted and recreated, resulting in permanent IP loss.

Steps to reproduce

1. Create DOKS cluster A in nyc3
2. Deploy a LoadBalancer service with annotation do-loadbalancer-name: "my-lb" → LB created with IP X.X.X.X in nyc3
3. Create DOKS cluster B in fra1
4. Deploy a LoadBalancer service with the same annotation do-loadbalancer-name: "my-lb"
5. The LB from cluster A is destroyed. A new LB is created in fra1 with a different IP
6. Cluster A's service still shows the old IP in EXTERNAL-IP but the LB no longer exists
7. All traffic to the old IP is permanently lost

Expected behavior

Step 4 should fail with an error such as:

• "Load balancer name already in use"
• "Quota exceeded (2/2)"

The existing LB should never be silently destroyed.

Actual behavior

• The existing production LB was destroyed without any warning or error
• A new LB was created in a different region (fra1 instead of nyc3) with a new IP
• The old IP was permanently lost (DO does not support reserved IPs for LBs)
• The LB quota (2/2) was not enforced — instead of rejecting the creation, the backend replaced the existing LB
• Cluster A's service was left in a stale state, showing a dead EXTERNAL-IP

Impact

• Full production outage across all services behind the LB
• Permanent loss of the original IP address
• All Cloudflare DNS records had to be manually updated to the new IP
• Required manual firewall reconfiguration and LB forwarding rule fixes

Environment

• DOKS (DigitalOcean Managed Kubernetes)
• Two clusters in different regions (nyc3 and fra1)
• CCM managed by DOKS control plane (not self-hosted)
• No kubernetes.digitalocean.com/load-balancer-id annotation was set — only do-loadbalancer-name

Key difference from #556

In #556, the LB ownership "moved" between clusters but the LB itself (and its IP) was preserved. In our case, the LB was destroyed and recreated in a different region, causing irreversible IP loss. This suggests the behavior may differ when clusters are in different regions — instead of reassigning the existing LB, the backend deletes it and creates a new one.

Suggested fix

1. The API/CCM should reject creating a LB with a name that already exists (unless explicit ownership transfer via load-balancer-id annotation)
2. At minimum, the quota check should prevent this — if the account is at 2/2 LBs, creating a new one should fail, not replace an existing one
3. Cross-region name conflicts should be treated as errors, not as implicit delete-and-recreate