Skip to content

feat(dns): NSX DNS record service and zone validation#1423

Merged
wenyingd merged 1 commit into
vmware-tanzu:mainfrom
wenyingd:dns-only
May 28, 2026
Merged

feat(dns): NSX DNS record service and zone validation#1423
wenyingd merged 1 commit into
vmware-tanzu:mainfrom
wenyingd:dns-only

Conversation

@wenyingd

@wenyingd wenyingd commented May 6, 2026

Copy link
Copy Markdown
Contributor
  1. Implement DNSRecordService for NSX ProjectDnsRecord CRUD operations
  2. Validate hostnames against VPCNetworkConfiguration allowed DNS zones
  3. Wrap hostname-mismatch error as DNSZoneValidationError for accurate
    DNSRecordReady condition reporting
  4. Remove the realization state check before NSX API is ready.

Testing Done:

  1. Prepare a DNS zone in NSX as below,
{
  "dns_domain_name": "example.com",
  "ttl": 300,
  "resource_type": "ProjectDnsZone",
  "id": "project-zone",
  "display_name": "project-zone",
  "path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
  "relative_path": "project-zone",
  "parent_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service",
  "remote_path": "",
  "unique_id": "1e159072-29c4-469d-adee-7b460dd4c640",
  "realization_id": "1e159072-29c4-469d-adee-7b460dd4c640",
  "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
  "marked_for_delete": false,
  "overridden": false,
  "_system_owned": false,
  "_protection": "NOT_PROTECTED",
  "_create_time": 1779435122113,
  "_create_user": "admin",
  "_last_modified_time": 1779435122113,
  "_last_modified_user": "admin",
  "_revision": 0
}
  1. Create a LB Service , and annotate it with annotation
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1
NAME            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE
gateway-istio   LoadBalancer   172.24.184.123   192.168.0.8   15021:31957/TCP,80:32095/TCP   21m
httpbin         ClusterIP      172.24.60.20     <none>        8000/TCP                       21m
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname="my-svc.example.com"
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    nsx.vmware.com/hostname: my-svc.example.com
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1783567"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2026-05-22T11:00:26Z"
    message: ""
    reason: DNSRecordConfigured
    status: "True"
    type: Ready
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check DNS record in NSX policy API, the following records are returned,
{
  "results": [
    {
      "zone_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
      "record_name": "my-svc",
      "record_type": "A",
      "record_values": [
        "192.168.0.8"
      ],
      "ttl": 300,
      "resource_type": "ProjectDnsRecord",
      "id": "my-svc_project-zone_a",
      "display_name": "my-svc",
      "tags": [
        {
          "scope": "nsx-op/cluster",
          "tag": "46c622d5-811b-4c50-a033-d79b13aff596"
        },
        {
          "scope": "nsx-op/version",
          "tag": "1.0.0"
        },
        {
          "scope": "nsx-op/namespace_uid",
          "tag": "b629cf71-de07-49af-ac87-6264c664a139"
        },
        {
          "scope": "nsx-op/dns_for",
          "tag": "service"
        },
        {
          "scope": "nsx-op/dns_owner_namespace",
          "tag": "test1"
        },
        {
          "scope": "nsx-op/dns_owner_name",
          "tag": "gateway-istio"
        }
      ],
      "path": "/orgs/default/projects/project-quality/dns-records/my-svc_project-zone_a",
      "relative_path": "my-svc_project-zone_a",
      "parent_path": "/orgs/default/projects/project-quality",
      "remote_path": "",
      "unique_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "realization_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
      "marked_for_delete": false,
      "overridden": false,
      "_system_owned": false,
      "_protection": "REQUIRE_OVERRIDE",
      "_create_time": 1779447625667,
      "_create_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_last_modified_time": 1779447625667,
      "_last_modified_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_revision": 0
    }
  ],
  "result_count": 1,
  "sort_by": "display_name",
  "sort_ascending": true
}
  1. Add multiple hostnames in the service with annotation
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname="my-svc.example.com,istio-gw-svc.example.com" --overwrite
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    nsx.vmware.com/hostname: my-svc.example.com,istio-gw-svc.example.com
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1792158"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2026-05-22T11:00:26Z"
    message: ""
    reason: DNSRecordConfigured
    status: "True"
    type: Ready
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check NSX DNS record in NSX policy API, two records are returned,
{
  "results": [
    {
      "zone_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
      "record_name": "istio-gw-svc",
      "record_type": "A",
      "record_values": [
        "192.168.0.8"
      ],
      "ttl": 300,
      "resource_type": "ProjectDnsRecord",
      "id": "istio-gw-svc_project-zone_a",
      "display_name": "istio-gw-svc",
      "tags": [
        {
          "scope": "nsx-op/cluster",
          "tag": "46c622d5-811b-4c50-a033-d79b13aff596"
        },
        {
          "scope": "nsx-op/version",
          "tag": "1.0.0"
        },
        {
          "scope": "nsx-op/namespace_uid",
          "tag": "b629cf71-de07-49af-ac87-6264c664a139"
        },
        {
          "scope": "nsx-op/dns_for",
          "tag": "service"
        },
        {
          "scope": "nsx-op/dns_owner_namespace",
          "tag": "test1"
        },
        {
          "scope": "nsx-op/dns_owner_name",
          "tag": "gateway-istio"
        }
      ],
      "path": "/orgs/default/projects/project-quality/dns-records/istio-gw-svc_project-zone_a",
      "relative_path": "istio-gw-svc_project-zone_a",
      "parent_path": "/orgs/default/projects/project-quality",
      "remote_path": "",
      "unique_id": "5ab46632-d8f9-45a8-a036-2d7f6aa378a1",
      "realization_id": "5ab46632-d8f9-45a8-a036-2d7f6aa378a1",
      "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
      "marked_for_delete": false,
      "overridden": false,
      "_system_owned": false,
      "_protection": "REQUIRE_OVERRIDE",
      "_create_time": 1779448227678,
      "_create_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_last_modified_time": 1779448227678,
      "_last_modified_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_revision": 0
    },
    {
      "zone_path": "/orgs/default/projects/project-quality/dns-services/default-dns-service/zones/project-zone",
      "record_name": "my-svc",
      "record_type": "A",
      "record_values": [
        "192.168.0.8"
      ],
      "ttl": 300,
      "resource_type": "ProjectDnsRecord",
      "id": "my-svc_project-zone_a",
      "display_name": "my-svc",
      "tags": [
        {
          "scope": "nsx-op/cluster",
          "tag": "46c622d5-811b-4c50-a033-d79b13aff596"
        },
        {
          "scope": "nsx-op/version",
          "tag": "1.0.0"
        },
        {
          "scope": "nsx-op/namespace_uid",
          "tag": "b629cf71-de07-49af-ac87-6264c664a139"
        },
        {
          "scope": "nsx-op/dns_for",
          "tag": "service"
        },
        {
          "scope": "nsx-op/dns_owner_namespace",
          "tag": "test1"
        },
        {
          "scope": "nsx-op/dns_owner_name",
          "tag": "gateway-istio"
        }
      ],
      "path": "/orgs/default/projects/project-quality/dns-records/my-svc_project-zone_a",
      "relative_path": "my-svc_project-zone_a",
      "parent_path": "/orgs/default/projects/project-quality",
      "remote_path": "",
      "unique_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "realization_id": "907fe7d4-fefc-4135-9f24-52a77a847a19",
      "owner_id": "8cc4ef5e-4d1c-4461-a49d-231bd02e718e",
      "marked_for_delete": false,
      "overridden": false,
      "_system_owned": false,
      "_protection": "REQUIRE_OVERRIDE",
      "_create_time": 1779447625667,
      "_create_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_last_modified_time": 1779447625667,
      "_last_modified_user": "wcp-cluster-user-46c622d5-811b-4c50-a033-d79b13aff596-445b9b82-2c0b-44b0-9f03-3a4ac89e2b1c",
      "_revision": 0
    }
  ],
  "result_count": 2,
  "sort_by": "display_name",
  "sort_ascending": true
}
  1. Remove the annotation from the LB Service,
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname-
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# 
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# 
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1794628"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check NSX DNS record in NSX policy API, all records are removed.
{
  "results": [],
  "result_count": 0,
  "sort_by": "display_name",
  "sort_ascending": true
}
  1. Annotate the Service with a hostname not matching any permitted DNS domains,
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl -n test1 annotate svc gateway-istio nsx.vmware.com/hostname="my-svc.invalid-zone.com"
service/gateway-istio annotated
root@4228083f110a51804d63bffc28b7c09f [ ~ ]# kubectl get svc -n test1 gateway-istio -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    nsx.vmware.com/hostname: my-svc.invalid-zone.com
  creationTimestamp: "2026-05-22T10:39:30Z"
  finalizers:
  - netoperator.vmware.com/service
  labels:
    gateway.istio.io/managed: istio.io-gateway-controller
    gateway.networking.k8s.io/gateway-class-name: istio
    gateway.networking.k8s.io/gateway-name: gateway
    service.route.lbapi.run.tanzu.vmware.com/gateway-name: gateway-istio
    service.route.lbapi.run.tanzu.vmware.com/gateway-namespace: test1
    service.route.lbapi.run.tanzu.vmware.com/type: direct
  name: gateway-istio
  namespace: test1
  ownerReferences:
  - apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    name: gateway
    uid: f9934710-5c19-459f-b38a-6beefa31f5b8
  resourceVersion: "1795871"
  uid: 58630181-4f48-4a34-9dd7-ce0dc075f3c0
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 172.24.184.123
  clusterIPs:
  - 172.24.184.123
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: PreferDualStack
  ports:
  - appProtocol: tcp
    name: status-port
    nodePort: 31957
    port: 15021
    protocol: TCP
    targetPort: 15021
  - appProtocol: http
    name: default
    nodePort: 32095
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    gateway.networking.k8s.io/gateway-name: gateway
  sessionAffinity: None
  type: LoadBalancer
status:
  conditions:
  - lastTransitionTime: "2026-05-22T11:14:50Z"
    message: hostname "my-svc.invalid-zone.com" does not match any allowed DNS domain
      in the namespace
    reason: DNSRecordFailed
    status: "False"
    type: Ready
  loadBalancer:
    ingress:
    - ip: 192.168.0.8
      ipMode: Proxy
  1. Check NSX DNS record in NSX policy API, no records are created
{
  "results": [],
  "result_count": 0,
  "sort_by": "display_name",
  "sort_ascending": true
}

@wenyingd
wenyingd requested a review from TaoZou1 May 6, 2026 04:59
@codecov-commenter

codecov-commenter commented May 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.51686% with 115 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.62%. Comparing base (934da6b) to head (8e17584).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
pkg/nsx/services/dns/recordservice.go 82.55% 26 Missing and 19 partials ⚠️
pkg/nsx/services/dns/contributing.go 85.84% 8 Missing and 7 partials ⚠️
pkg/nsx/services/dns/compare.go 84.00% 9 Missing and 3 partials ⚠️
pkg/nsx/services/dns/store.go 94.71% 6 Missing and 6 partials ⚠️
pkg/nsx/services/dns/zones.go 88.77% 6 Missing and 5 partials ⚠️
pkg/nsx/services/common/wrap.go 73.33% 2 Missing and 2 partials ⚠️
pkg/nsx/services/dns/cleanup.go 73.33% 2 Missing and 2 partials ⚠️
pkg/nsx/services/dns/initialize.go 88.57% 2 Missing and 2 partials ⚠️
pkg/clean/clean.go 71.42% 1 Missing and 1 partial ⚠️
pkg/nsx/services/dns/builder.go 97.46% 1 Missing and 1 partial ⚠️
... and 2 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1423      +/-   ##
==========================================
+ Coverage   77.00%   77.62%   +0.61%     
==========================================
  Files         156      169      +13     
  Lines       22205    23312    +1107     
==========================================
+ Hits        17100    18097     +997     
- Misses       3892     3954      +62     
- Partials     1213     1261      +48     
Flag Coverage Δ
unit-tests 77.62% <89.51%> (+0.61%) ⬆️
Files with missing lines Coverage Δ
pkg/nsx/client.go 93.42% <100.00%> (+0.11%) ⬆️
pkg/nsx/services/common/policy_tree.go 86.48% <100.00%> (+0.24%) ⬆️
pkg/nsx/services/common/types.go 100.00% <ø> (ø)
pkg/nsx/services/dns/errors.go 100.00% <100.00%> (ø)
pkg/nsx/services/dns/types.go 100.00% <100.00%> (ø)
pkg/third_party/externaldns/endpoint/endpoint.go 100.00% <100.00%> (ø)
pkg/util/utils.go 87.12% <100.00%> (+0.12%) ⬆️
pkg/clean/clean.go 86.88% <71.42%> (-0.95%) ⬇️
pkg/nsx/services/dns/builder.go 97.46% <97.46%> (ø)
pkg/third_party/externaldns/endpoint/utils.go 90.47% <90.47%> (ø)
... and 9 more

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wenyingd
wenyingd force-pushed the dns-only branch 8 times, most recently from 0825801 to 7f460a0 Compare May 12, 2026 02:16
@wenyingd
wenyingd force-pushed the dns-only branch 4 times, most recently from 944694e to d5459d9 Compare May 14, 2026 11:40
TaoZou1
TaoZou1 previously approved these changes May 15, 2026
Comment thread pkg/nsx/services/dns/recordservice.go
@TaoZou1

TaoZou1 commented May 15, 2026

Copy link
Copy Markdown
Contributor

Please add enough UT later.

@wenyingd
wenyingd force-pushed the dns-only branch 2 times, most recently from ebdc0d8 to 236e7bf Compare May 15, 2026 06:01
@wenyingd
wenyingd requested a review from TaoZou1 May 15, 2026 06:44
@wenyingd
wenyingd force-pushed the dns-only branch 2 times, most recently from 2593094 to c882eb5 Compare May 15, 2026 07:55
TaoZou1
TaoZou1 previously approved these changes May 15, 2026
@wenyingd

Copy link
Copy Markdown
Contributor Author

/e2e

@dantingl
dantingl requested a review from heypnus May 18, 2026 02:20
@heypnus

heypnus commented May 21, 2026

Copy link
Copy Markdown
Contributor

/e2e

@wenyingd
wenyingd force-pushed the dns-only branch 2 times, most recently from 54552ba to c5a41f0 Compare May 22, 2026 11:20
@wenyingd

Copy link
Copy Markdown
Contributor Author

@TaoZou1 @heypnus I removed the logic with realization state check for the NSX API is not ready yet.

Testing Done section is added.

@wenyingd
wenyingd requested a review from TaoZou1 May 25, 2026 02:01
Comment thread pkg/nsx/services/dns/contributing.go
Comment thread pkg/nsx/services/dns/zones.go
Comment thread pkg/nsx/services/dns/builder.go
Comment thread pkg/nsx/services/dns/contributing.go Outdated
Comment thread pkg/third_party/externaldns/endpoint/endpoint.go
Comment thread pkg/nsx/services/dns/builder.go Outdated
Comment thread pkg/nsx/services/dns/builder.go Outdated
- Implement DNSRecordService for NSX ProjectDnsRecord CRUD operations
- Validate hostnames against VPCNetworkConfiguration allowed DNS zones
- Wrap hostname-mismatch error as DNSZoneValidationError for accurate
  DNSRecordReady condition reporting
- Remove the realization state check before NSX API is ready
Comment thread pkg/nsx/services/dns/contributing.go
@wenyingd

Copy link
Copy Markdown
Contributor Author

/e2e

@wenyingd
wenyingd merged commit e07ae39 into vmware-tanzu:main May 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants