Relay server not working

[christianhuth]

The issue

My peers are able to connect to the Management Platform, but aren't able to reach each other. I assume that this is due to a broken relay server.

The symptoms

When I run netbird status --detail I can see, that only the STUN server is available:

Relays: 
  [stun:netbird.<my-domain>:3478] is Available
  [rels://netbird.<my-domain>:443] is Unavailable, reason: relay client not connected

The Stack

• Netbird Management Platform self-hosted; installed with your Chart v0.4.2 (I have added your Route templates locally as they aren't part of that release yet)
• Netbird Kubernetes Operator; installed with the official Chart v0.3.1
• Cilium as CNI
• Cilium for Gateway API (shared gateway)
• Cilium for LoadBalancer services

The variation of the Netbird Management Platform:

• Dashboard exposed through HTTPRoute
• Server exposed through HTTPRoute
• Server-Relay exposed through HTTPRoute
• Server-GRPC exposed through GRPCRoute
• STUN exposed through LoadBalancer service (shared IP with the Ingress Gateway)
• Everything uses the subdomain netbird. --> resolves to the IP of the STUN server as well
• All Routes protected through TLS certificates

The values.yaml

---
dashboard:
  config:
    mgmtApiEndpoint: https://netbird.<my-domain>
    mgmtGrpcApiEndpoint: https://netbird.<my-domain>
    authAuthority: "https://netbird.<my-domain>/oauth2"
    authClientId: "netbird-dashboard"
    authAudience: "netbird-dashboard"
  httpRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - path: { type: PathPrefix, value: / }
  image:
    repository: <my-private-registry>/netbirdio/dashboard
    tag: v2.37.1
oidc:
  enabled: true
  idpManager:
    enabled: true
    managerType: embedded
server:
  config:
    auth:
      dashboardRedirectURIs:
        - https://netbird.<my-domain>/nb-auth
        - https://netbird.<my-domain>/nb-silent-auth
      issuer: https://netbird.<my-domain>oauth2
    exposedAddress: https://<my-domain>:443
  grpcRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - method: { service: signalexchange.SignalExchange }
      - matches:
          - method: { service: management.ManagementService }
  httpRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - path: { type: PathPrefix, value: /api }
          - path: { type: PathPrefix, value: /oauth2 }
  image:
    repository: <my-private-registry>/netbirdio/netbird-server
    tag: 0.70.0
  relayHttpRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - path: { type: PathPrefix, value: /relay }
          - path: { type: PathPrefix, value: /ws-proxy }
  secrets:
    authSecret:
      secretName: netbird-server
    storeEncryptionKey:
      secretName: netbird-server
  stunService:
    annotations:
      lbipam.cilium.io/ips: "<my-public-ip>"
      lbipam.cilium.io/sharing-cross-namespace: "*"
      lbipam.cilium.io/sharing-key: "<a-secret-sharing-key>"
    type: LoadBalancer

Do you see any issue with my current setup?

[christianhuth]

I also followed the official docs to test and debug TURN connections: https://docs.netbird.io/selfhosted/troubleshooting#debugging-turn-connections

The results look like this:

IceGatheringState: complete
host   udp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:53002    N/A
host   tcp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:9        N/A
host   udp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:53003    N/A
host   tcp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:9        N/A
srflx  udp   10.0.0.13:62611                                     0.0.0.0:0
srflx  udp   10.0.0.13:49039                                     0.0.0.0:0
null   null  undefined:null                                      N/A

According to the docs there should be two relay entries that don't show up here.