feat: Integrate listener operator (#731)

* add integration test

* fix dns issue

* update discovery cm

* fix rustdoc

* create headless service name in function

* update integration tests with new headless service name

* add external-access test

* add permissions on listeners for service account

* fix authorizer test

* move listenerClass to roleConfig

* add changelog entry

* rename headless service

* fix tls test

* fix docs

* remove unnecessary listener from test

* create separate headless services and fix discovery cm

* fix rustdocs

* use listener scope for tls

Author: labrenbe

CHANGELOG.md

@@ -11,6 +11,7 @@ All notable changes to this project will be documented in this file.
   - Use `--file-log-rotation-period` (or `FILE_LOG_ROTATION_PERIOD`) to configure the frequency of rotation.
   - Use `--console-log-format` (or `CONSOLE_LOG_FORMAT`) to set the format to `plain` (default) or `json`.
 - Add support for `33.0.0` ([#722]).
+- Add Listener support for Druid ([#731]).
 
 ### Changed
 
@@ -52,6 +53,7 @@ All notable changes to this project will be documented in this file.
 [#721]: https://github.com/stackabletech/druid-operator/pull/721
 [#722]: https://github.com/stackabletech/druid-operator/pull/722
 [#725]: https://github.com/stackabletech/druid-operator/pull/725
+[#731]: https://github.com/stackabletech/druid-operator/pull/731
 
 ## [25.3.0] - 2025-03-21
 

deploy/helm/druid-operator/crds/crds.yaml

@@ -245,11 +245,15 @@ spec:
                       x-kubernetes-preserve-unknown-fields: true
                     roleConfig:
                       default:
+                        listenerClass: cluster-internal
                         podDisruptionBudget:
                           enabled: true
                           maxUnavailable: null
                       description: This is a product-agnostic RoleConfig, which is sufficient for most of the products.
                       properties:
+                        listenerClass:
+                          default: cluster-internal
+                          type: string
                         podDisruptionBudget:
                           default:
                             enabled: true
@@ -879,19 +883,6 @@ spec:
                               type: string
                           type: object
                       type: object
-                    listenerClass:
-                      default: cluster-internal
-                      description: |-
-                        This field controls which type of Service the Operator creates for this DruidCluster:
-
-                        * `cluster-internal`: Use a ClusterIP service * `external-unstable`: Use a NodePort service * `external-stable`: Use a LoadBalancer service
-
-                        This is a temporary solution with the goal to keep yaml manifests forward compatible. In the future, this setting will control which [ListenerClass](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass.html) will be used to expose the service, and ListenerClass names will stay the same, allowing for a non-breaking change.
-                      enum:
-                        - cluster-internal
-                        - external-unstable
-                        - external-stable
-                      type: string
                     metadataStorageDatabase:
                       description: Druid requires an SQL database to store metadata into. Specify connection information here.
                       properties:
@@ -1182,11 +1173,15 @@ spec:
                       x-kubernetes-preserve-unknown-fields: true
                     roleConfig:
                       default:
+                        listenerClass: cluster-internal
                         podDisruptionBudget:
                           enabled: true
                           maxUnavailable: null
                       description: This is a product-agnostic RoleConfig, which is sufficient for most of the products.
                       properties:
+                        listenerClass:
+                          default: cluster-internal
+                          type: string
                         podDisruptionBudget:
                           default:
                             enabled: true
@@ -2720,11 +2715,15 @@ spec:
                       x-kubernetes-preserve-unknown-fields: true
                     roleConfig:
                       default:
+                        listenerClass: cluster-internal
                         podDisruptionBudget:
                           enabled: true
                           maxUnavailable: null
                       description: This is a product-agnostic RoleConfig, which is sufficient for most of the products.
                       properties:
+                        listenerClass:
+                          default: cluster-internal
+                          type: string
                         podDisruptionBudget:
                           default:
                             enabled: true

deploy/helm/druid-operator/templates/roles.yaml

@@ -84,6 +84,17 @@ rules:
       - customresourcedefinitions
     verbs:
       - get
+  - apiGroups:
+      - listeners.stackable.tech
+    resources:
+      - listeners
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - watch
   - apiGroups:
       - events.k8s.io
     resources:

docs/modules/druid/examples/getting_started/druid.yaml

@@ -7,7 +7,6 @@ spec:
   image:
     productVersion: 33.0.0
   clusterConfig:
-    listenerClass: external-stable # This exposes your Stacklet outside of Kubernetes. Remove this configuration if this is not desired
     zookeeperConfigMapName: simple-druid-znode
     deepStorage:
       hdfs:
@@ -23,10 +22,14 @@ spec:
     roleGroups:
       default:
         replicas: 1
+    roleConfig:
+      listenerClass: external-stable # This exposes this role outside of Kubernetes. Remove this configuration if this is not desired
   coordinators:
     roleGroups:
       default:
         replicas: 1
+    roleConfig:
+      listenerClass: external-stable # This exposes this role outside of Kubernetes. Remove this configuration if this is not desired
   historicals:
     roleGroups:
       default:
@@ -39,6 +42,8 @@ spec:
     roleGroups:
       default:
         replicas: 1
+    roleConfig:
+      listenerClass: external-stable # This exposes this role outside of Kubernetes. Remove this configuration if this is not desired
 ---
 apiVersion: v1
 kind: Secret

docs/modules/druid/examples/getting_started/druid.yaml.j2

@@ -7,7 +7,6 @@ spec:
   image:
     productVersion: 33.0.0
   clusterConfig:
-    listenerClass: external-stable # This exposes your Stacklet outside of Kubernetes. Remove this configuration if this is not desired
     zookeeperConfigMapName: simple-druid-znode
     deepStorage:
       hdfs:
@@ -23,10 +22,14 @@ spec:
     roleGroups:
       default:
         replicas: 1
+    roleConfig:
+      listenerClass: external-stable # This exposes your Stacklet outside of Kubernetes. Remove this configuration if this is not desired
   coordinators:
     roleGroups:
       default:
         replicas: 1
+    roleConfig:
+      listenerClass: external-stable # This exposes your Stacklet outside of Kubernetes. Remove this configuration if this is not desired
   historicals:
     roleGroups:
       default:
@@ -39,6 +42,8 @@ spec:
     roleGroups:
       default:
         replicas: 1
+    roleConfig:
+      listenerClass: external-stable # This exposes your Stacklet outside of Kubernetes. Remove this configuration if this is not desired
 ---
 apiVersion: v1
 kind: Secret

docs/modules/druid/pages/usage-guide/listenerclass.adoc

@@ -2,20 +2,23 @@
 :description: Configure Apache Druid service exposure using ListenerClass to control service types: cluster-internal, external-unstable, or external-stable.
 
 Apache Druid offers a web UI and an API, both are exposed by the `router` role.
-Other roles also expose API endpoints such as the `broker` and `coordinator`.
-The Operator deploys services called `<name>-<role>` (where `<name>` is the name of the DruidCluster and `<role>` is the role for the Service) through which Druid can be reached.
+The `broker` and `coordinator` roles expose additional API endpoints.
 
-These services can have three different types: `cluster-internal`, `external-unstable` and `external-stable`.
-Read more about the types in the xref:concepts:service-exposition.adoc[service exposition] documentation at platform level.
-
-This is how the listener class is configured:
+The operator deploys a xref:listener-operator:listener.adoc[Listener] for the `router`, `broker` and `coordinator` roles.
+The Listener defaults to only being accessible from within the Kubernetes cluster, but this can be changed by configuring a ListenerClass
+for specific roles:
 
 [source,yaml]
 ----
 spec:
-  clusterConfig:
-    listenerClass: cluster-internal  # <1>
+  brokers:
+    roleConfig:
+      listenerClass: cluster-internal  # <1>
+  coordinators:
+    roleConfig:
+      listenerClass: cluster-internal  # <1>
+  routers:
+    roleConfig:
+      listenerClass: external-unstable  # <1>
 ----
-<1> The default `cluster-internal` setting.
-
-This setting affects all role Services at once.
+<1> Specify one of `external-stable`, `external-unstable`, `cluster-internal` or a custom ListenerClass (the default setting is `cluster-internal`).