Facets-cloud · anshulsao · Jul 15, 2026
diff --git a/modules/clickhouse/k8s/1.0/README.md b/modules/clickhouse/k8s/1.0/README.md
@@ -0,0 +1,91 @@
+# clickhouse / k8s / 1.0
+
+Deploys a ClickHouse cluster as a `ClickHouseInstallation` (CHI) custom
+resource, applied through a small local Helm chart under `chart/` (no plan-time
+CRD schema check). Inputs `clickhouse_operator` (CRDs) and `clickhouse_keeper`
+(coordination) are declared dependencies, so Facets deploys them first.
+
+- **CRD apiVersion / kind**: `clickhouse.altinity.com/v1` /
+  `ClickHouseInstallation`.
+- **Keeper wiring**: `configuration.zookeeper.nodes[0]` is populated from
+  `var.inputs.clickhouse_keeper.attributes.service_host` and `.port`.
+- **Service DNS (`service_host` output)**:
+  `clickhouse-<cluster_name>.<namespace>.svc.cluster.local`. Altinity's CHI
+  service is `clickhouse-<chi-name>`; the CHI name here is `<cluster_name>`.
+  Verify against your operator version and adjust `outputs.tf` if it differs.
+- **Ports**: HTTP `8123`, native TCP `9000`.
+
+## Users (`users_json`)
+
+`users_json` is a JSON array. Each element:
+
+```json
+{
+  "name": "app",
+  "password_secret": "clickhouse-app-password",
+  "password_sha256_hex": "<optional precomputed sha256 hex>",
+  "networks": ["::/0"],
+  "profile": "default"
+}
+```
+
+- **No plaintext passwords are hardcoded.** `password_secret` is the *name* of a
+  Facets secret intended to hold the credential; wire it operationally (e.g. via
+  a dedicated `--secret` spec field / external-secret) — the module does not
+  resolve secret values from inside a JSON string.
+- `password_sha256_hex` (optional) renders directly into the CHI as
+  `<name>/password_sha256_hex`. Omit both fields for a passwordless user.
+- `networks` → `<name>/networks/ip`; `profile` → `<name>/profile`.
+
+`settings_json` is a JSON object merged into `configuration.settings`.
+
+## Dedicated node-pool placement (`node_pool` input)
+
+Optional input `node_pool` (`@facets/kubernetes_nodepool`, `optional: true`).
+Wire it to pin ClickHouse onto a dedicated, tainted node pool; leave it
+unwired and the module schedules anywhere (no `nodeSelector`/`tolerations`
+rendered). Placement is read defensively in `main.tf`:
+
+- `node_selector` — a **merge** of the output's custom `node_selector` and the
+  always-present GKE pool label:
+  `merge(try(var.inputs.node_pool.attributes.node_selector, {}), local.node_pool_name != "" ? { "cloud.google.com/gke-nodepool" = local.node_pool_name } : {})`
+  → `spec.templates.podTemplates[].spec.nodeSelector` (guarded by `{{- if }}`).
+- `taints` ← `try(var.inputs.node_pool.attributes.taints, [])`, each
+  `{key,value,effect}` mapped to a toleration `{key,value,effect,operator:"Equal"}`
+  → `spec.templates.podTemplates[].spec.tolerations` (guarded by `{{- if }}`).
+
+**Why the GKE pool label (eu-prod pilot finding):** the node_pool output's
+custom `node_selector` arrived **empty** at apply, so pods landed on regular
+nodes. Every node in a GKE pool reliably carries
+`cloud.google.com/gke-nodepool=<pool-name>`, and the output always exposes
+`node_pool_name` (e.g. `clickhouse-node-pool`) — so the module pins on that
+label. Any custom labels from the output still merge in alongside it. When
+`node_pool` is unwired, `node_pool_name` is `""`, the merge yields `{}`, and no
+`nodeSelector` renders.
+
+The `kubernetes_node_pool/gcp/1.0` output emits `taints[].effect` already in
+Kubernetes form (`NoSchedule`/`PreferNoSchedule`/`NoExecute`), so the effect is
+passed through verbatim — no `NO_SCHEDULE`→`NoSchedule` normalization is needed.
+`variables.tf` types `node_selector` as `map(string)` (not the schema-default
+empty `object({})`, which silently drops label keys during Terraform coercion).
+If a future `raptor module` mutation regenerates `variables.tf`, re-widen
+`node_selector` back to `map(string)`.
+
+### Replica spread (operator-native podDistribution)
+
+Replica spreading is delegated to the Altinity operator via `podDistribution`
+(`ClickHouseAntiAffinity` + `ReplicaAntiAffinity`) — **not** a raw
+`podAntiAffinity` in the podTemplate.
+
+`podDistribution` is a **direct child of the podTemplate entry** —
+`spec.templates.podTemplates[].podDistribution`, a sibling of that entry's
+`spec:` — matching Altinity's `chi-examples/02-templates-01` reference. The
+podTemplate is applied because the CHI references it via
+`spec.defaults.templates.podTemplate: default` (+ `dataVolumeClaimTemplate`).
+
+- **eu-prod pilot findings:** (1) a raw `podAntiAffinity` injected into the CHI
+  podTemplate `spec` made the operator loop the StatefulSet `Update→Recreate` and
+  stall with no STS/pod created. (2) Placing `podDistribution` under the cluster
+  entry (`configuration.clusters[].templates`) also broke the STS reconcile — it
+  belongs on the podTemplate. Moving it there lets the operator manage the spread
+  as part of its own reconcile, no churn.
diff --git a/modules/clickhouse/k8s/1.0/chart/Chart.yaml b/modules/clickhouse/k8s/1.0/chart/Chart.yaml
@@ -0,0 +1,6 @@
+apiVersion: v2
+name: clickhouse-chi
+description: Renders an Altinity ClickHouseInstallation (CHI) custom resource
+type: application
+version: 1.0.0
+appVersion: "1.0"
diff --git a/modules/clickhouse/k8s/1.0/chart/templates/chi.yaml b/modules/clickhouse/k8s/1.0/chart/templates/chi.yaml
@@ -0,0 +1,95 @@
+apiVersion: "clickhouse.altinity.com/v1"
+kind: "ClickHouseInstallation"
+metadata:
+  name: {{ .Values.clusterName | quote }}
+  namespace: {{ .Values.namespace | quote }}
+spec:
+  # Cluster DNS domain override. The operator hardcodes cluster.local when
+  # constructing replica host FQDNs; on clusters with a custom DNS domain (some
+  # GKE clusters use the env name, not cluster.local) that yields unresolvable
+  # hosts and the CHI deadlocks on replica health-checks. namespaceDomainPattern
+  # makes the operator build <pod>.<ns>.svc.<clusterDomain> instead.
+  namespaceDomainPattern: "%s.svc.{{ .Values.clusterDomain }}"
+  configuration:
+    {{- if .Values.zookeeper.host }}
+    zookeeper:
+      nodes:
+        - host: {{ .Values.zookeeper.host | quote }}
+          port: {{ int .Values.zookeeper.port }}
+    {{- end }}
+    clusters:
+      - name: {{ .Values.clusterName | quote }}
+        layout:
+          shardsCount: {{ int .Values.shards }}
+          replicasCount: {{ int .Values.replicas }}
+    {{- if .Values.users }}
+    users:
+      {{- range .Values.users }}
+      {{ .name }}/profile: {{ .profile | default "default" | quote }}
+      {{- if .password_sha256_hex }}
+      {{ .name }}/password_sha256_hex: {{ .password_sha256_hex | quote }}
+      {{- end }}
+      {{- if .networks }}
+      {{ .name }}/networks/ip:
+        {{- range .networks }}
+        - {{ . | quote }}
+        {{- end }}
+      {{- end }}
+      {{- end }}
+    {{- end }}
+    {{- if .Values.settings }}
+    settings:
+      {{- range $k, $v := .Values.settings }}
+      {{ $k }}: {{ $v | quote }}
+      {{- end }}
+    {{- end }}
+  defaults:
+    templates:
+      podTemplate: default
+      dataVolumeClaimTemplate: default
+  templates:
+    podTemplates:
+      - name: default
+        # Operator-native replica spreading. podDistribution is a direct child of
+        # the podTemplate entry (sibling of spec:), NOT under the cluster's
+        # templates — putting it on the cluster broke the STS reconcile. A raw
+        # podAntiAffinity in the podTemplate spec also caused STS churn; this lets
+        # Altinity manage the spread (across hosts, then across replicas) natively.
+        podDistribution:
+          - type: ClickHouseAntiAffinity
+          - type: ReplicaAntiAffinity
+        spec:
+          {{- if .Values.nodeSelector }}
+          nodeSelector:
+            {{- range $k, $v := .Values.nodeSelector }}
+            {{ $k }}: {{ $v | quote }}
+            {{- end }}
+          {{- end }}
+          {{- if .Values.tolerations }}
+          tolerations:
+            {{- range .Values.tolerations }}
+            - key: {{ .key | quote }}
+              value: {{ .value | quote }}
+              effect: {{ .effect | quote }}
+              operator: {{ .operator | default "Equal" | quote }}
+            {{- end }}
+          {{- end }}
+          containers:
+            - name: clickhouse
+              image: {{ .Values.image | quote }}
+              resources:
+                requests:
+                  cpu: {{ .Values.resources.cpu | quote }}
+                  memory: {{ .Values.resources.memory | quote }}
+                limits:
+                  cpu: {{ .Values.resources.cpu | quote }}
+                  memory: {{ .Values.resources.memory | quote }}
+    volumeClaimTemplates:
+      - name: default
+        spec:
+          storageClassName: {{ .Values.storageClass | quote }}
+          accessModes:
+            - ReadWriteOnce
+          resources:
+            requests:
+              storage: {{ .Values.storageSize | quote }}
diff --git a/modules/clickhouse/k8s/1.0/chart/values.yaml b/modules/clickhouse/k8s/1.0/chart/values.yaml
@@ -0,0 +1,26 @@
+# Defaults are overridden by the Terraform helm_release values block.
+clusterName: default
+namespace: clickhouse
+image: clickhouse/clickhouse-server:24.8
+shards: 1
+replicas: 3
+storageClass: premium-rwo
+storageSize: 100Gi
+# Kubernetes cluster DNS domain (overridden by the Terraform values block).
+clusterDomain: cluster.local
+resources:
+  cpu: "1"
+  memory: "2Gi"
+# zookeeper (ClickHouse Keeper) coordination endpoint
+zookeeper:
+  host: ""
+  port: 2181
+# users: list of {name, password_sha256_hex?, networks: [..], profile}
+users: []
+# settings: map of clickhouse server settings
+settings: {}
+# Dedicated node-pool placement (populated from the optional node_pool input).
+# nodeSelector: map of labels; tolerations: list of {key,value,effect,operator}.
+# Both empty by default so an unwired node_pool renders no placement.
+nodeSelector: {}
+tolerations: []
diff --git a/modules/clickhouse/k8s/1.0/facets.yaml b/modules/clickhouse/k8s/1.0/facets.yaml
@@ -0,0 +1,153 @@
+# ╔═══════════════════════════════════════════════════════════╗
+# ║ MANAGED BY: raptor module — DO NOT EDIT DIRECTLY         ║
+# ║                                                          ║
+# ║ Modify via:                                              ║
+# ║   raptor module set-spec --add <field>                   ║
+# ║   raptor module add-input / remove-input                 ║
+# ║   raptor module add-output / remove-output               ║
+# ║   raptor module show | raptor module validate            ║
+# ╚═══════════════════════════════════════════════════════════╝
+intent: clickhouse
+flavor: k8s
+version: "1.0"
+description: ClickHouse cluster (ClickHouseInstallation) on k8s via Altinity operator
+clouds:
+    - kubernetes
+    - gcp
+inputs:
+    clickhouse_keeper:
+        type: '@facets/clickhouse-keeper'
+    clickhouse_operator:
+        type: '@facets/clickhouse-operator'
+    kubernetes_details:
+        type: '@facets/kubernetes-details'
+        providers:
+            - helm
+            - kubernetes
+    node_pool:
+        type: '@facets/kubernetes_nodepool'
+outputs:
+    default:
+        type: '@facets/clickhouse'
+spec:
+    properties:
+        clickhouse_version:
+            default: "24.8"
+            description: ClickHouse server image tag (an LTS tag is recommended, e.g. 24.8)
+            maxLength: 30
+            minLength: 1
+            title: ClickHouse Version
+            type: string
+        cluster_domain:
+            default: cluster.local
+            description: Kubernetes cluster DNS domain. Override per-env when the cluster is not the default cluster.local (some GKE clusters use a custom domain e.g. the env name).
+            maxLength: 253
+            minLength: 1
+            title: Cluster DNS Domain
+            type: string
+        cluster_name:
+            default: default
+            description: Logical ClickHouse cluster name (used in the CHI layout)
+            maxLength: 40
+            minLength: 1
+            pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
+            title: Cluster Name
+            type: string
+        namespace:
+            default: clickhouse
+            description: Kubernetes namespace to deploy the ClickHouse cluster into
+            maxLength: 63
+            minLength: 1
+            pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
+            title: Namespace
+            type: string
+        replicas:
+            default: 3
+            description: Number of replicas per shard
+            maximum: 20
+            minimum: 1
+            title: Replicas
+            type: number
+        resources:
+            properties:
+                cpu:
+                    default: "1"
+                    description: CPU request and limit per ClickHouse pod (e.g. 1)
+                    maxLength: 16
+                    minLength: 1
+                    title: CPU Request/Limit
+                    type: string
+                memory:
+                    default: 2Gi
+                    description: Memory request and limit per ClickHouse pod (e.g. 2Gi)
+                    maxLength: 16
+                    minLength: 1
+                    title: Memory Request/Limit
+                    type: string
+            title: Resources
+            type: object
+        settings_json:
+            default: '{}'
+            description: JSON object of ClickHouse server settings merged into configuration.settings (e.g. {"max_concurrent_queries":"200"})
+            title: Settings (JSON)
+            type: string
+            x-ui-editor: true
+            x-ui-editor-language: json
+        shards:
+            default: 1
+            description: Number of shards in the cluster
+            maximum: 100
+            minimum: 1
+            title: Shards
+            type: number
+        storage_class:
+            default: premium-rwo
+            description: StorageClass for ClickHouse persistent volumes
+            maxLength: 63
+            minLength: 1
+            title: Storage Class
+            type: string
+        storage_size:
+            default: 100Gi
+            description: Persistent volume size per ClickHouse pod (e.g. 100Gi)
+            maxLength: 16
+            minLength: 2
+            title: Storage Size
+            type: string
+        users_json:
+            default: '[]'
+            description: 'JSON array of ClickHouse users. Each item: {name, password_secret?, networks (list of CIDRs/host regex), profile}. password_secret is the NAME of a Facets secret holding the SHA256 password; leave empty for a passwordless user. Example: [{"name":"app","password_secret":"clickhouse-app-password","networks":["::/0"],"profile":"default"}]'
+            title: Users (JSON)
+            type: string
+            x-ui-editor: true
+            x-ui-editor-language: json
+    type: object
+    x-ui-order:
+        - cluster_name
+        - namespace
+        - clickhouse_version
+        - shards
+        - replicas
+        - storage_size
+        - storage_class
+        - resources
+        - users_json
+        - settings_json
+        - cluster_domain
+sample:
+    flavor: k8s
+    kind: clickhouse
+    spec:
+        clickhouse_version: "24.8"
+        cluster_domain: cluster.local
+        cluster_name: default
+        namespace: clickhouse
+        replicas: 3
+        resources:
+            cpu: "1"
+        settings_json: '{}'
+        shards: 1
+        storage_class: premium-rwo
+        storage_size: 100Gi
+        users_json: '[]'
+    version: "1.0"