Skip to content

Commit c7493d2

Browse files
TechassiTechassi
committed
docs: Add examples of generic database connections
1 parent 7e455ca commit c7493d2

1 file changed

Lines changed: 140 additions & 16 deletions

File tree

‎modules/ROOT/partials/release-notes/release-26.7.adoc‎

Lines changed: 140 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -64,26 +64,150 @@ This is an early version of the Hub and we're happy to receive feedback.
6464
* [[generic-db-26_7_0]]The SDP now provides a generic database connection mechanism, giving a consistent, typed way to configure connections to external SQL databases across operators.
6565
The same typed structure is used in every CRD, so it can be copied between stacklets.
6666
+
67-
--
68-
[WARNING]
69-
====
7067
*Breaking:* This replaces the previous free-form connection strings with a typed struct, and requires changes to your custom resources (and, in some cases, your Secrets):
7168

72-
* Apache Airflow: the `.clusterConfig.credentialsSecret` field is renamed to `.clusterConfig.credentialsSecretName`, and the database connection becomes a typed struct.
73-
Consult the xref:airflow:usage-guide/database-connections.adoc[database connections] documentation page for more details.
74-
Implemented in https://github.com/stackabletech/airflow-operator/pull/754[airflow-operator#754].
75-
* Apache Druid: the `metadataStorageDatabase` field is renamed to `metadataDatabase` and is now a typed struct.
76-
Implemented in https://github.com/stackabletech/druid-operator/pull/814[druid-operator#814].
77-
* Apache Hive: the connection becomes a typed struct; because the `database` field is removed, the `username` field must be transferred to the Secret referenced in `credentialsSecretName` under the `username` key before upgrading.
78-
Implemented in https://github.com/stackabletech/hive-operator/pull/674[hive-operator#674].
79-
* Apache Superset: the connection becomes a typed struct, and the operator now uses an internal Secret for the Superset `SECRET_KEY` (created automatically if missing).
80-
Consult the xref:superset:usage-guide/database-connections.adoc[database connections] documentation page for more details.
81-
Implemented in https://github.com/stackabletech/superset-operator/pull/722[superset-operator#722].
82-
====
69+
** *Apache Airflow:* The `.clusterConfig.credentialsSecret` field is renamed to `.clusterConfig.credentialsSecretName`, and the database connection becomes a typed struct.
70+
Consult the xref:airflow:usage-guide/database-connections.adoc[database connections] documentation page for more details.
71+
+
72+
--
73+
[source,yaml]
74+
----
75+
kind: AirflowCluster
76+
spec:
77+
clusterConfig:
78+
credentialsSecretName: airflow-admin-credentials
79+
metadataDatabase:
80+
postgresql:
81+
host: airflow-postgresql
82+
database: airflow
83+
credentialsSecretName: airflow-postgresql-credentials
84+
# Only needed when using celery workers (instead of Kubernetes executors)
85+
celeryExecutors:
86+
resultBackend:
87+
postgresql:
88+
host: airflow-postgresql
89+
database: airflow
90+
credentialsSecretName: airflow-postgresql-credentials
91+
broker:
92+
redis:
93+
host: airflow-redis-master
94+
credentialsSecretName: airflow-redis-credentials
95+
---
96+
apiVersion: v1
97+
kind: Secret
98+
metadata:
99+
name: airflow-admin-credentials
100+
type: Opaque
101+
stringData:
102+
adminUser.username: airflow
103+
# ...
104+
---
105+
# Only needed when using celery workers (instead of Kubernetes executors)
106+
apiVersion: v1
107+
kind: Secret
108+
metadata:
109+
name: airflow-postgresql-credentials
110+
stringData:
111+
username: airflow
112+
password: airflow
113+
---
114+
# Only needed when using celery workers (instead of Kubernetes executors)
115+
apiVersion: v1
116+
kind: Secret
117+
metadata:
118+
name: airflow-redis-credentials
119+
stringData:
120+
username: ""
121+
password: redis
122+
----
83123

84-
// TODO(verify): xref to the per-operator database-connection usage guides (docs checking).
85-
Tracked in https://github.com/stackabletech/issues/issues/238[issues#238].
124+
Implemented in https://github.com/stackabletech/airflow-operator/pull/754[airflow-operator#754].
125+
--
126+
** *Apache Druid:* The `metadataStorageDatabase` field is renamed to `metadataDatabase` and is now a typed struct.
127+
+
86128
--
129+
[source,yaml]
130+
----
131+
# PostgreSQL or MySQL
132+
kind: DruidCluster
133+
spec:
134+
clusterConfig:
135+
metadataDatabase:
136+
postgresql: # or mysql
137+
host: postgresql-druid
138+
database: druid
139+
credentialsSecretName: druid-db-credentials
140+
141+
# Derby
142+
kind: DruidCluster
143+
spec:
144+
clusterConfig:
145+
metadataDatabase:
146+
derby: {}
147+
----
148+
149+
Implemented in https://github.com/stackabletech/druid-operator/pull/814[druid-operator#814].
150+
--
151+
** *Apache Hive:* the connection becomes a typed struct; because the `database` field is removed, the `username` field must be transferred to the Secret referenced in `credentialsSecretName` under the `username` key before upgrading.
152+
+
153+
--
154+
[source,yaml]
155+
----
156+
kind: HiveCluster
157+
spec:
158+
clusterConfig:
159+
metadataDatabase:
160+
postgresql:
161+
host: postgresql
162+
database: hive
163+
credentialsSecretName: hive-credentials
164+
----
165+
166+
Implemented in https://github.com/stackabletech/hive-operator/pull/674[hive-operator#674].
167+
--
168+
** Apache Superset: the connection becomes a typed struct, and the operator now uses an internal Secret for the Superset `SECRET_KEY` (created automatically if missing).
169+
Consult the xref:superset:usage-guide/database-connections.adoc[database connections] documentation page for more details.
170+
+
171+
--
172+
[source,yaml]
173+
----
174+
kind: SupersetCluster
175+
spec:
176+
clusterConfig:
177+
credentialsSecret: superset-admin-credentials
178+
metadataDatabase:
179+
postgresql:
180+
host: superset-postgresql
181+
database: superset
182+
credentialsSecretName: superset-postgresql-credentials
183+
---
184+
apiVersion: v1
185+
kind: Secret
186+
metadata:
187+
name: superset-admin-credentials
188+
type: Opaque
189+
stringData:
190+
adminUser.username: admin
191+
adminUser.firstname: Superset
192+
adminUser.lastname: Admin
193+
adminUser.email: admin@superset.com
194+
adminUser.password: admin
195+
---
196+
apiVersion: v1
197+
kind: Secret
198+
metadata:
199+
name: superset-postgresql-credentials
200+
stringData:
201+
username: superset
202+
password: superset
203+
----
204+
205+
Implemented in https://github.com/stackabletech/superset-operator/pull/722[superset-operator#722].
206+
--
207+
208+
+
209+
Tracked in https://github.com/stackabletech/issues/issues/238[issues#238].
210+
87211
* Setting a custom client authentication method is now supported for Apache Airflow, Apache Superset and Apache Druid.
88212
Since Apache Druid 35 and 36, OIDC authentication can fail if the provider (for example Keycloak) advertises `private_key_jwt`, because Druid may attempt to use it.
89213
This feature lets you override the method so that OIDC authentication works again (this was a known issue in the 26.3.0 release).

0 commit comments

Comments
 (0)