Skip to content

Commit 801ca53

Browse files
sbernauersbernauer
authored
Merge branch 'main' into docs/demu-upgrade-issue-template
2 parents 1a59f11 + aaf28b3 commit 801ca53

14 files changed

Lines changed: 181 additions & 173 deletions

File tree

‎docs/modules/demos/images/end-to-end-security/keycloak_2.png‎

-3.45 KB
Loading

‎docs/modules/demos/images/end-to-end-security/keycloak_3.png‎

39.4 KB
Loading

‎docs/modules/demos/images/end-to-end-security/keycloak_4.png‎

24.2 KB
Loading

‎docs/modules/demos/images/end-to-end-security/keycloak_5.png‎

36.6 KB
Loading

‎docs/modules/demos/images/end-to-end-security/superset_1.png‎

7.86 KB
Loading

‎docs/modules/demos/images/end-to-end-security/superset_2.png‎

22.4 KB
Loading

‎docs/modules/demos/images/logging/logs.png‎

342 KB
Loading

‎docs/modules/demos/pages/end-to-end-security.adoc‎

Lines changed: 21 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -142,31 +142,27 @@ To list the installed Stackable services run the following command:
142142
[source,console]
143143
----
144144
$ stackablectl stacklet list
145-
┌───────────┬──────────────┬───────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────┐
146-
│ PRODUCT ┆ NAME ┆ NAMESPACE ┆ ENDPOINTS ┆ CONDITIONS │
147-
╞═══════════╪══════════════╪═══════════╪══════════════════════════════════════════════════════════════════════════════════════════════════════════════╪═════════════════════════════════╡
148-
│ hdfs ┆ hdfs ┆ default ┆ datanode-default-0-listener-data hdfs-datanode-default-0-listener.default.svc.cluster.local:9866 ┆ Available, Reconciling, Running │
149-
│ ┆ ┆ ┆ datanode-default-0-listener-https https://hdfs-datanode-default-0-listener.default.svc.cluster.local:9865 ┆ │
150-
│ ┆ ┆ ┆ datanode-default-0-listener-ipc hdfs-datanode-default-0-listener.default.svc.cluster.local:9867 ┆ │
151-
│ ┆ ┆ ┆ datanode-default-0-listener-metrics hdfs-datanode-default-0-listener.default.svc.cluster.local:8082 ┆ │
152-
│ ┆ ┆ ┆ namenode-default-0-https https://listener-hdfs-namenode-default-0.default.svc.cluster.local:9871 ┆ │
153-
│ ┆ ┆ ┆ namenode-default-0-metrics listener-hdfs-namenode-default-0.default.svc.cluster.local:8183 ┆ │
154-
│ ┆ ┆ ┆ namenode-default-0-rpc listener-hdfs-namenode-default-0.default.svc.cluster.local:8020 ┆ │
155-
│ ┆ ┆ ┆ namenode-default-1-https https://listener-hdfs-namenode-default-1.default.svc.cluster.local:9871 ┆ │
156-
│ ┆ ┆ ┆ namenode-default-1-metrics listener-hdfs-namenode-default-1.default.svc.cluster.local:8183 ┆ │
157-
│ ┆ ┆ ┆ namenode-default-1-rpc listener-hdfs-namenode-default-1.default.svc.cluster.local:8020 ┆ │
158-
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
159-
│ hive ┆ hive-iceberg ┆ default ┆ ┆ Available, Reconciling, Running │
160-
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
161-
│ opa ┆ opa ┆ default ┆ ┆ Available, Reconciling, Running │
162-
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
163-
│ superset ┆ superset ┆ default ┆ external-http http://172.18.0.2:30443 ┆ Available, Reconciling, Running │
164-
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
165-
│ trino ┆ trino ┆ default ┆ coordinator-metrics 172.18.0.2:32156 ┆ Available, Reconciling, Running │
166-
│ ┆ ┆ ┆ coordinator-https https://172.18.0.2:31604 ┆ │
167-
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
168-
│ zookeeper ┆ zookeeper ┆ default ┆ ┆ Available, Reconciling, Running │
169-
└───────────┴──────────────┴───────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────┘
145+
┌───────────┬──────────────┬───────────┬──────────────────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────┐
146+
│ PRODUCT ┆ NAME ┆ NAMESPACE ┆ ENDPOINTS ┆ CONDITIONS │
147+
╞═══════════╪══════════════╪═══════════╪══════════════════════════════════════════════════════════════════════════════════════════╪═════════════════════════════════╡
148+
│ hdfs ┆ hdfs ┆ default ┆ datanode-default-0-listener-data 172.18.0.2:31942 ┆ Available, Reconciling, Running │
149+
│ ┆ ┆ ┆ datanode-default-0-listener-https https://172.18.0.2:32525 ┆ │
150+
│ ┆ ┆ ┆ datanode-default-0-listener-ipc 172.18.0.2:32319 ┆ │
151+
│ ┆ ┆ ┆ namenode-default-0-https https://172.18.0.2:30658 ┆ │
152+
│ ┆ ┆ ┆ namenode-default-0-rpc 172.18.0.2:31140 ┆ │
153+
│ ┆ ┆ ┆ namenode-default-1-https https://172.18.0.2:31719 ┆ │
154+
│ ┆ ┆ ┆ namenode-default-1-rpc 172.18.0.2:30396 ┆ │
155+
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
156+
│ hive ┆ hive-iceberg ┆ default ┆ metastore-hive hive-iceberg-metastore.default.svc.cluster.local:9083 ┆ Available, Reconciling, Running │
157+
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
158+
│ opa ┆ opa ┆ default ┆ ┆ Available, Reconciling, Running │
159+
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
160+
│ superset ┆ superset ┆ default ┆ node-http http://172.18.0.2:32116 ┆ Available, Reconciling, Running │
161+
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
162+
│ trino ┆ trino ┆ default ┆ coordinator-https https://172.18.0.2:31154 ┆ Available, Reconciling, Running │
163+
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
164+
│ zookeeper ┆ zookeeper ┆ default ┆ server-zk zookeeper-server.default.svc.cluster.local:2282 ┆ Available, Reconciling, Running │
165+
└───────────┴──────────────┴───────────┴──────────────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────┘
170166
----
171167

172168
include::partial$instance-hint.adoc[]

‎stacks/data-lakehouse-iceberg-trino-spark/trino.yaml‎

Lines changed: 50 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,53 @@
1+
# For now, on K8s 1.35, TrinoCatalogs need to be deployed before the TrinoCluster
2+
# See: https://github.com/stackabletech/trino-operator/issues/854
3+
---
4+
apiVersion: trino.stackable.tech/v1alpha1
5+
kind: TrinoCatalog
6+
metadata:
7+
name: staging
8+
labels:
9+
trino: trino
10+
spec:
11+
connector:
12+
hive:
13+
metastore:
14+
configMap: hive
15+
s3:
16+
reference: minio
17+
---
18+
apiVersion: trino.stackable.tech/v1alpha1
19+
kind: TrinoCatalog
20+
metadata:
21+
name: lakehouse
22+
labels:
23+
trino: trino
24+
spec:
25+
connector:
26+
iceberg:
27+
metastore:
28+
configMap: hive-iceberg
29+
s3:
30+
reference: minio
31+
---
32+
apiVersion: trino.stackable.tech/v1alpha1
33+
kind: TrinoCatalog
34+
metadata:
35+
name: tpcds
36+
labels:
37+
trino: trino
38+
spec:
39+
connector:
40+
tpcds: {}
41+
---
42+
apiVersion: trino.stackable.tech/v1alpha1
43+
kind: TrinoCatalog
44+
metadata:
45+
name: tpch
46+
labels:
47+
trino: trino
48+
spec:
49+
connector:
50+
tpch: {}
151
---
252
apiVersion: trino.stackable.tech/v1alpha1
353
kind: TrinoCluster
@@ -62,54 +112,6 @@ type: kubernetes.io/opaque
62112
stringData:
63113
admin: "{{ trinoAdminPassword }}"
64114
---
65-
apiVersion: trino.stackable.tech/v1alpha1
66-
kind: TrinoCatalog
67-
metadata:
68-
name: staging
69-
labels:
70-
trino: trino
71-
spec:
72-
connector:
73-
hive:
74-
metastore:
75-
configMap: hive
76-
s3:
77-
reference: minio
78-
---
79-
apiVersion: trino.stackable.tech/v1alpha1
80-
kind: TrinoCatalog
81-
metadata:
82-
name: lakehouse
83-
labels:
84-
trino: trino
85-
spec:
86-
connector:
87-
iceberg:
88-
metastore:
89-
configMap: hive-iceberg
90-
s3:
91-
reference: minio
92-
---
93-
apiVersion: trino.stackable.tech/v1alpha1
94-
kind: TrinoCatalog
95-
metadata:
96-
name: tpcds
97-
labels:
98-
trino: trino
99-
spec:
100-
connector:
101-
tpcds: {}
102-
---
103-
apiVersion: trino.stackable.tech/v1alpha1
104-
kind: TrinoCatalog
105-
metadata:
106-
name: tpch
107-
labels:
108-
trino: trino
109-
spec:
110-
connector:
111-
tpch: {}
112-
---
113115
apiVersion: opa.stackable.tech/v1alpha1
114116
kind: OpaCluster
115117
metadata:

‎stacks/end-to-end-security/trino.yaml‎

Lines changed: 52 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,55 @@
1+
# For now, on K8s 1.35, TrinoCatalogs need to be deployed before the TrinoCluster
2+
# See: https://github.com/stackabletech/trino-operator/issues/854
3+
---
4+
apiVersion: trino.stackable.tech/v1alpha1
5+
kind: TrinoCatalog
6+
metadata:
7+
name: lakehouse
8+
labels:
9+
trino: trino
10+
spec:
11+
connector:
12+
iceberg:
13+
metastore:
14+
configMap: hive-iceberg
15+
hdfs:
16+
configMap: hdfs
17+
configOverrides:
18+
# HDFS configuration
19+
hive.hdfs.authentication.type: KERBEROS
20+
hive.hdfs.trino.principal: trino/trino.default.svc.cluster.local@KNAB.COM
21+
hive.hdfs.trino.keytab: /stackable/kerberos/keytab
22+
hive.hdfs.impersonation.enabled: "false"
23+
hive.hdfs.wire-encryption.enabled: "true"
24+
# HMS configuration
25+
hive.metastore.authentication.type: KERBEROS
26+
hive.metastore.client.principal: trino/trino.default.svc.cluster.local@KNAB.COM
27+
hive.metastore.client.keytab: /stackable/kerberos/keytab
28+
hive.metastore.service.principal: hive/hive-iceberg.default.svc.cluster.local@KNAB.COM
29+
hive.metastore.thrift.impersonation.enabled: "false"
30+
# By default, Hive views are executed with the RUN AS DEFINER security mode. Set the hive.hive-views.run-as-invoker catalog configuration property to true to use RUN AS INVOKER semantics.
31+
# However, this does *not* work for Iceberg catalogs :/ (I asked on the Trino slack: https://trinodb.slack.com/archives/CJ6UC075E/p1711449384648869)
32+
# hive.hive-views.run-as-invoker: "true"
33+
---
34+
apiVersion: trino.stackable.tech/v1alpha1
35+
kind: TrinoCatalog
36+
metadata:
37+
name: tpcds
38+
labels:
39+
trino: trino
40+
spec:
41+
connector:
42+
tpcds: {}
43+
---
44+
apiVersion: trino.stackable.tech/v1alpha1
45+
kind: TrinoCatalog
46+
metadata:
47+
name: tpch
48+
labels:
49+
trino: trino
50+
spec:
51+
connector:
52+
tpch: {}
153
---
254
apiVersion: trino.stackable.tech/v1alpha1
355
kind: TrinoCluster
@@ -67,56 +119,6 @@ spec:
67119
default:
68120
replicas: 1
69121
---
70-
apiVersion: trino.stackable.tech/v1alpha1
71-
kind: TrinoCatalog
72-
metadata:
73-
name: lakehouse
74-
labels:
75-
trino: trino
76-
spec:
77-
connector:
78-
iceberg:
79-
metastore:
80-
configMap: hive-iceberg
81-
hdfs:
82-
configMap: hdfs
83-
configOverrides:
84-
# HDFS configuration
85-
hive.hdfs.authentication.type: KERBEROS
86-
hive.hdfs.trino.principal: trino/trino.default.svc.cluster.local@KNAB.COM
87-
hive.hdfs.trino.keytab: /stackable/kerberos/keytab
88-
hive.hdfs.impersonation.enabled: "false"
89-
hive.hdfs.wire-encryption.enabled: "true"
90-
# HMS configuration
91-
hive.metastore.authentication.type: KERBEROS
92-
hive.metastore.client.principal: trino/trino.default.svc.cluster.local@KNAB.COM
93-
hive.metastore.client.keytab: /stackable/kerberos/keytab
94-
hive.metastore.service.principal: hive/hive-iceberg.default.svc.cluster.local@KNAB.COM
95-
hive.metastore.thrift.impersonation.enabled: "false"
96-
# By default, Hive views are executed with the RUN AS DEFINER security mode. Set the hive.hive-views.run-as-invoker catalog configuration property to true to use RUN AS INVOKER semantics.
97-
# However, this does *not* work for Iceberg catalogs :/ (I asked on the Trino slack: https://trinodb.slack.com/archives/CJ6UC075E/p1711449384648869)
98-
# hive.hive-views.run-as-invoker: "true"
99-
---
100-
apiVersion: trino.stackable.tech/v1alpha1
101-
kind: TrinoCatalog
102-
metadata:
103-
name: tpcds
104-
labels:
105-
trino: trino
106-
spec:
107-
connector:
108-
tpcds: {}
109-
---
110-
apiVersion: trino.stackable.tech/v1alpha1
111-
kind: TrinoCatalog
112-
metadata:
113-
name: tpch
114-
labels:
115-
trino: trino
116-
spec:
117-
connector:
118-
tpch: {}
119-
---
120122
apiVersion: v1
121123
kind: Secret
122124
metadata:

0 commit comments

Comments
 (0)