|
2 | 2 | :description: The Stackable Operator for Apache Superset is a Kubernetes operator that can manage Apache Superset clusters. Learn about its features, resources, dependencies and demos, and see the list of supported Superset versions. |
3 | 3 | :keywords: Stackable Operator, Apache Superset, Kubernetes, operator, data science, data exploration, SQL, engineer, big data, CRD, StatefulSet, ConfigMap, Service, Druid, Trino, S3, demo, version |
4 | 4 |
|
5 | | -The Stackable Operator for Apache Superset is an operator that can deploy and manage https://superset.apache.org/[Apache Superset] clusters on Kubernetes. Superset is a data exploration and visualization tool that connects to data sources via SQL. Store your data in Apache Druid or Trino, and manage your Druid and Trino instances with the Stackable Operators for xref:druid:index.adoc[Apache Druid] or xref:trino:index.adoc[Trino]. This operator helps you manage your Superset instances on Kubernetes efficiently. |
| 5 | +The Stackable Operator for Apache Superset is an operator that can deploy and manage https://superset.apache.org/[Apache |
| 6 | +Superset] clusters on Kubernetes. Superset is a data exploration and visualization tool that connects to data sources |
| 7 | +via SQL. Store your data in Apache Druid or Trino, and manage your Druid and Trino instances with the Stackable |
| 8 | +Operators for xref:druid:index.adoc[Apache Druid] or xref:trino:index.adoc[Trino]. This operator helps you manage your |
| 9 | +Superset instances on Kubernetes efficiently. |
6 | 10 |
|
7 | 11 | == Getting started |
8 | 12 |
|
9 | | -Get started using Superset with Stackable Operator by following the xref:getting_started/index.adoc[]. It guides you through installing the Operator alongside a PostgreSQL database, connecting to your Superset instance and analyzing some preloaded example data. |
| 13 | +Get started using Superset with Stackable Operator by following the xref:getting_started/index.adoc[]. It guides you |
| 14 | +through installing the Operator alongside a PostgreSQL database, connecting to your Superset instance and analyzing some |
| 15 | +preloaded example data. |
10 | 16 |
|
11 | 17 | == Resources |
12 | 18 |
|
13 | | -The Operator manages three https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/[custom resources]: The _SupersetCluster_, _SupersetDB_ and _DruidConnection_. It creates a number of different Kubernetes resources based on the custom resources. |
| 19 | +The Operator manages three https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/[custom |
| 20 | +resources]: The _SupersetCluster_, _SupersetDB_ and _DruidConnection_. It creates a number of different Kubernetes |
| 21 | +resources based on the custom resources. |
14 | 22 |
|
15 | 23 | === Custom resources |
16 | 24 |
|
17 | | -The SupersetCluster is the main resource for the configuration of the Superset instance. The resource defines only one xref:concepts:roles-and-role-groups.adoc[role], the `node`. The various configuration options are explained in the xref:usage-guide/index.adoc[]. It helps you tune your cluster to your needs by configuring xref:usage-guide/storage-resource-configuration.adoc[resource usage], xref:usage-guide/security.adoc[security], xref:usage-guide/logging.adoc[logging] and more. |
| 25 | +The SupersetCluster is the main resource for the configuration of the Superset instance. The resource defines only one |
| 26 | +xref:concepts:roles-and-role-groups.adoc[role], the `node`. The various configuration options are explained in the |
| 27 | +xref:usage-guide/index.adoc[]. It helps you tune your cluster to your needs by configuring |
| 28 | +xref:usage-guide/storage-resource-configuration.adoc[resource usage], xref:usage-guide/security.adoc[security], |
| 29 | +xref:usage-guide/logging.adoc[logging] and more. |
18 | 30 |
|
19 | | -When a SupersetCluster is first deployed, a SupersetDB resource is created. The SupersetDB resource is a wrapper resource for the SQL database that is used by Superset for its metadata. The resource contains some configuration but also keeps track of whether the database has been initialized or not. It is not deleted automatically if a SupersetCluster is deleted, and so can be reused. |
| 31 | +When a SupersetCluster is first deployed, a SupersetDB resource is created. The SupersetDB resource is a wrapper |
| 32 | +resource for the SQL database that is used by Superset for its metadata. The resource contains some configuration but |
| 33 | +also keeps track of whether the database has been initialized or not. It is not deleted automatically if a |
| 34 | +SupersetCluster is deleted, and so can be reused. |
20 | 35 |
|
21 | | -DruidConnection resources link a Superset and Druid instance. It lets you define this connection in the familiar way of deploying a resource (instead of configuring the connection via the Superset UI or API). The operator configures the connection between Druid and the Superset instance. |
| 36 | +DruidConnection resources link a Superset and Druid instance. It lets you define this connection in the familiar way of |
| 37 | +deploying a resource (instead of configuring the connection via the Superset UI or API). The operator configures the |
| 38 | +connection between Druid and the Superset instance. |
22 | 39 |
|
23 | 40 | === Kubernetes resources |
24 | 41 |
|
25 | 42 | Based on the custom resources you define, the Operator creates ConfigMaps, StatefulSets and Services. |
26 | 43 |
|
27 | 44 | image::superset_overview.drawio.svg[A diagram depicting the Kubernetes resources created by the operator] |
28 | 45 |
|
29 | | -The diagram above depicts all the Kubernetes resources created by the operator, and how they relate to each other. The Jobs created for the SupersetDB and DruidConnnection resources are not shown. |
| 46 | +The diagram above depicts all the Kubernetes resources created by the operator, and how they relate to each other. The |
| 47 | +Jobs created for the SupersetDB and DruidConnnection resources are not shown. |
30 | 48 |
|
31 | | -For every xref:concepts:roles-and-role-groups.adoc#_role_groups[role group] you define, the Operator creates a StatefulSet with the amount of replicas defined in the RoleGroup. Every Pod in the StatefulSet has two containers: the main container running Superset and a sidecar container gathering metrics for xref:operators:monitoring.adoc[]. The Operator creates a Service for the `node` role as well as a single service per role group. |
| 49 | +For every xref:concepts:roles-and-role-groups.adoc#_role_groups[role group] you define, the Operator creates a |
| 50 | +StatefulSet with the amount of replicas defined in the RoleGroup. Every Pod in the StatefulSet has two containers: the |
| 51 | +main container running Superset and a sidecar container gathering metrics for xref:operators:monitoring.adoc[]. The |
| 52 | +Operator creates a Service for the `node` role as well as a single service per role group. |
32 | 53 |
|
33 | | -ConfigMaps are created, one per RoleGroup and also one for the SupersetDB. Both ConfigMaps contains two files: `log_config.py` and `superset_config.py` which contain logging and general Superset configuration respectively. |
| 54 | +ConfigMaps are created, one per RoleGroup and also one for the SupersetDB. Both ConfigMaps contains two files: |
| 55 | +`log_config.py` and `superset_config.py` which contain logging and general Superset configuration respectively. |
34 | 56 |
|
35 | 57 | == Dependencies |
36 | 58 |
|
37 | 59 | Superset requires an SQL database in which to store its metadata, dashboards and users. The Stackable platform does not have its own Operator for an SQL database but the xref:getting_started/index.adoc[] guides you through installing an example database with a Superset instance that you can use to get started. |
38 | 60 |
|
39 | 61 | == Connecting to data sources |
40 | 62 |
|
41 | | -Superset does not store its own data, instead it connects to other products where data is stored. On the Stackable Platform the two commonly used choices are xref:druid:index.adoc[Apache Druid] and xref:trino:index.adoc[Trino]. For Druid there is a way to xref:usage-guide/connecting-druid.adoc[connect a Druid instance declaratively] with a custom resource. For Trino this is on the roadmap. Have a look at the demos linked <<demos, below>> for examples of using Superset with Druid or Trino. |
| 63 | +Superset does not store its own data, instead it connects to other products where data is stored. On the Stackable |
| 64 | +Platform the two commonly used choices are xref:druid:index.adoc[Apache Druid] and xref:trino:index.adoc[Trino]. For |
| 65 | +Druid there is a way to xref:usage-guide/connecting-druid.adoc[connect a Druid instance declaratively] with a custom |
| 66 | +resource. For Trino this is on the roadmap. Have a look at the demos linked <<demos, below>> for examples of using |
| 67 | +Superset with Druid or Trino. |
42 | 68 |
|
43 | 69 | == [[demos]]Demos |
44 | 70 |
|
45 | | -Many of the Stackable xref:stackablectl::demos/index.adoc[demos] use Superset in the stack for data visualization and explaration. The demos come in two main variants. |
| 71 | +Many of the Stackable xref:demos:index.adoc[demos] use Superset in the stack for data visualization and explaration. The |
| 72 | +demos come in two main variants. |
46 | 73 |
|
47 | 74 | === With Druid |
48 | 75 |
|
49 | | -The xref:stackablectl::demos/nifi-kafka-druid-earthquake-data.adoc[] and xref:stackablectl::demos/nifi-kafka-druid-water-level-data.adoc[] demos show Superset connected to xref:druid:index.adoc[Druid], exploring earthquake and water level data respectively. |
| 76 | +The xref:demos:nifi-kafka-druid-earthquake-data.adoc[] and xref:demos:nifi-kafka-druid-water-level-data.adoc[] demos |
| 77 | +show Superset connected to xref:druid:index.adoc[Druid], exploring earthquake and water level data respectively. |
50 | 78 |
|
51 | 79 | === With Trino |
52 | 80 |
|
53 | | -The xref:stackablectl::demos/spark-k8s-anomaly-detection-taxi-data.adoc[], xref:stackablectl::demos/trino-taxi-data.adoc[], xref:stackablectl::demos/trino-iceberg.adoc[] and xref:stackablectl::demos/data-lakehouse-iceberg-trino-spark.adoc[] demos all use a xref:trino:index.adoc[Trino] instance on top of S3 storage that hold data to analyze. Superset is connected to Trino to analyze a variety of different datasets. |
| 81 | +The xref:demos:spark-k8s-anomaly-detection-taxi-data.adoc[], xref:demos:trino-taxi-data.adoc[], |
| 82 | +xref:demos:trino-iceberg.adoc[] and xref:demos:data-lakehouse-iceberg-trino-spark.adoc[] demos all use a |
| 83 | +xref:trino:index.adoc[Trino] instance on top of S3 storage that hold data to analyze. Superset is connected to Trino to |
| 84 | +analyze a variety of different datasets. |
54 | 85 |
|
55 | 86 | == Supported Versions |
56 | 87 |
|
|
0 commit comments