Skip to content

Commit 2553597

Browse files
Felix Hennigfhennig
Felix Hennig
and
fhennig
committed
restructured usage guide & new landing page (#344)
# Description :computer: [preview](https://deploy-preview-374--stackable-docs.netlify.app/home/nightly/superset/index.html) 💻 Co-authored-by: Felix Hennig <fhennig@users.noreply.github.com>
1 parent c17e941 commit 2553597

14 files changed

Lines changed: 309 additions & 262 deletions

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,13 @@
1212
- `operator-rs` `0.31.0` -> `0.35.0` ([#322], [#326]).
1313
- Bumped stackable image versions to "23.4.0-rc2" ([#322], [#326]).
1414
- Fragmented `SupersetConfig` ([#323]).
15+
- Restructured documentation ([#344]).
1516

1617
[#322]: https://github.com/stackabletech/superset-operator/pull/322
1718
[#323]: https://github.com/stackabletech/superset-operator/pull/323
1819
[#326]: https://github.com/stackabletech/superset-operator/pull/326
1920
[#337]: https://github.com/stackabletech/superset-operator/pull/337
21+
[#344]: https://github.com/stackabletech/superset-operator/pull/344
2022

2123
## [23.1.0] - 2023-01-23
2224

‎docs/modules/superset/images/superset_overview.drawio.svg‎

Lines changed: 4 additions & 0 deletions
Loading

‎docs/modules/superset/pages/getting_started/first_steps.adoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,4 +106,4 @@ Great! You have set up a Superset instance and connected to it!
106106

107107
== What's next
108108

109-
Look at the xref:usage.adoc[Usage page] to find out more about configuring your Superset instance or have a look at the Superset documentation to link:https://superset.apache.org/docs/creating-charts-dashboards/creating-your-first-dashboard[create your first dashboard].
109+
Look at the xref:usage-guide/index.adoc[] to find out more about configuring your Superset instance or have a look at the Superset documentation to https://superset.apache.org/docs/creating-charts-dashboards/creating-your-first-dashboard[create your first dashboard].

‎docs/modules/superset/pages/index.adoc‎

Lines changed: 50 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,59 @@
11
= Stackable Operator for Apache Superset
2+
:description: The Stackable Operator for Apache Superset is a Kubernetes operator that can manage Apache Superset clusters. Learn about its features, resources, dependencies and demos, and see the list of supported Superset versions.
3+
:keywords: Stackable Operator, Apache Superset, Kubernetes, operator, data science, data exploration, SQL, engineer, big data, CRD, StatefulSet, ConfigMap, Service, Druid, Trino, S3, demo, version
24

3-
This is an operator for Kubernetes that can manage https://superset.apache.org/[Apache Superset]
4-
clusters.
5+
The Stackable Operator for Apache Superset is an operator that can deploy and manage https://superset.apache.org/[Apache Superset] clusters on Kubernetes. Superset is a data exploration and visualization tool that connects to data sources via SQL. Store your data in Apache Druid or Trino, and manage your Druid and Trino instances with the Stackable Operators for xref:druid:index.adoc[Apache Druid] or xref:trino:index.adoc[Trino]. This operator helps you manage your Superset instances on Kubernetes efficiently.
56

6-
WARNING: This operator is part of the Stackable Data Platform and only works with images from the
7-
https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable%2Fsuperset[Stackable] repository.
7+
== Getting started
8+
9+
Get started using Superset with Stackable Operator by following the xref:getting_started/index.adoc[]. It guides you through installing the Operator alongside a PostgreSQL database, connecting to your Superset instance and analyzing some preloaded example data.
10+
11+
== Resources
12+
13+
The Operator manages three https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/[custom resources]: The _SupersetCluster_, _SupersetDB_ and _DruidConnection_. It creates a number of different Kubernetes resources based on the custom resources.
14+
15+
=== Custom resources
16+
17+
The SupersetCluster is the main resource for the configuration of the Superset instance. The resource defines only one xref:concepts:roles-and-role-groups.adoc[role], the `node`. The various configuration options are explained in the xref:usage-guide/index.adoc[]. It helps you tune your cluster to your needs by configuring xref:usage-guide/storage-resource-configuration.adoc[resource usage], xref:usage-guide/security.adoc[security], xref:usage-guide/logging.adoc[logging] and more.
18+
19+
When a SupersetCluster is first deployed, a SupersetDB resource is created. The SupersetDB resource is a wrapper resource for the SQL database that is used by Superset for its metadata. The resource contains some configuration but also keeps track of whether the database has been initialized or not. It is not deleted automatically if a SupersetCluster is deleted, and so can be reused.
20+
21+
DruidConnection resources link a Superset and Druid instance. It lets you define this connection in the familiar way of deploying a resource (instead of configuring the connection via the Superset UI or API). The operator configures the connection between Druid and the Superset instance.
22+
23+
=== Kubernetes resources
24+
25+
Based on the custom resources you define, the Operator creates ConfigMaps, StatefulSets and Services.
26+
27+
image::superset_overview.drawio.svg[A diagram depicting the Kubernetes resources created by the operator]
28+
29+
The diagram above depicts all the Kubernetes resources created by the operator, and how they relate to each other. The Jobs created for the SupersetDB and DruidConnnection resources are not shown.
30+
31+
For every xref:concepts:roles-and-role-groups.adoc#_role_groups[role group] you define, the Operator creates a StatefulSet with the amount of replicas defined in the RoleGroup. Every Pod in the StatefulSet has two containers: the main container running Superset and a sidecar container gathering metrics for xref:operators:monitoring.adoc[]. The Operator creates a Service for the `node` role as well as a single service per role group.
32+
33+
ConfigMaps are created, one per RoleGroup and also one for the SupersetDB. Both ConfigMaps contains two files: `log_config.py` and `superset_config.py` which contain logging and general Superset configuration respectively.
34+
35+
== Dependencies
36+
37+
Superset requires an SQL database in which to store its metadata, dashboards and users. The Stackable platform does not have its own Operator for an SQL database but the xref:getting_started/index.adoc[] guides you through installing an example database with a Superset instance that you can use to get started.
38+
39+
== Connecting to data sources
40+
41+
Superset does not store its own data, instead it connects to other products where data is stored. On the Stackable Platform the two commonly used choices are xref:druid:index.adoc[Apache Druid] and xref:trino:index.adoc[Trino]. For Druid there is a way to xref:usage-guide/connecting-druid.adoc[connect a Druid instance declaratively] with a custom resource. For Trino this is on the roadmap. Have a look at the demos linked <<demos, below>> for examples of using Superset with Druid or Trino.
42+
43+
== [[demos]]Demos
44+
45+
Many of the Stackable xref:stackablectl::demos/index.adoc[demos] use Superset in the stack for data visualization and explaration. The demos come in two main variants.
46+
47+
=== With Druid
48+
49+
The xref:stackablectl::demos/nifi-kafka-druid-earthquake-data.adoc[] and xref:stackablectl::demos/nifi-kafka-druid-water-level-data.adoc[] demos show Superset connected to xref:druid:index.adoc[Druid], exploring earthquake and water level data respectively.
50+
51+
=== With Trino
52+
53+
The xref:stackablectl::demos/spark-k8s-anomaly-detection-taxi-data.adoc[], xref:stackablectl::demos/trino-taxi-data.adoc[], xref:stackablectl::demos/trino-iceberg.adoc[] and xref:stackablectl::demos/data-lakehouse-iceberg-trino-spark.adoc[] demos all use a xref:trino:index.adoc[Trino] instance on top of S3 storage that hold data to analyze. Superset is connected to Trino to analyze a variety of different datasets.
854

955
== Supported Versions
1056

1157
The Stackable Operator for Apache Superset currently supports the following versions of Superset:
1258

1359
include::partial$supported-versions.adoc[]
14-
15-
== Getting the Docker image
16-

17-
[source]
18-
----
19-
docker pull docker.stackable.tech/stackable/superset:<version>-stackable<stackable-version>
20-
----

‎docs/modules/superset/pages/usage-guide/configuration-environment-overrides.adoc‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
= Configuration & Environment Overrides
2+
3+
The cluster definition also supports overriding configuration properties and environment variables,
4+
either per role or per role group, where the more specific override (role group) has precedence over
5+
the less specific one (role).
6+
7+
IMPORTANT: Overriding certain properties which are set by the operator (such as the `STATS_LOGGER`)
8+
can interfere with the operator and can lead to problems.
9+
10+
== Configuration Properties
11+
12+
For a role or role group, at the same level of `config`, you can specify `configOverrides` for the
13+
`superset_config.py`. For example, if you want to set the CSV export encoding and the preferred
14+
databases adapt the `nodes` section of the cluster resource as follows:
15+
16+
[source,yaml]
17+
----
18+
nodes:
19+
roleGroups:
20+
default:
21+
config: {}
22+
configOverrides:
23+
superset_config.py:
24+
CSV_EXPORT: "{'encoding': 'utf-8'}"
25+
PREFERRED_DATABASES: |-
26+
[
27+
'PostgreSQL',
28+
'Presto',
29+
'MySQL',
30+
'SQLite',
31+
# etc.
32+
]
33+
----
34+
35+
Just as for the `config`, it is possible to specify this at the role level as well:
36+
37+
[source,yaml]
38+
----
39+
nodes:
40+
configOverrides:
41+
superset_config.py:
42+
CSV_EXPORT: "{'encoding': 'utf-8'}"
43+
PREFERRED_DATABASES: |-
44+
[
45+
'PostgreSQL',
46+
'Presto',
47+
'MySQL',
48+
'SQLite',
49+
# etc.
50+
]
51+
roleGroups:
52+
default:
53+
config: {}
54+
----
55+
56+
All override property values must be strings. They are treated as Python expressions. So care must
57+
be taken to produce a valid configuration.
58+
59+
For a full list of configuration options we refer to the
60+
https://github.com/apache/superset/blob/master/superset/config.py[main config file for Superset].
61+
62+
== Environment Variables
63+
64+
In a similar fashion, environment variables can be (over)written. For example per role group:
65+
66+
[source,yaml]
67+
----
68+
nodes:
69+
roleGroups:
70+
default:
71+
config: {}
72+
envOverrides:
73+
FLASK_ENV: development
74+
----
75+
76+
or per role:
77+
78+
[source,yaml]
79+
----
80+
nodes:
81+
envOverrides:
82+
FLASK_ENV: development
83+
roleGroups:
84+
default:
85+
config: {}
86+
----
87+
88+
// cliOverrides don't make sense for this operator, so the feature is omitted for now

‎docs/modules/superset/pages/usage-guide/connecting-druid.adoc‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
= Connecting Apache Druid clusters
2+
3+
The operator can automatically connect Superset to Apache Druid clusters managed by the https://docs.stackable.tech/druid/index.html[Stackable Druid Cluster].
4+
5+
To do so, create a _DruidConnection_ resource:
6+
7+
[source,yaml]
8+
----
9+
apiVersion: superset.stackable.tech/v1alpha1
10+
kind: DruidConnection
11+
metadata:
12+
name: superset-druid-connection
13+
spec:
14+
superset:
15+
name: superset
16+
namespace: default
17+
druid:
18+
name: my-druid-cluster
19+
namespace: default
20+
21+
----
22+
23+
The `name` and `namespace` in `spec.superset` refer to the Superset cluster that you want to connect. Following our example above, the name is `superset`.
24+
25+
In `spec.druid` you specify the `name` and `namespace` of your Druid cluster.
26+
27+
The `namespace` part is optional; if it is omitted it will default to the namespace of the DruidConnection.
28+
29+
The namespace for both the Superset and Druid clusters can be omitted, in which case the Operator will assume that they are in the same namespace as the DruidConnection.
30+
31+
Once the database is initialized, the connection will be added to the cluster by the operator. You can see it in the user interface under Data > Databases:
32+
33+
image::superset-databases.png[Superset databases showing the connected Druid cluster]

‎docs/modules/superset/pages/usage-guide/index.adoc‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
= Usage guide
2+
:page-aliases: usage.doc
3+
4+
The usage guide covers various aspects of configuring Superset and interconnection with other tools.
5+
6+
Learn about defining the amount of xref:usage-guide/storage-resource-configuration.adoc[resources] Superset uses and how to configure xref:usage-guide/pod-placement.adoc[]. Learn how to xref:usage-guide/connecting-druid.adoc[connect to Apache Druid] operated by the xref:druid:index.adoc[].
7+
8+
Configure xref:usage-guide/security.adoc#authentication[authentication] with LDAP and xref:usage-guide/security.adoc#authorization[authorization]. Observe your Superset instance with xref:usage-guide/logging.adoc[log aggregation] and xref:usage-guide/monitoring.adoc[monitoring].

‎docs/modules/superset/pages/usage-guide/logging.adoc‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
= Log aggregation
2+
3+
The logs can be forwarded to a Vector log aggregator by providing a discovery
4+
ConfigMap for the aggregator and by enabling the log agent:
5+
6+
[source,yaml]
7+
----
8+
spec:
9+
vectorAggregatorConfigMapName: vector-aggregator-discovery
10+
nodes:
11+
config:
12+
logging:
13+
enableVectorAgent: true
14+
databaseInitialization:
15+
logging:
16+
enableVectorAgent: true
17+
----
18+
19+
Further information on how to configure logging, can be found in
20+
xref:home:concepts:logging.adoc[].

‎docs/modules/superset/pages/usage-guide/monitoring.adoc‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
= Monitoring
2+
3+
The managed Superset instances are automatically configured to export Prometheus metrics. See
4+
xref:home:operators:monitoring.adoc[] for more details.

‎docs/modules/superset/pages/usage-guide/pod-placement.adoc‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
= Pod Placement
2+
3+
You can configure the Pod placement of the Superset pods as described in xref:concepts:pod_placement.adoc[].
4+
5+
The default affinities created by the operator are:
6+
7+
1. Distribute all the Superset Pods (weight 70)

0 commit comments

Comments
 (0)