feat(end-to-end-security): add superset OPA

Author: sweb

demos/end-to-end-security/README.md

@@ -1,10 +1,14 @@
 # How to persist changes in Superset
 
-1. Log into Keycloak as the user `admin` (in the master realm) and create a user called `admin` in the demo realm.
-2. Use that to log into Superset with the `Admin` role.
-3. Optional: Add Database connection
-4. Add admin user in Keycloak to all relevant groups (so that he has access to the tables, so he can create datasets, charts and dashboards).
-5. `pgdump` the Postgres and update the dump in Git. For that shell into `postgresql-superset-0` and execute
+Note: User role assignments no longer need to be maintained in the dump.
+They are assigned by OPA on every login based on the Keycloak group memberships
+(see `stacks/end-to-end-security/superset-regorules.yaml`).
+
+1. Log into Superset via `Sign in with Keycloak` as the user `admin` (default password `adminadmin`).
+   The user is part of the demo realm in `keycloak-realm-config.yaml` and is automatically assigned the Superset `Admin` role by OPA.
+2. Optional: Add Database connection
+3. Add admin user in Keycloak to all relevant groups (so that he has access to the tables, so he can create datasets, charts and dashboards).
+4. `pgdump` the Postgres and update the dump in Git. For that shell into `postgresql-superset-0` and execute
 
 ```sh
 export PGPASSWORD=$(cat "${POSTGRES_POSTGRES_PASSWORD_FILE}")

docs/modules/demos/pages/end-to-end-security.adoc

@@ -4,6 +4,7 @@
 :k8s-cpu: https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/#cpu
 :rego: https://www.openpolicyagent.org/docs/latest/policy-language/
 :trino-policies: https://github.com/stackabletech/demos/blob/main/stacks/end-to-end-security/trino-policies.yaml
+:superset-regorules: https://github.com/stackabletech/demos/blob/main/stacks/end-to-end-security/superset-regorules.yaml
 
 This is a demo to showcase what can be done with Open Policy Agent around authorization in the Stackable Data Platform.
 It covers the following aspects of security:
@@ -16,7 +17,7 @@ This demo will:
 ** *Spark*: A multi-language engine for executing data engineering, data science, and machine learning. This demo uses it to create a (rather simple) report and write the results back into the persistence.
 ** *HDFS*: A distributed file system that is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
 ** *Hive metastore*: A service that stores metadata related to Apache Hive and other services. This demo uses it as metadata storage for Trino and Spark.
-** *Open policy agent (OPA)*: An open-source, general-purpose policy engine unifies policy enforcement across the stack. This demo uses it as the authorizer for Trino and HDFS, which decides which user can query which data.
+** *Open policy agent (OPA)*: An open-source, general-purpose policy engine unifies policy enforcement across the stack. This demo uses it as the authorizer for Trino and HDFS, which decides which user can query which data. It also assigns Superset roles based on the Keycloak group memberships.
 ** *Superset*: A modern data exploration and visualization platform. This demo utilizes Superset to retrieve data from Trino via SQL queries and build dashboards on top of that data.
 * Configure security to showcase the following features
 ** Column- and row-level filtering
@@ -101,6 +102,10 @@ The following user accounts are configured in Keycloak:
 |mark.ketting
 |mark.ketting
 |Head of Marketing
+
+|admin
+|adminadmin
+|Superset administrator, not member of any department
 |===
 
 [#ruleset]
@@ -323,6 +328,24 @@ When clicking on the button, you are redirected to the Superset UI being already
 
 image::end-to-end-security/superset_2.png[]
 
+=== Superset role assignment via OPA
+
+Which dashboards a user can see in Superset is determined by Superset roles.
+Instead of assigning these roles manually, Superset asks OPA for the roles of a user at every login (configured via `roleMappingFromOpa` on the SupersetCluster).
+OPA derives the roles from the Keycloak group memberships using the user-info-fetcher - the same group lookup the Trino rules are based on.
+The mapping is defined in the {superset-regorules}[superset-regorules ConfigMap]:
+
+* Every user gets the `Gamma` and `sql_lab` roles, granting access to the Superset UI including SQL Lab.
+* Members of a department additionally get the role granting access to the datasets of their department, e.g. members of `/Customer Service/Analytics` get the `Customer Service Analytics` role and with it the `Customer analytics` dashboard.
+* The `admin` user is assigned the Superset `Admin` role, so Superset can be administered via Single Sign On as well.
+
+You can try this out by logging in to Superset as `pamela.scott` (Customer Analytics) and `william.lewis` (Compliance Analytics) and comparing the dashboards each of them sees.
+Adding a user to a department group in Keycloak is all that is needed: on the next login, the user is assigned the matching Superset roles automatically.
+
+Note that the Superset roles only control which dashboards and datasets are visible.
+The data shown in the charts is still protected by Trino and OPA, because Superset impersonates the logged-in user when querying Trino.
+Also note that role assignments made in the Superset UI are overwritten on the next login of the affected user - the Rego rules are the single source of truth.
+
 === Open Policy Agent for the utmost flexibility in building access rules
 
 Some examples of access rules you can define for the Open Policy Agent authorization mechanism were already shown in the <<column-row-filtering, Column- and row-level filtering section>>. For more examples, check out the {trino-policies}[trino-policies ConfigMap] of this demo and also the https://www.youtube.com/watch?v=ATlq_l3WNiA&t=1970s[ACL section of the 'Mastering Data Platform Security' recording from above].

stacks/end-to-end-security/keycloak-realm-config.yaml

@@ -46,6 +46,24 @@ stringData:
         "subGroups" : [ ]
       } ],
       "users" : [ {
+        "username" : "admin",
+        "enabled" : true,
+        "totp" : false,
+        "emailVerified" : true,
+        "firstName" : "Superset",
+        "lastName" : "Admin",
+        "email" : "admin@superset.com",
+        "credentials" : [ {
+          "type" : "password",
+          "value" : "{{ supersetAdminPassword }}",
+          "temporary" : false
+        } ],
+        "disableableCredentialTypes" : [ ],
+        "requiredActions" : [ ],
+        "realmRoles" : [ "default-roles-demo" ],
+        "notBefore" : 0,
+        "groups" : [ ]
+      }, {
         "id" : "e44a09fa-bce1-40e8-a1da-28902b79dcf0",
         "createdTimestamp" : 1711375603780,
         "username" : "daniel.king",

stacks/end-to-end-security/superset-regorules.yaml

@@ -0,0 +1,45 @@
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: superset-regorules
+  labels:
+    opa.stackable.tech/bundle: superset
+data:
+  superset_roles.rego: |
+    package superset
+
+    # Superset roles assigned to users by the OpaSupersetSecurityManager on every
+    # login, replacing any roles assigned via the Superset UI or directly in the
+    # metadata database. The roles must already exist in the Superset metadata
+    # database, otherwise the assignment is skipped and an error is logged.
+
+    # Roles every user gets: Gamma plus SQL Lab access
+    base_roles := ["Gamma", "sql_lab"]
+
+    # Keycloak groups -> additional Superset roles granting dashboard access.
+    # Note that this only controls dashboard visibility in Superset. Access to
+    # the underlying data is enforced by Trino (see trino-policies.yaml) based
+    # on the same Keycloak groups.
+    group_roles := {
+        "/Customer Service/Analytics": ["Customer Service Analytics"],
+        "/Compliance and Regulation/Analytics": ["Compliance and Regulation Analytics"],
+        "/Marketing": ["Marketing"],
+    }
+
+    # The "admin" account is created by the superset-operator in the metadata
+    # database. The Keycloak realm (see keycloak-realm-config.yaml) contains a
+    # matching user, so that the Superset admin can log in via Single Sign On.
+    user_roles := ["Admin"] if {
+        input.username == "admin"
+    }
+
+    user_roles := roles if {
+        input.username != "admin"
+        groups := data.stackable.opa.userinfo.v1.userInfoByUsername(input.username).groups
+        department_roles := [role |
+            some group in groups
+            some role in group_roles[group]
+        ]
+        roles := array.concat(base_roles, department_roles)
+    }

stacks/end-to-end-security/superset.yaml

@@ -28,6 +28,12 @@ spec:
         oidc:
           clientCredentialsSecret: superset-keycloak-client
         userRegistrationRole: Gamma_extended
+    # Superset roles are assigned at every login based on the Keycloak group
+    # memberships, see superset-regorules.yaml
+    authorization:
+      roleMappingFromOpa:
+        configMapName: opa
+        package: superset
   nodes:
     roleConfig:
       listenerClass: external-stable

stacks/stacks-v2.yaml

@@ -526,7 +526,7 @@ stacks:
       pvc: 40Gi
     manifests:
       - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/_templates/keycloak-serviceaccount.yaml
-      - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/keycloak-realm-config.yaml
+      - plainYaml: stacks/end-to-end-security/keycloak-realm-config.yaml
       - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/_templates/keycloak.yaml
       - helmChart: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/_templates/postgresql-hive-iceberg.yaml
       - helmChart: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/_templates/postgresql-superset.yaml
@@ -543,6 +543,7 @@ stacks:
       - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/trino-policies.yaml
       - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/rbac.yaml
       - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/setup-postgresql.yaml
+      - plainYaml: stacks/end-to-end-security/superset-regorules.yaml
       - plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/superset.yaml
     parameters:
       - name: keycloakAdminPassword