feat(keycloak): replace shell-based realm setup with keycloak-config-cli

Phase 1a of the declarative Keycloak configuration design proposed in
#154. Replaces the imperative shell script in realm-setup-job.yaml with
a declarative YAML applied by keycloak-config-cli (kcc) at PostSync.

Changes:

- Add `realm-config-cm.yaml`: ConfigMap carrying the kcc-format YAML for
  the nebari realm. Parity with the previous shell setup, including the
  `groups` client scope + group-membership mapper (load-bearing for
  operator-managed app authorization), the `argocd` OIDC client (PR
  #234), the `argocd-admins`/`argocd-viewers` groups, and the admin
  user's `argocd-admins` membership.
- Rewrite `realm-setup-job.yaml`: pulls
  `quay.io/adorsys/keycloak-config-cli:6.5.0-26.5.5`, mounts the
  ConfigMap, and applies it against Keycloak. Secrets are mounted from
  the existing `nebari-realm-admin-credentials` and
  `argocd-oidc-client-secret` Secrets and substituted into the realm
  YAML by kcc at apply time via `$(env:VAR)` placeholders.

Operator coexistence:

- `IMPORT_REMOTESTATE_ENABLED=true` (kcc default since v6.x): kcc tracks
  resources it created in a Keycloak realm attribute. Re-runs only
  reconcile what kcc itself owns; operator-managed OAuth2 clients
  created at runtime via NebariApplication CRDs are invisible to kcc's
  delete logic.
- `IMPORT_MANAGED_CLIENT=no-delete`: belt-and-braces. Even if a future
  input change accidentally removes the argocd client from `clients:`,
  kcc will not delete unlisted clients.

The strict "omit `clients:` entirely" pattern from my #154 review was
too restrictive: NIC owns the `argocd` client at bootstrap, so it has
to be in the input. The two-layer protection above is the practical
shape of the coexistence story.

Phase 1a scope (per #154 review): only replaces the existing realm
setup. No `keycloak:` section in `config.yaml`, no NIC-side env-var
scanning, no merge engine. Defaults are static. Future phases add the
user-facing schema (1b) and the user-secrets pipeline (1c).

Migration: on existing deployments, kcc detects existing resources by
their natural keys (realm name, role name, username, scope name) and
updates them in place, adding them to remote state. First run on an
already-configured cluster is a no-op for existing managed values; the
pre-generated argocd OIDC client secret (same value on both sides) is
re-applied verbatim.

Refs #154.

Author: viniciusdc

pkg/argocd/templates/manifests/keycloak/realm-config-cm.yaml

@@ -0,0 +1,86 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: keycloak-realm-config
+  namespace: keycloak
+  labels:
+    app.kubernetes.io/part-of: nebari-foundational
+    app.kubernetes.io/managed-by: nebari-infrastructure-core
+data:
+  # keycloak-config-cli input for the nebari realm.
+  # Parity with the previous imperative shell setup, expressed declaratively.
+  # `$(env:VAR)` placeholders are substituted by kcc at apply time from env
+  # vars set on the Job (see realm-setup-job.yaml).
+  realm.yaml: |
+    realm: nebari
+    enabled: true
+    displayName: Nebari
+    sslRequired: external
+    registrationAllowed: false
+    loginWithEmailAllowed: true
+    resetPasswordAllowed: true
+    bruteForceProtected: true
+
+    defaultDefaultClientScopes:
+      - groups
+
+    roles:
+      realm:
+        - name: admin
+          description: Administrator role
+        - name: user
+          description: Regular user role
+
+    groups:
+      - name: argocd-admins
+      - name: argocd-viewers
+
+    users:
+      - username: admin
+        enabled: true
+        emailVerified: true
+        firstName: Admin
+        lastName: User
+        email: admin@nebari.local
+        realmRoles:
+          - admin
+          - user
+        credentials:
+          - type: password
+            value: $(env:REALM_ADMIN_PASSWORD)
+        groups:
+          - /argocd-admins
+
+    clientScopes:
+      - name: groups
+        protocol: openid-connect
+        attributes:
+          include.in.token.scope: "true"
+          display.on.consent.screen: "true"
+        protocolMappers:
+          - name: group-membership
+            protocol: openid-connect
+            protocolMapper: oidc-group-membership-mapper
+            config:
+              full.path: "false"
+              introspection.token.claim: "true"
+              userinfo.token.claim: "true"
+              multivalued: "true"
+              id.token.claim: "true"
+              access.token.claim: "true"
+              claim.name: groups
+
+    clients:
+      - clientId: argocd
+        enabled: true
+        protocol: openid-connect
+        publicClient: false
+        secret: $(env:ARGOCD_CLIENT_SECRET)
+        redirectUris:
+          - https://argocd.{{ .Domain }}/auth/callback
+        webOrigins:
+          - https://argocd.{{ .Domain }}
+        directAccessGrantsEnabled: false
+        standardFlowEnabled: true
+        defaultClientScopes:
+          - groups

pkg/argocd/templates/manifests/keycloak/realm-setup-job.yaml

@@ -16,137 +16,61 @@ spec:
       restartPolicy: OnFailure
       containers:
         - name: realm-setup
-          image: quay.io/keycloak/keycloak:24.0
+          # Pinned to a kcc release verified in CI against the deployed
+          # Keycloak version. See:
+          # https://redirect.github.com/adorsys/keycloak-config-cli/blob/v6.5.0/.github/workflows/ci.yaml
+          image: quay.io/adorsys/keycloak-config-cli:6.5.0-26.5.5
           env:
-            - name: KEYCLOAK_ADMIN_PASSWORD
+            # Keycloak connection
+            - name: KEYCLOAK_URL
+              value: {{ .KeycloakServiceURL }}
+            - name: KEYCLOAK_USER
+              value: admin
+            - name: KEYCLOAK_PASSWORD
               valueFrom:
                 secretKeyRef:
                   name: {{ .KeycloakAdminSecretName }}
                   key: admin-password
+            - name: KEYCLOAK_AVAILABILITYCHECK_ENABLED
+              value: "true"
+            - name: KEYCLOAK_AVAILABILITYCHECK_TIMEOUT
+              value: 120s
+
+            # kcc import behavior
+            - name: IMPORT_FILES_LOCATIONS
+              value: /config/realm.yaml
+            - name: IMPORT_VARSUBSTITUTION_ENABLED
+              value: "true"
+            - name: IMPORT_VARSUBSTITUTION_UNDEFINEDISERROR
+              value: "true"
+            # Default is true since kcc v6.x; explicit here for review
+            # clarity. kcc tracks resources it created in a Keycloak realm
+            # attribute, so re-runs only touch resources kcc itself manages.
+            - name: IMPORT_REMOTESTATE_ENABLED
+              value: "true"
+            # Belt-and-braces: never delete clients on re-run, even ones
+            # that disappear from the input. Protects operator-managed
+            # OAuth2 clients (created at runtime via NebariApplication CRDs)
+            # if the input config ever changes shape.
+            - name: IMPORT_MANAGED_CLIENT
+              value: "no-delete"
+
+            # Substituted into realm.yaml at kcc apply time
             - name: REALM_ADMIN_PASSWORD
               valueFrom:
                 secretKeyRef:
                   name: nebari-realm-admin-credentials
                   key: password
-            - name: KEYCLOAK_URL
-              value: {{ .KeycloakServiceURL }}
             - name: ARGOCD_CLIENT_SECRET
               valueFrom:
                 secretKeyRef:
                   name: argocd-oidc-client-secret
                   key: client-secret
-            - name: DOMAIN
-              value: {{ .Domain }}
-          command:
-            - /bin/bash
-            - -c
-            - |
-              set -e
-              KCADM="/opt/keycloak/bin/kcadm.sh"
-
-              echo "Waiting for Keycloak to be ready..."
-              for i in $(seq 1 60); do
-                if $KCADM config credentials --server "$KEYCLOAK_URL" --realm master --user admin --password "$KEYCLOAK_ADMIN_PASSWORD" 2>/dev/null; then
-                  echo "Keycloak is ready"
-                  break
-                fi
-                echo "Keycloak not ready, waiting... ($i/60)"
-                sleep 5
-              done
-
-              echo "Creating nebari realm..."
-              $KCADM create realms \
-                -s realm=nebari \
-                -s enabled=true \
-                -s displayName="Nebari" \
-                -s sslRequired=external \
-                -s registrationAllowed=false \
-                -s loginWithEmailAllowed=true \
-                -s resetPasswordAllowed=true \
-                -s bruteForceProtected=true || echo "Realm may already exist"
-
-              echo "Creating realm roles..."
-              $KCADM create roles -r nebari -s name=admin -s description="Administrator role" || true
-              $KCADM create roles -r nebari -s name=user -s description="Regular user role" || true
-
-              echo "Creating admin user in nebari realm..."
-              $KCADM create users -r nebari \
-                -s username=admin \
-                -s enabled=true \
-                -s emailVerified=true \
-                -s firstName=Admin \
-                -s lastName=User \
-                -s email=admin@nebari.local || echo "User may already exist"
-
-              echo "Setting admin user password..."
-              $KCADM set-password -r nebari \
-                --username admin \
-                --new-password "$REALM_ADMIN_PASSWORD"
-
-              echo "Assigning roles to admin user..."
-              $KCADM add-roles -r nebari --uusername admin --rolename admin || true
-              $KCADM add-roles -r nebari --uusername admin --rolename user || true
-
-              echo "Configuring groups client scope with group-membership mapper..."
-              # Look up the groups client scope ID using grep/sed (no python3 in keycloak image)
-              GROUPS_SCOPE_ID=$($KCADM get client-scopes -r nebari --fields id,name | \
-                grep -B1 '"name" *: *"groups"' | sed -n 's/.*"id" *: *"\([^"]*\)".*/\1/p')
-
-              if [ -z "$GROUPS_SCOPE_ID" ]; then
-                echo "Creating groups client scope..."
-                $KCADM create client-scopes -r nebari \
-                  -s name=groups \
-                  -s protocol=openid-connect \
-                  -s 'attributes={"include.in.token.scope":"true","display.on.consent.screen":"true"}' || true
-                GROUPS_SCOPE_ID=$($KCADM get client-scopes -r nebari --fields id,name | \
-                  grep -B1 '"name" *: *"groups"' | sed -n 's/.*"id" *: *"\([^"]*\)".*/\1/p')
-              fi
-
-              if [ -n "$GROUPS_SCOPE_ID" ]; then
-                echo "Adding group-membership mapper to groups scope (id=$GROUPS_SCOPE_ID)..."
-                $KCADM create client-scopes/$GROUPS_SCOPE_ID/protocol-mappers/models -r nebari \
-                  -s name=group-membership \
-                  -s protocol=openid-connect \
-                  -s protocolMapper=oidc-group-membership-mapper \
-                  -s 'config={"full.path":"false","introspection.token.claim":"true","userinfo.token.claim":"true","id.token.claim":"true","access.token.claim":"true","claim.name":"groups"}' || echo "Mapper may already exist"
-
-                echo "Ensuring groups is a realm default client scope..."
-                $KCADM update realms/nebari/default-default-client-scopes/$GROUPS_SCOPE_ID -r nebari || true
-              fi
-
-              echo "Creating ArgoCD OIDC client..."
-              $KCADM create clients -r nebari \
-                -s clientId=argocd \
-                -s enabled=true \
-                -s protocol=openid-connect \
-                -s publicClient=false \
-                -s "secret=$ARGOCD_CLIENT_SECRET" \
-                -s "redirectUris=[\"https://argocd.$DOMAIN/auth/callback\"]" \
-                -s directAccessGrantsEnabled=false \
-                -s standardFlowEnabled=true || echo "Client may already exist"
-
-              # Add groups scope to argocd client as a default scope
-              ARGOCD_CLIENT_ID=$($KCADM get clients -r nebari --fields id,clientId | \
-                grep -B1 '"clientId" *: *"argocd"' | sed -n 's/.*"id" *: *"\([^"]*\)".*/\1/p')
-
-              if [ -n "$ARGOCD_CLIENT_ID" ] && [ -n "$GROUPS_SCOPE_ID" ]; then
-                echo "Adding groups scope to argocd client..."
-                $KCADM update clients/$ARGOCD_CLIENT_ID/default-client-scopes/$GROUPS_SCOPE_ID -r nebari || true
-              fi
-
-              echo "Creating ArgoCD access groups..."
-              $KCADM create groups -r nebari -s name=argocd-admins || echo "Group may already exist"
-              $KCADM create groups -r nebari -s name=argocd-viewers || echo "Group may already exist"
-
-              echo "Adding admin user to argocd-admins group..."
-              ADMIN_USER_ID=$($KCADM get users -r nebari -q username=admin --fields id | \
-                sed -n 's/.*"id" *: *"\([^"]*\)".*/\1/p')
-              ADMINS_GROUP_ID=$($KCADM get groups -r nebari --fields id,name | \
-                grep -B1 '"name" *: *"argocd-admins"' | sed -n 's/.*"id" *: *"\([^"]*\)".*/\1/p')
-
-              if [ -n "$ADMIN_USER_ID" ] && [ -n "$ADMINS_GROUP_ID" ]; then
-                $KCADM update users/$ADMIN_USER_ID/groups/$ADMINS_GROUP_ID -r nebari \
-                  -s realm=nebari -s userId=$ADMIN_USER_ID -s groupId=$ADMINS_GROUP_ID -n || true
-              fi
-
-              echo "Realm setup complete!"
+          volumeMounts:
+            - name: realm-config
+              mountPath: /config
+              readOnly: true
+      volumes:
+        - name: realm-config
+          configMap:
+            name: keycloak-realm-config