Feat/kustomize deployment

.gitignore

@@ -137,4 +137,7 @@ scratch/
 
 # Untracked personal Claude Code files (e.g. CLAUDE.local.md, commands/my-command.local.md)
 .claude/**/*.local.*
-CLAUDE.local.md
+CLAUDE.local.md
+
+# Kustomize secrets (real values — never commit)
+kustomize/**/secrets.yaml

CLAUDE.md

@@ -28,6 +28,10 @@ When pulling in upstream Apache Superset changes, update the **"Based on"** fiel
 
 Wiki pages live in `wiki/` and are synced to the GitHub Wiki on merge to main via `.github/workflows/sync-wiki.yml`. Documentation sync runs automatically as a background process during `/commit-and-push` and `/merge-request` — it audits wiki pages, README.md, and inline documentation against code changes. See `.claude/skills/sync-documentation.md` for details.
 
+## Kubernetes Deployment
+
+Kustomize overlays live in `kustomize/`. This is GeoSet-specific (not upstream Superset). `base/` is the dev/demo overlay analogous to Docker Compose; `full/` adds Redis, Celery, and Flux GitOps for production.
+
 ## Important Notes
 
 - Always use context7 when I need code generation, setup or configuration steps, or

README.md

@@ -116,6 +116,10 @@ The Dockerfile at the root of the repository uses the same Debian-based image us
 DOCKERFILE=Dockerfile.rhel docker compose up
 ```
 
+### Kubernetes Deployment
+
+Kustomize overlays are available in [`kustomize/`](./kustomize/) for deploying GeoSet to Kubernetes. The `base` overlay mirrors the Docker Compose stack (dev/demo), while `full` adds Redis, Celery workers, and Flux GitOps for production use. See [`kustomize/README.md`](./kustomize/README.md) for details.
+
 ### Step 3 - Open GeoSet and Explore
 
 We've created an example dashboard accessible at [http://localhost:9001/superset/dashboard/geoset-example-dashboard](http://localhost:9001/superset/dashboard/geoset-example-dashboard).

kustomize/README.md

@@ -0,0 +1,112 @@
+# GeoSet Kustomize Deployment
+
+Kubernetes manifests for deploying GeoSet, organized as Kustomize overlays.
+
+## Overlays
+
+| Overlay | Description | Use case |
+|---------|-------------|----------|
+| **base** | PostGIS, metadata DB, Superset web, sample data ingest. No Redis or Celery. | Local/dev clusters, quick demos |
+| **full** | Everything in base + Redis, Celery workers, Celery beat, cache warmup configuration, Flux GitOps | Staging/production starting point |
+
+## Prerequisites
+
+- A Kubernetes cluster (minikube, kind, EKS, etc.)
+- `kubectl` installed and configured
+- Container images pushed (default: `jmeegan607/geoset`, `ebienstock/geoset:data-ingest-latest`)
+
+## Setup
+
+### 1. Create secrets
+
+Copy the example and fill in real values:
+
+```bash
+cp kustomize/base/secrets.yaml.example kustomize/base/secrets.yaml
+```
+
+Edit `kustomize/base/secrets.yaml` and set:
+
+| Secret | Description | Required |
+|--------|-------------|----------|
+| `DATABASE_PASSWORD` | Superset metadata Postgres password | Yes |
+| `POSTGIS_PASSWORD` | PostGIS (geospatial data) password | Yes |
+| `EXAMPLES_PASSWORD` | Superset examples DB password | Yes |
+| `SUPERSET_SECRET_KEY` | Flask secret key — generate with `openssl rand -base64 42` | Yes |
+| `ADMIN_PASSWORD` | Superset admin user password | Yes |
+| `MAPBOX_API_KEY` | Mapbox GL token for map tiles | Yes |
+
+> **Never commit `secrets.yaml`** — it is gitignored. Only `secrets.yaml.example` is tracked.
+
+### 2. Review environment variables
+
+Base env config lives in `base/config/superset.env`. The defaults work out of the box for most setups. Key variables:
+
+| Variable | Default | Notes |
+|----------|---------|-------|
+| `SUPERSET_CONFIG_PATH` | `superset_config_docker_light.py` (base) / `superset_config.py` (full) | Auto-switched by overlay |
+| `DATABASE_HOST` | `postgres-metadata` | K8s service name |
+| `POSTGIS_HOST` | `postgis` | K8s service name |
+| `REDIS_HOST` | `redis` | Only used in full overlay |
+
+You generally don't need to change these unless you're pointing at external databases.
+
+### 3. Update container images (if needed)
+
+The base and full overlays use the image tags from the individual manifests. Add an `images` block to the relevant `kustomization.yaml` if you need to pin a different tag for your environment.
+
+## Deploy
+
+### Base (dev/demo)
+
+```bash
+kubectl apply -k kustomize/base
+```
+
+### Full (staging/production)
+
+```bash
+kubectl apply -k kustomize/full
+```
+
+### Verify
+
+```bash
+kubectl -n geoset get pods
+kubectl -n geoset get svc
+```
+
+Superset web will be available on port `8088` via the `superset-web` service. To access locally:
+
+```bash
+kubectl -n geoset port-forward svc/superset-web 8088:8088
+```
+
+### Validate manifests
+
+The deployment requires a local `secrets.yaml`, which is intentionally gitignored. To render the manifests for review or CI without creating local untracked files, validate against a temporary copy:
+
+```bash
+tmp="$(mktemp -d)"
+cp -R kustomize "$tmp/"
+cp "$tmp/kustomize/base/secrets.yaml.example" "$tmp/kustomize/base/secrets.yaml"
+kubectl kustomize "$tmp/kustomize/base"
+kubectl kustomize "$tmp/kustomize/full"
+```
+
+## Full overlay extras
+
+The full overlay adds on top of base:
+
+- **Redis** — caching backend and Celery message broker
+- **Celery workers** (2 replicas) — async query execution
+- **Celery beat** — scheduled tasks including cache warmup
+- **Flux GitOps** — auto-syncs from `raft-tech/GeoSet` main branch
+
+It also patches `superset-web` to 2 replicas, mounts the full deployment Superset config override, and adds a Redis readiness check to its init container.
+
+## Teardown
+
+```bash
+kubectl delete -k kustomize/base   # or kustomize/full
+```

kustomize/base/config/superset.env

@@ -0,0 +1,29 @@
+# Superset runtime
+PYTHONUNBUFFERED=1
+PYTHONPATH=/app/pythonpath:/app/docker/pythonpath_dev
+FLASK_DEBUG=false
+SUPERSET_ENV=production
+SUPERSET_LOG_LEVEL=info
+SUPERSET_LOAD_EXAMPLES=no
+
+# Superset config — light mode (no Redis/Celery) for base deployment
+SUPERSET_CONFIG_PATH=/app/docker/pythonpath_dev/superset_config_docker_light.py
+
+# Postgres metadata DB
+DATABASE_DIALECT=postgresql
+DATABASE_HOST=postgres-metadata
+DATABASE_PORT=5432
+DATABASE_DB=superset
+DATABASE_USER=superset
+
+# Examples DB (same Postgres instance as metadata)
+EXAMPLES_HOST=postgres-metadata
+EXAMPLES_PORT=5432
+EXAMPLES_DB=examples
+EXAMPLES_USER=examples
+
+# PostGIS (GeoSet geospatial data)
+POSTGIS_HOST=postgis
+POSTGIS_PORT=5432
+POSTGIS_DB=geoset
+POSTGIS_USER=geoset

kustomize/base/kustomization.yaml

@@ -0,0 +1,47 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: geoset
+
+configMapGenerator:
+  - name: superset-env
+    envs:
+      - config/superset.env
+  - name: postgres-metadata-initdb
+    files:
+      - postgres-metadata/examples-init.sh
+  - name: postgis-initdb
+    files:
+      - postgis/init.sql
+
+generatorOptions:
+  disableNameSuffixHash: true
+
+# images:
+#   - name: jmeegan607/geoset
+#     newTag: "6.0.48"
+
+resources:
+  # Cluster setup
+  - namespace.yaml
+  # NOTE: Copy secrets.yaml.example to secrets.yaml and fill in real values
+  # secrets.yaml is gitignored — never commit real secrets
+  - secrets.yaml
+
+  # StatefulSets (data layer)
+  - postgis/statefulset.yaml
+  - postgis/service.yaml
+  - postgres-metadata/statefulset.yaml
+  - postgres-metadata/service.yaml
+
+  # Jobs (init before app starts)
+  - superset-init/job.yaml
+  - sample-data-ingest/job.yaml
+
+  # Deployments (app layer)
+  - superset-web/deployment.yaml
+  - superset-web/service.yaml
+  # superset-frontend is dev-only (webpack dev server) — the production image
+  # already includes built frontend assets. Uncomment for a dev overlay if needed.
+  # - superset-frontend/deployment.yaml
+  # - superset-frontend/service.yaml

kustomize/base/namespace.yaml

@@ -0,0 +1,4 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: geoset

kustomize/base/postgis/init.sql

@@ -0,0 +1,62 @@
+CREATE EXTENSION IF NOT EXISTS postgis;
+
+CREATE TABLE IF NOT EXISTS census_state_boundaries (
+    id SERIAL PRIMARY KEY,
+    state_code VARCHAR(2) NOT NULL,
+    state_gnis_code VARCHAR(8),
+    state_abbrev VARCHAR(2) NOT NULL,
+    full_geoid VARCHAR(14),
+    geoid VARCHAR(2),
+    legal_statistical_code VARCHAR(2),
+    land_area BIGINT,
+    water_area BIGINT,
+    state_name VARCHAR(100) NOT NULL,
+    state_boundary TEXT
+);
+
+CREATE TABLE IF NOT EXISTS nifc_wildfire_locations (
+    id SERIAL PRIMARY KEY,
+    fire_id INTEGER,
+    irwin_id TEXT,
+    incident_size DOUBLE PRECISION,
+    containment_time TIMESTAMPTZ,
+    percent_contained DOUBLE PRECISION,
+    control_time TIMESTAMPTZ,
+    incident_description TEXT,
+    discovery_acres DOUBLE PRECISION,
+    final_acres DOUBLE PRECISION,
+    fire_cause TEXT,
+    origin_coordinate TEXT,
+    dispatch_center_id TEXT,
+    fire_discovery_time TIMESTAMPTZ,
+    nifc_created_time TIMESTAMPTZ,
+    nifc_modified_time TIMESTAMPTZ,
+    estimated_cost_to_date DOUBLE PRECISION,
+    incident_name TEXT,
+    origin_fips_code CHAR(5),
+    origin_city_name TEXT,
+    origin_state_code CHAR(5),
+    origin_county_name TEXT,
+    landowner_type TEXT,
+    is_multijurisdictional BOOLEAN
+);
+
+CREATE TABLE IF NOT EXISTS nhc_best_track (
+    id SERIAL PRIMARY KEY,
+    effective_timestamp TIMESTAMPTZ NOT NULL,
+    min_sea_level_pressure_mb INTEGER,
+    max_gust_mph INTEGER,
+    storm_name TEXT NOT NULL,
+    nhc_identifier TEXT,
+    year INTEGER NOT NULL,
+    observation_point GEOGRAPHY(POINT, 4326)
+);
+
+CREATE INDEX IF NOT EXISTS idx_nhc_identifier ON nhc_best_track (nhc_identifier);
+CREATE INDEX IF NOT EXISTS idx_nhc_year ON nhc_best_track (year);
+
+-- Drop schemas that GeoSet doesn't use so Superset's schema picker only shows public.
+DROP SCHEMA IF EXISTS tiger_data CASCADE;
+DROP SCHEMA IF EXISTS tiger CASCADE;
+DROP SCHEMA IF EXISTS topology CASCADE;
+DROP SCHEMA IF EXISTS information_schema CASCADE;

kustomize/base/postgis/service.yaml

@@ -0,0 +1,11 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: postgis
+spec:
+  clusterIP: None
+  selector:
+    app: postgis
+  ports:
+    - port: 5432
+      targetPort: 5432

kustomize/base/postgis/statefulset.yaml

@@ -0,0 +1,71 @@
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: postgis
+spec:
+  serviceName: postgis
+  replicas: 1
+  selector:
+    matchLabels:
+      app: postgis
+  template:
+    metadata:
+      labels:
+        app: postgis
+    spec:
+      containers:
+        - name: postgis
+          image: postgis/postgis:16-3.4
+          ports:
+            - containerPort: 5432
+          env:
+            - name: POSTGRES_DB
+              valueFrom:
+                configMapKeyRef:
+                  name: superset-env
+                  key: POSTGIS_DB
+            - name: POSTGRES_USER
+              valueFrom:
+                configMapKeyRef:
+                  name: superset-env
+                  key: POSTGIS_USER
+            - name: POSTGRES_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: superset-secrets
+                  key: POSTGIS_PASSWORD
+          resources:
+            requests:
+              cpu: 250m
+              memory: 256Mi
+            limits:
+              cpu: "1"
+              memory: 1Gi
+          volumeMounts:
+            - name: postgis-data
+              mountPath: /var/lib/postgresql/data
+              subPath: pgdata
+            - name: initdb
+              mountPath: /docker-entrypoint-initdb.d
+          readinessProbe:
+            exec:
+              command: ["pg_isready", "-U", "geoset", "-d", "geoset"]
+            initialDelaySeconds: 10
+            periodSeconds: 10
+          livenessProbe:
+            exec:
+              command: ["pg_isready", "-U", "geoset", "-d", "geoset"]
+            initialDelaySeconds: 30
+            periodSeconds: 30
+      volumes:
+        - name: initdb
+          configMap:
+            name: postgis-initdb
+  volumeClaimTemplates:
+    - metadata:
+        name: postgis-data
+      spec:
+        accessModes: ["ReadWriteOnce"]
+        resources:
+          requests:
+            storage: 5Gi

kustomize/base/postgres-metadata/examples-init.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+# Required: Superset needs a separate "examples" database and user in the metadata
+# Postgres for `superset load_examples` to work. This is the k8s equivalent of
+# docker/docker-entrypoint-initdb.d/examples-init.sh in the Superset repo.
+set -e
+psql -v ON_ERROR_STOP=1 --username "${POSTGRES_USER}" <<-EOSQL
+  CREATE USER ${EXAMPLES_USER} WITH PASSWORD '${EXAMPLES_PASSWORD}';
+  CREATE DATABASE ${EXAMPLES_DB};
+  GRANT ALL PRIVILEGES ON DATABASE ${EXAMPLES_DB} TO ${EXAMPLES_USER};
+EOSQL
+psql -v ON_ERROR_STOP=1 --username "${POSTGRES_USER}" -d "${EXAMPLES_DB}" <<-EOSQL
+  GRANT ALL ON SCHEMA public TO ${EXAMPLES_USER};
+EOSQL