Skip to content

Commit 92b487d

Browse files
JPadovano1483JPadovano1483
authored
vms helm deployment (#64)
* controller and postgres deploy if cert-manager is pre-installed * cert-manager install working * converted to helmfile for dependency management * templatize postgres config to apply to multiple releases * deploy works with clusterIP + port-forward * deploy skupper for cross cluster connectivity * remove unneeded files and values * local keycloak deploy working * configurable keycloak deployment * persistent keycloak * cleanup variables * updated README * fixes connection pool passwords * fixed chart defaults * use secrets instead of plain text * removed keycloak and multicluster * update README * point getting-started.md to helmfile * revert management-controller.yaml changes * removing /scripts and /yaml directories * set management-server chart to use production node env
1 parent 46a491b commit 92b487d

30 files changed

Lines changed: 904 additions & 346 deletions

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ SkupperVMS management plane components are deployed on Kubernetes. Backbone and
6767

6868
#### Database
6969

70-
A relational database (Postgres) is used as the central persistent store of configuration and current state. The schema for the database can be found in `scripts/db-setup.sql`.
70+
A relational database (Postgres) is used as the central persistent store of configuration and current state. The schema for the database can be found in `charts/helmfile/resources/db-setup.sql` (optional teardown script: `charts/helmfile/resources/drop.sql`).
7171

7272
#### Certificate Management
7373

‎charts/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Charts
2+
3+
Helmfile and Helm charts for deploying cert-manager, PostgreSQL, and the management server live under **`helmfile/`**. Full deployment steps are in **[helmfile/README.md](./helmfile/README.md)**.
4+
5+
PostgreSQL schema (`db-setup.sql`) and the optional teardown helper (`drop.sql`) live under **`helmfile/resources/`**.

‎charts/helmfile/README.md‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# VMS deployment
2+
3+
The **`charts/helmfile`** directory is a [Helmfile](https://github.com/helmfile/helmfile) environment that installs:
4+
5+
| Chart | Role |
6+
| ----- | ---- |
7+
| **cert-manager** | Jetstack OCI chart (`v1.20.0`) — TLS issuers / certificates (optional). |
8+
| **postgresql** | Bitnami `postgresql` `18.3.0` — application database; schema from `resources/db-setup.sql`. |
9+
| **management-server** | Local chart at `../management-server` — VMS management controller. This chart can be deployed/managed by itself with standard Helm commands. |
10+
11+
Helmfile uses your **current** `kubectl` context. Namespace behavior is described below.
12+
13+
Keycloak is **not** installed here. Supply the controller with a **`keycloak-config`** Secret as in [Keycloak adapter](#keycloak-adapter).
14+
15+
## Layout under `charts/`
16+
17+
```
18+
charts/
19+
├── helmfile/ # This README
20+
│ ├── helmfile.yaml.gotmpl
21+
│ ├── values/
22+
│ │ ├── common.yaml # postgres.* + releases.*
23+
│ │ ├── postgres.yaml.gotmpl
24+
│ │ └── management-server.yaml.gotmpl
25+
│ └── resources/
26+
│ ├── db-setup.sql # Applied to Postgres on init (via ConfigMap)
27+
│ └── drop.sql # Optional manual teardown helper (not used by Helmfile)
28+
└── management-server/ # Helm chart for the management controller
29+
```
30+
31+
## Prerequisites
32+
33+
- **[kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)** (1.15+).
34+
35+
- **[Helm](https://helm.sh/docs/intro/install/)** and **[Helmfile](https://github.com/helmfile/helmfile)** on your `PATH`.
36+
37+
- **[helm-diff](https://github.com/databus23/helm-diff)**:
38+
39+
```shell
40+
helm plugin install https://github.com/databus23/helm-diff
41+
```
42+
43+
- Keycloak instance running and configured (see [Keycloak Setup](/docs/notes/keycloak-setup.md)).
44+
45+
## Credentials (Kubernetes Secrets)
46+
47+
Passwords are **not** stored in `values/common.yaml`. Create Secrets **before** install; names and keys come from **`postgres.credentialsSecret`** (defaults below).
48+
49+
### PostgreSQL + management-server
50+
51+
Create **`postgres-credentials`** (default name from `common.yaml`) in:
52+
53+
1. The **same namespace as the PostgreSQL release**.
54+
2. The **same namespace as the management-server release**.
55+
56+
If Postgres and management-server run in different namespaces, duplicate the Secret in both with identical keys.
57+
58+
| Key (defaults in `common.yaml`) | Purpose |
59+
| ------------------------------- | ------- |
60+
| `postgres-password` | Bitnami superuser (`auth.secretKeys.adminPasswordKey`). |
61+
| `app-user-password` | `db-setup.sql` and management controller `APP_USER_PASSWORD`. |
62+
| `app-system-password` | `db-setup.sql` and management controller `APP_SYSTEM_PASSWORD`. |
63+
64+
Example:
65+
66+
```shell
67+
kubectl create secret generic postgres-credentials \
68+
--from-literal=postgres-password='REPLACE_SUPERUSER_PASSWORD' \
69+
--from-literal=app-user-password='REPLACE_APP_USER_PASSWORD' \
70+
--from-literal=app-system-password='REPLACE_APP_SYSTEM_PASSWORD' \
71+
-n <namespace>
72+
```
73+
74+
## Keycloak adapter
75+
76+
The **management-server** chart expects a Secret **`keycloak-config`** with key **`keycloak.json`**. Helmfile does not create it.
77+
78+
1. Create a Keycloak instance that the management-controller can connect to.
79+
2. Configure the client per [Keycloak setup guide](/docs/notes/keycloak-setup.md).
80+
3. Create the Secret in the management-server namespace:
81+
82+
```shell
83+
kubectl create secret generic keycloak-config \
84+
--from-file=/path/to/your-keycloak.json \
85+
-n <management-server-namespace>
86+
```
87+
88+
## Configuration (`values/common.yaml`)
89+
90+
### `releases`
91+
92+
Toggles and PostgreSQL namespace (from inline comments in `common.yaml`):
93+
94+
| Key | Purpose |
95+
| --- | ------- |
96+
| `releases.certManager.enabled` | Install Jetstack cert-manager into namespace **`cert-manager`** (created if missing). |
97+
| `releases.postgresql.enabled` | Install Bitnami PostgreSQL. |
98+
| `releases.postgresql.namespace` | If **non-empty**, PostgreSQL is installed in that namespace (`createNamespace: true`). If **empty**, the release uses the current namespace used when running the helmfile command. |
99+
| `releases.managementServer.enabled` | Install `../management-server`. Namespace follows Helmfile’s default unless you set release-level namespace in `helmfile.yaml.gotmpl`. |
100+
101+
### `postgres`
102+
103+
Used by **`values/postgres.yaml.gotmpl`** for Bitnami auth, persistence, and init SQL env vars. Passed through **`values/management-server.yaml.gotmpl`** as `PGHOST`, `PGPORT`, `PGDATABASE`, and `credentialsSecret` for app role passwords.
104+
105+
Set **`postgres.host`** / **`postgres.port`** to a hostname and port reachable from management-server pods (for example `postgresql` or `postgresql.<namespace>.svc.cluster.local`).
106+
107+
## Database init hook
108+
109+
Before sync, Helmfile runs a **`presync`** hook that applies ConfigMap **`db-init-configmap`** from **`resources/db-setup.sql`**. The `kubectl` command passes **`-n <namespace>`** when **`releases.postgresql.namespace`** is set; otherwise it uses your current kubectl namespace context—align that with where PostgreSQL is installed so the ConfigMap lands in the correct namespace.
110+
111+
## Components (Helm releases)
112+
113+
| Release | Source | Installed when |
114+
| ------- | ------ | -------------- |
115+
| `cert-manager` | `oci://quay.io/jetstack/charts/cert-manager` | `releases.certManager.enabled` |
116+
| `postgresql` | `bitnami/postgresql` `18.3.0` | `releases.postgresql.enabled` |
117+
| `management-server` | `../management-server` | `releases.managementServer.enabled` |
118+
119+
Select a specific release to manage with **`helmfile -l component=<label>`** (labels are `cert-manager`, `postgresql`, `management-server` in `helmfile.yaml.gotmpl`).
120+
121+
## Deploying
122+
123+
1. Edit **`values/common.yaml`**: **`releases.*`**, **`postgres.*`**.
124+
2. Create **`postgres-credentials`** (and **`keycloak-config`** when ready) in the required namespaces.
125+
3. From **`charts/helmfile`**:
126+
127+
```shell
128+
cd charts/helmfile
129+
helmfile sync
130+
```
131+
132+
Use **`helmfile apply`** for incremental diffs. To release an individual chart, run `helmfile -l component=<label> apply`.
133+
134+
## Teardown
135+
136+
To destroy all resources created by the Helmfile releases (including the management server, PostgreSQL, and cert-manager, if installed), use:
137+
138+
```shell
139+
helmfile destroy
140+
```
141+
142+
You can selectively destroy a single release (component) using the label, just like for deploys. For example, to tear down only the Bitnami PostgreSQL release:
143+
144+
```shell
145+
helmfile -l component=postgresql destroy
146+
```
147+
148+
Or, for cert-manager:
149+
150+
```shell
151+
helmfile -l component=cert-manager destroy
152+
```
153+
154+
This will remove the resources managed by the specified release from your cluster. Double-check your namespace/context to ensure the correct resources are modified.
155+
156+
## Related documentation
157+
158+
- **[getting-started](/docs/notes/getting-started.md)** — broader VMS setup. This file documents only **`charts/helmfile`** and the **`management-server`** chart path used from it.

‎charts/helmfile/helmfile.yaml.gotmpl‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
environments:
2+
default:
3+
values:
4+
- ./values/common.yaml
5+
---
6+
{{- $deployCertManager := .Values.releases.certManager.enabled }}
7+
{{- $deployPostgresql := .Values.releases.postgresql.enabled }}
8+
{{- $deployManagementServer := .Values.releases.managementServer.enabled }}
9+
{{- $postgresqlNamespace := .Values.releases.postgresql.namespace }}
10+
{{- if ne $postgresqlNamespace "" }}
11+
{{- $postgresqlNamespace = printf "-n %s" $postgresqlNamespace }}
12+
{{- end }}
13+
14+
repositories:
15+
- name: bitnami
16+
url: https://charts.bitnami.com/bitnami
17+
18+
releases:
19+
- name: cert-manager
20+
namespace: cert-manager
21+
createNamespace: true
22+
chart: oci://quay.io/jetstack/charts/cert-manager
23+
version: v1.20.0
24+
installed: {{ $deployCertManager }}
25+
labels:
26+
component: cert-manager
27+
values:
28+
- crds:
29+
enabled: true
30+
keep: true
31+
- name: postgresql
32+
{{- if ne .Values.releases.postgresql.namespace "" }}
33+
namespace: {{ .Values.releases.postgresql.namespace }}
34+
{{- end }}
35+
createNamespace: true
36+
chart: bitnami/postgresql
37+
version: 18.3.0
38+
installed: {{ $deployPostgresql }}
39+
labels:
40+
component: postgresql
41+
values:
42+
- ./values/postgres.yaml.gotmpl
43+
hooks:
44+
- events: ["presync"]
45+
showlogs: true
46+
command: sh
47+
args:
48+
- -c
49+
- |
50+
kubectl create configmap db-init-configmap {{ $postgresqlNamespace }} --from-file=./resources/db-setup.sql -o yaml --dry-run=client | kubectl apply -f -
51+
- name: management-server
52+
chart: ../management-server
53+
installed: {{ $deployManagementServer }}
54+
labels:
55+
component: management-server
56+
values:
57+
- ./values/management-server.yaml.gotmpl

scripts/db-setup.sql renamed to charts/helmfile/resources/db-setup.sql

File renamed without changes.

scripts/drop.sql renamed to charts/helmfile/resources/drop.sql

File renamed without changes.

‎charts/helmfile/values/common.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Shared Postgres settings (environments.default.values). Consumed by values/*.gotmpl.
2+
#
3+
# host: PGHOST (in-cluster Service name, usually postgresql).
4+
# port: PGPORT.
5+
# user: database bootstrap user for Bitnami PostgreSQL (typically postgres).
6+
# database: PGDATABASE / Bitnami created database name.
7+
# credentialsSecret — create this Secret in the PostgreSQL release namespace before
8+
# helmfile sync. The management-server chart reads the same secret name in its own
9+
# namespace for APP_* passwords; duplicate the Secret there if namespaces differ.
10+
# Required keys (unless you override key names below):
11+
# postgres-password — Bitnami superuser password (see adminPasswordKey)
12+
# app-user-password — passed to db-setup.sql for CREATE ROLE app_user
13+
# app-system-password — passed to db-setup.sql for CREATE ROLE app_system
14+
# persistence:
15+
# enabled: whether to deploy a pv for postgres.
16+
# size: size of the pv.
17+
# storageClass: storage class if postgres persistence is enabled. Leave empty for default storage class.
18+
postgres:
19+
host: postgresql
20+
port: 5432
21+
user: postgres
22+
database: studiodb
23+
credentialsSecret:
24+
name: postgres-credentials
25+
adminPasswordKey: postgres-password
26+
appUserPasswordKey: app-user-password
27+
appSystemPasswordKey: app-system-password
28+
persistence:
29+
enabled: true
30+
size: 5Gi
31+
storageClass: ""
32+
imagePullSecret: ""
33+
34+
# Release configuration:
35+
# certManager: installed into the cert-manager namespace (will be created if not exists)
36+
# postgresql: installed into the namespace specified in the namespace key (will be created if not exists). If namespace is empty, the release will be installed into the current namespace.
37+
# managementServer: installed into the current namespace.
38+
releases:
39+
certManager:
40+
enabled: true
41+
postgresql:
42+
enabled: true
43+
namespace: ""
44+
managementServer:
45+
enabled: true

‎charts/helmfile/values/management-server.yaml.gotmpl‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
postgres:
2+
host: {{ .Values.postgres.host | quote }}
3+
port: {{ .Values.postgres.port | quote }}
4+
database: {{ .Values.postgres.database | quote }}
5+
credentialsSecret:
6+
{{- toYaml .Values.postgres.credentialsSecret | nindent 4 }}

‎charts/helmfile/values/postgres.yaml.gotmpl‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{{- $pg := .Values.postgres | required "environment values must define 'postgres' (charts/helmfile/values/common.yaml)" }}
2+
{{- $sec := $pg.credentialsSecret | required "postgres.credentialsSecret is required (create the Secret before install; see charts/helmfile/README.md)" }}
3+
{{- $pullSecret := .Values.postgres.imagePullSecret | default "" }}
4+
{{- if ne $pullSecret "" }}
5+
global:
6+
imagePullSecrets:
7+
- name: {{ $pullSecret | quote }}
8+
{{- end }}
9+
auth:
10+
username: {{ $pg.user | quote }}
11+
database: {{ $pg.database | quote }}
12+
existingSecret: {{ $sec.name | quote }}
13+
secretKeys:
14+
adminPasswordKey: {{ $sec.adminPasswordKey | quote }}
15+
primary:
16+
initdb:
17+
scriptsConfigMap: db-init-configmap
18+
service:
19+
type: ClusterIP
20+
ports:
21+
postgresql: {{ $pg.port | quote }}
22+
persistence:
23+
enabled: {{ $pg.persistence.enabled }}
24+
size: {{ $pg.persistence.size | default "5Gi" }}
25+
{{- if ne $pg.persistence.storageClass "" }}
26+
storageClass: {{ $pg.persistence.storageClass | quote }}
27+
{{- end }}
28+
extraEnvVars:
29+
- name: APP_USER_PASSWORD
30+
valueFrom:
31+
secretKeyRef:
32+
name: {{ $sec.name | quote }}
33+
key: {{ $sec.appUserPasswordKey | quote }}
34+
- name: APP_SYSTEM_PASSWORD
35+
valueFrom:
36+
secretKeyRef:
37+
name: {{ $sec.name | quote }}
38+
key: {{ $sec.appSystemPasswordKey | quote }}

‎charts/management-server/.helmignore‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

0 commit comments

Comments
 (0)