You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/demos/pages/argo-cd-git-ops.adoc
+39-66Lines changed: 39 additions & 66 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,8 +14,8 @@ All products and manifests are synced and deployed via ArgoCD (except ArgoCD its
14
14
15
15
The key points to show are:
16
16
17
-
* GitOps: How to deploy changes from a Git repository to my cluster?
18
-
* Secrets: How to safely store and deploy credentials, access keys from a Git repository?
17
+
* GitOps: How to deploy changes from a Git repository to a cluster?
18
+
* Secrets: How to safely store and deploy credentials or access keys from a Git repository?
19
19
* Interaction with Stackable products (e.g. Airflow and DAGs via gitsync)
20
20
21
21
ArgoCD and other deployed products and dependencies are illustrated in the following diagram:
@@ -49,22 +49,20 @@ To run this demo, your system needs at least:
49
49
50
50
This demo consists of multiple parts:
51
51
52
-
* Bootstrapping via `stackablectl`:
53
-
** Install {argo-cd}[Argo CD] via `stackablectl`
52
+
* Bootstrapping {argo-cd}[Argo CD] via `stackablectl`:
54
53
* Deploy components via ArgoCD:
55
-
** Install a {sealed-secrets}[Sealed Secrets] controller to handle sensitive data like credentials or secret keys
56
-
** Install all Stackable operators using an `ApplicationSet`
57
-
** Install requirements like Minio and Postgres as `Application`
58
-
** Deploy Stackable Airflow manifests into their respective `Projects`
59
-
* Inspect ArgoCD UI
60
-
* Inspect Airflow and start DAGs
61
-
** Checkout Airflow UI and synced DAGs (the DAGs are synced via gitsync, not ArgoCD)
62
-
** Start a DAG and inspect the logs written to S3 / Minio by the Kubernetes executor
63
-
* Optional: Inspect Minio UI for logs
54
+
** Install a {sealed-secrets}[Sealed Secrets] controller to decrypt secrets stored in a Git repository
55
+
** Install all Stackable operators using an ArgoCD `ApplicationSet`
56
+
** Install the Airflow dependencies Minio and Postgres as ArgoCD `Application`
57
+
** Deploy Stackable Airflow manifests into their respective ArgoCD `Projects`
58
+
* Inspect Airflow web UI
59
+
** DAGs are synced from a Git repository
60
+
** Start a DAG and check the results
61
+
** Check Kubernetes Executor logs persisted in Minio
64
62
65
63
== Sealed Secrets
66
64
67
-
When managing all resources and configs via Git, deploying sensitive properties like certificates or credentials via Git becomes a problem.
65
+
When managing all resources and configuration via Git, deploying sensitive data like certificates or credentials via Git becomes a problem.
68
66
69
67
There are multiple solutions to this such as Hashicorp or Bitwarden, which depend heavily on the infrastructure already available.
70
68
@@ -75,8 +73,7 @@ This way, everything will be stored and managed in Git.
75
73
76
74
== ArgoCD UI
77
75
78
-
ArgoCD will be the first product that is deployed in this Demo.
79
-
Once the pods are ready, you can port-forward the argocd-server in order to access the web UI.
76
+
After bootstrapping ArgoCD via `stackablectl`, once the pods are ready, you can port-forward the argocd-server Service in order to access the web UI.
80
77
81
78
[source,console]
82
79
----
@@ -88,16 +85,16 @@ In your browser, go to `https://localhost:8443` and login with username `admin`
88
85
NOTE: There will be an initial warning from the browser, stating that the site is insecure due to self-signed certificates.
89
86
This can be ignored in this case.
90
87
91
-
The ArgoCD Web UI entry page shows an overview of deployed applications and their status and other metadata as the repository or the date of the last synchronization run.
88
+
The ArgoCD Web UI entry page shows an overview of deployed applications, their status and other metadata as the repository or the date of the last synchronization run.
92
89
93
90
TODO: Screenshot - Argo UI overview
94
91
95
92
Single applications can be inspected closer after clicking on e.g. the `airflow` project.
96
93
97
94
TODO: Screenshot - Argo UI - Airflow
98
95
99
-
Detailed information about the cluster, the cluster status and deployed components can be accessed in the application details.
100
-
Additionally, if the Git repository and the cluster state itself differ, these differences can be previewed in a code diff similar to Git pull requests.
96
+
Detailed information about the Airflow cluster, the cluster status and deployed components can be accessed in the application details.
97
+
Additionally, if the Git repository and the cluster state itself differ, these differences can be previewed in a code diff preview.
101
98
102
99
TODO: Screenshot - Argo UI - application details Network / List tab -> mark network tabs etc.
103
100
@@ -116,11 +113,13 @@ using different versions and Git sources (repository & branch) as well as the po
116
113
117
114
NOTE: This demo does not use a multi cluster environment for the sake of simplicity.
118
115
119
-
TODO: Do the code snippets actually help? Maybe text is better... its about showing at least some parts of a multi cluster setup...
116
+
The following part is dives deeper into the definition of theStackable operator `ApplicationSet` and can be skipped.
120
117
118
+
.Stackable operators `ApplicationSet` details
119
+
[%collapsible]
120
+
====
121
121
[source,yaml]
122
122
----
123
-
---
124
123
apiVersion: argoproj.io/v1alpha1
125
124
kind: ApplicationSet
126
125
metadata:
@@ -130,7 +129,7 @@ spec:
130
129
- matrix:
131
130
generators:
132
131
- list:
133
-
elements:
132
+
elements: # <1>
134
133
- operator: commons
135
134
- operator: listener
136
135
- operator: secret
@@ -147,7 +146,7 @@ spec:
147
146
- operator: trino
148
147
- operator: zookeeper
149
148
- list:
150
-
elements:
149
+
elements: # <2>
151
150
- cluster: demo
152
151
server: https://kubernetes.default.svc
153
152
targetRevision: 25.7.0
@@ -176,6 +175,8 @@ spec:
176
175
# targetRevision: 25.3.0
177
176
# [...]
178
177
----
178
+
<1> List of Stackable operators to install.
179
+
<2> List of clusters and Stackable release versions for each cluster.
179
180
180
181
The `matrix.generators.list[].elements[]` will create a union of parameters that may be used in the `ApplicationSet` template as follows:
The templated version for e.g. the parameters `operator=zookeeper`, `server=https://kubernetes.default.svc` and `targetRevision=25.7.0` will result in the following template:
<2> The Stackable release version, e.g. `25.7.0` (templated from the matrix generators).
218
+
<3> The Chart name in the repository e.g. `"sdp-charts/airflow-operator"` (templated from the matrix generators).
219
+
<4> The Helm release name e.g. `airflow-operator` (templated from the matrix generators).
220
+
<5> The Kubernetes cluster, e.g. `https://kubernetes.default.svc` for this demo (templated from the matrix generators).
221
+
<6> The namespace to deploy the operators in. Will be created if `spec.syncPolicy.syncOptions[].CreateNamespace` is set to true.
222
+
<7> Automatically create missing namespaces.
251
223
252
224
This allows control over which releases and versions are deployed to which cluster.
225
+
====
253
226
254
227
Now with ArgoCD deployed, the Sealed Secrets controller and Stackable operators up and running, you can inspect Airflow as the first Stackable product.
0 commit comments