Skip to content

Commit 8e781fd

Browse files
committed
add images, improve docs
1 parent fb74c68 commit 8e781fd

5 files changed

Lines changed: 44 additions & 15 deletions

File tree

145 KB
Loading
286 KB
Loading
85.1 KB
Loading
168 KB
Loading

docs/modules/demos/pages/argo-cd-git-ops.adoc

Lines changed: 44 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
:sealed-secrets: https://github.com/bitnami-labs/sealed-secrets
99
:stackable-demo-repository: https://github.com/stackabletech/demos/
1010
:airflow: https://airflow.apache.org/
11+
:github-fork: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo
1112

1213
This demo shows how to utilize GitOps and Infrastructure as Code (IaC) with Stackable and {argo-cd}[ArgoCD].
1314
All products and manifests are synced and deployed via ArgoCD (except ArgoCD itself, which is bootstrapped via `stackablectl`).
@@ -24,7 +25,7 @@ image::argo-cd-git-ops/architecture-overview.drawio.svg[]
2425

2526
Install this demo on an existing Kubernetes cluster:
2627

27-
TODO: We need a pointer here to the last paragraph (forking repo) if people want to "interact" and not just install the demo (install command requires extra parameters)
28+
// TODO: We need a pointer here to the last paragraph (forking repo) if people want to "interact" and not just install the demo (install command requires extra parameters)
2829

2930
[source,console]
3031
----
@@ -87,21 +88,21 @@ This can be ignored in this case.
8788

8889
The ArgoCD Web UI entry page shows an overview of deployed applications, their status and other metadata as the repository or the date of the last synchronization run.
8990

90-
TODO: Screenshot - Argo UI overview
91+
// TODO: Screenshot - Argo UI overview
9192

9293
Single applications can be inspected closer after clicking on e.g. the `airflow` project.
9394

94-
TODO: Screenshot - Argo UI - Airflow
95+
// TODO: Screenshot - Argo UI - Airflow
9596

9697
Detailed information about the Airflow cluster, the cluster status and deployed components can be accessed in the application details.
9798
Additionally, if the Git repository and the cluster state itself differ, these differences can be previewed in a code diff preview.
9899

99-
TODO: Screenshot - Argo UI - application details Network / List tab -> mark network tabs etc.
100+
// TODO: Screenshot - Argo UI - application details Network / List tab -> mark network tabs etc.
100101

101102
Per default in this demo, the ArgoCD `Sync Policy` is set to `auto-sync`. This means that changes to the Git repository are immediatly synced into the cluster.
102103
This is nice in the demo case, but should be disabled for production use cases.
103104

104-
TODO: Screenshot - Argo UI - applications/airflow/details
105+
// TODO: Screenshot - Argo UI - applications/airflow/details
105106

106107
Now, after a quick overview of the ArgoCD web UI, the following part demonstrates how to sync and deploy Stackable operators via ArgoCD.
107108

@@ -224,7 +225,7 @@ template:
224225
This allows control over which releases and versions are deployed to which cluster.
225226
====
226227

227-
Now with ArgoCD deployed, the Sealed Secrets controller and Stackable operators up and running, you can inspect Airflow as the first Stackable product.
228+
Now with ArgoCD, the Sealed Secrets controller and Stackable operators up and running, you can inspect Airflow as the first Stackable product.
228229

229230
== Airflow
230231

@@ -236,12 +237,30 @@ kubectl --namespace stackable-airflow port-forward service/airflow-webserver 808
236237
----
237238

238239
In your browser, go to `http://localhost:8080` and login with username `admin` and password `adminadmin`.
240+
The welcome page and an overview over synced DAGs should be displayed.
239241

240-
=== Starting DAGs
242+
image::argo-cd-git-ops/airflow-welcome-page.png[]
241243

242-
TODO: screenshots?
244+
=== Start the DAG
243245

244-
== Minio / S3 - check logs
246+
The `date_demo` DAG can be started by moving the slider and trigger the DAG runs. The DAG itself can be inspected by clicking on it.
247+
248+
=== Inspect the DAG
249+
250+
The overview displays details about the DAG runs, durations and other metadata. The graph, code or events can be selected in the tabs for more details.
251+
252+
image::argo-cd-git-ops/airflow-dag-overview.png[]
253+
254+
=== Inspect a DAG run
255+
256+
A single DAG run can be selected by clicking on one of the green squares next to `run_every_minute` on the left.
257+
More information is displayed here, and the DAG logs written by the Kubernetes Executor to S3/Minio can be selected in the `Logs` tab.
258+
259+
image::argo-cd-git-ops/airflow-dag-run-logs.png[]
260+
261+
In the logs, the output of the DAG is printed under a line containing `Output:`, the timestamp of the DAG run.
262+
263+
== Minio
245264

246265
Since the Airflow Kubernetes Executor will be deleted after its run, the logs are written to an S3 bucket. When accessing the logs via the Airflow webserver,
247266
the logs are fetched from S3 instead of the (already deleted) executor pods. The Minio / S3 instance can be accessed via port-forward:
@@ -257,27 +276,37 @@ After the successful Airflow DAG run, logs should be stored in `demo/airflow-tas
257276
NOTE: There will be an initial warning from the browser, stating that the site is insecure due to self-signed certificates.
258277
This can be ignored in this case.
259278

260-
TODO: Screenshot - minio UI with log data
279+
image::argo-cd-git-ops/minio-dag-run-logs.png[]
280+
281+
The log files contained in the single folders are the same as the logs shown above in the Airflow web UI.
282+
283+
== Conclusion
261284

262-
As a last step, in order to better interact and not just "sync" from the Git repository, the following paragraph demonstrates how to fork the demo repository and adapt the `stackablectl` install command to point to the forked repository.
263-
In this forked repository, changes can be made the code and synced into the cluster via ArgoCD.
285+
This demo illustrates the combination of ArgoCD and Stackable, using the full potential of GitOps and demonstrating key features for a successful IaC deployment using Stackable.
286+
287+
// TODO: extend...
288+
289+
As a last step, in order to synchronize changes made to manifests or secrets from the Git repository, the following paragraph demonstrates on how to run this demo on a forked GitHub repository.
264290

265291
== How to interact with ArgoCD and the Git repository
266292

267293
Since this Demo is hosted in the {stackable-demo-repository}[Stackable Demo repository], where merging etc. requires approval, the recommendation is to fork the {stackable-demo-repository}[Stackable Demo repository].
294+
This {github-fork}[GitHub tutorial] shows how to fork a repository.
268295

269-
Once forked, you can install this demo using `stackablectl` parameters to customize the forked repository:
296+
After forking the demo repository, a local copy can be cloned and the `stackablectl` install command must be parameterized with the fork URL and branch.
270297

271298
[source,console]
272299
----
273300
stackablectl demo install argo-cd-git-ops --namespace argo-cd --parameters customGitUrl=<my-demo-fork-url> --parameters customGitBranch=<my-custom-branch-with-changes>
274301
----
275302

276-
This way, ArgoCD is instructed to pull the Stackable manifests from the forked repository, where your changes can be properly synced via ArgoCD.
303+
In this forked and cloned repository, changes can be made the code and synced into the cluster via ArgoCD.
304+
305+
This way, ArgoCD is instructed to pull the Stackable manifests from the forked repository, where the changes are synced via ArgoCD into the cluster.
277306

278307
=== Increase Airflow webserver replicas
279308

280-
Assuming your working directory ist the root of the forked demo repository, try to increase the `spec.webservers.roleGroups.<role-group>.replicas` in the folder `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml`.
309+
Assuming your working directory is the root of the forked demo repository, try to increase the `spec.webservers.roleGroups.<role-group>.replicas` in the folder `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml`.
281310
Once this is pushed / merged, ArgoCD should sync the changes and you should see more webserver pods.
282311

283312
=== Add new Airflow DAGs

0 commit comments

Comments
 (0)