extend paragraphs, motivation, conclustion, git interaction

Author: maltesander

docs/modules/demos/pages/argo-cd-git-ops.adoc

@@ -12,25 +12,46 @@
 :airflow-git-sync: https://docs.stackable.tech/home/stable/airflow/usage-guide/mounting-dags/#_via_git_sync
 :github-fork: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo
 
-This demo shows how to utilize GitOps and Infrastructure as Code (IaC) with Stackable and {argo-cd}[ArgoCD].
-Basic knowledge about ArgoCD and its {argo-cd-core-concepts}[core concepts] are assumed and not explained further.
+== Motivation
+
+Modern Kubernetes environments thrive on automation, reproducibility, and strong version control. 
+GitOps provides a reliable way to manage infrastructure and applications declaratively: your desired cluster state is stored in Git,
+and changes are automatically synchronized. This tutorial demonstrates how to deploy Stackable operators and products using {argo-cd}[ArgoCD],
+following Infrastructure-as-Code (IaC) and GitOps principles.
+
+By the end, you will learn how to:
+* Deploy Stackable operators and products via Git-managed manifests.
+* Use Sealed Secrets to securely manage sensitive credentials.
+* Update Airflow DAGs or modify deployments simply by committing to Git.
+
+This hands-on approach illustrates how GitOps can simplify application lifecycle management and enforce a clear,
+auditable workflow across environments (development, staging, production).
 
 All products and manifests are synced and deployed via ArgoCD (except ArgoCD itself, which is bootstrapped via `stackablectl`).
 
-The key points to show are:
+[#system-requirements]
+== System requirements
+
+To run this demo, ensure you have the following prerequisites:
+* Kubernetes Cluster (e.g., Kind, Minikube, or managed services like EKS, GKE, AKS)
+* `kubectl`: Installed and configured to access your cluster.
+* Optional: GitHub Account - Required for forking and interacting with the demo repository.
+
+Resource requirements for this demo:
+* 15 {k8s-cpu}[cpu units] (core/hyperthread)
+* 15 GiB memory
+* 5 GiB disk storage
+
+Basic knowledge about ArgoCD and its {argo-cd-core-concepts}[core concepts] are assumed and not explained further.
 
-* Infrastructure - GitOps
-** How to synchronize changes from a Git repository?
-** How to apply the Git version and branch control to different Kubernetes clusters (development, staging, production)?
-** How to safely store and deploy sensitive data from a Git repository?
-* Interaction with Stackable products and Git
-** Use Airflow and {airflow-git-sync}[git-sync] to fetch DAGs from a Git repository
-** Add or update existing DAGs by commiting to the Git repository
+## Architecture
 
 ArgoCD and other deployed products and dependencies are illustrated in the following diagram:
 
 image::argo-cd-git-ops/architecture-overview.drawio.svg[]
 
+## Installation
+
 Install this demo on an existing Kubernetes cluster:
 
 NOTE: In order to interact with the Git repository, this repository must be forked and additional parameters must be provided to `stackablectl`.
@@ -46,30 +67,6 @@ WARNING: This demo should not be run alongside other demos.
 NOTE: ArgoCD will be deployed in the `argo-cd` namespace by `stackablectl`.
 ArgoCD itself will create other namespaces for the deployed products.
 
-[#system-requirements]
-== System requirements
-
-To run this demo, your system needs at least:
-
-* 15 {k8s-cpu}[cpu units] (core/hyperthread)
-* 15 GiB memory
-* 5 GiB disk storage
-
-== Overview
-
-This demo consists of multiple parts:
-
-* Bootstrapping {argo-cd}[Argo CD]  via `stackablectl`:
-* Deploy components via ArgoCD:
-** Install a {sealed-secrets}[Sealed Secrets] controller to decrypt secrets stored in a Git repository
-** Install all Stackable operators using an ArgoCD `ApplicationSet`
-** Install the Airflow dependencies Minio and Postgres as ArgoCD `Application`
-** Deploy Stackable Airflow manifests into their respective ArgoCD `Projects`
-* Inspect Airflow web UI
-** DAGs are synced from a Git repository
-** Start a DAG and check the results
-** Check Kubernetes Executor logs persisted in Minio
-
 == Sealed Secrets
 
 When managing all resources and configuration via Git, deploying sensitive data like certificates or credentials via Git becomes a problem.
@@ -123,7 +120,7 @@ using different versions and Git sources (repository & branch) as well as the po
 
 NOTE: This demo does not use a multi cluster environment for the sake of simplicity.
 
-The following part is dives deeper into the definition of theStackable operator `ApplicationSet` and can be skipped.
+The following part dives deeper into the definition of the Stackable operator `ApplicationSet` and can be skipped.
 
 .Stackable operators `ApplicationSet` details
 [%collapsible]
@@ -294,44 +291,109 @@ The log files contained in the single folders are the same as the logs shown abo
 
 == Conclusion
 
-This demo illustrates the combination of ArgoCD and Stackable, using the full potential of GitOps and demonstrating key features for a successful IaC deployment using Stackable.
+This demo illustrates a repeatable blueprint for managing complex data platform components with ArgoCD and GitOps. 
+Once familiar with this pattern, you can extend it to multi-cluster environments, add CI/CD pipelines for automated manifest testing, 
+or integrate external secret stores like HashiCorp Vault for production use. This setup lays the foundation for a fully automated, scalable, and secure Kubernetes-based data platform.
+
+This tutorial demonstrates how ArgoCD and Stackable integrate to deliver a streamlined GitOps experience:
+* All cluster resources and workloads are managed declaratively via Git.
+* ArgoCD continuously ensures the cluster state matches Git.
+* Sealed Secrets provide secure and auditable secret management.
+* Airflow DAG updates occur automatically by committing code to the repository.
 
-// TODO: extend...
+This approach scales naturally across environments—development, staging, and production—while reducing manual operations, improving visibility, 
+and enforcing consistency. By adopting GitOps with ArgoCD and Stackable, teams gain a clear, auditable, and automated path from code to production.
 
-The last step is to demonstrate synchronizing changes made to manifests or secrets from the Git repository (to do this the demo is run on a forked GitHub repository).
+Next steps:
+* Explore multi-cluster ApplicationSet deployments to target multiple Kubernetes clusters.
+* Integrate CI workflows to automatically validate and merge manifest updates.
+* Expand beyond Airflow to manage additional Stackable components (e.g., Kafka, Trino, Superset).
+* Experiment with DataOps (e.g., Airflow and Trino)
 
 [#interact-with-git-repository]
 == How to interact with ArgoCD, Airflow and the Git repository
 
 Since this Demo is hosted in the {stackable-demo-repository}[Stackable Demo repository], where merging etc. requires approval, the recommendation is to fork the {stackable-demo-repository}[Stackable Demo repository].
+
+=== Forking the demo repository
+
 This {github-fork}[GitHub tutorial] shows how to fork a repository.
 
+=== Cloning the demo repository
+
+Clone the demo repository:
+
+[source,console]
+----
+git clone https://github.com/<your-username>/demos.git
+cd demos
+----
+
 After forking the demo repository, a local copy can be cloned and the `stackablectl` install command must be parameterized with the fork URL and branch.
 
 [source,console]
 ----
 stackablectl demo install argo-cd-git-ops --namespace argo-cd --parameters customGitUrl=<my-demo-fork-url> --parameters customGitBranch=<my-custom-branch-with-changes>
 ----
 
-In this forked and cloned repository, changes can be made the code and synced into the cluster via ArgoCD.
+=== Making changes to the repository
+
+Edit manifests or add new DAG files within your cloned repository:
 
-This way, ArgoCD is instructed to pull the Stackable manifests from the forked repository, where the changes are synced via ArgoCD into the cluster.
+* Manifests are in: `demos/argo-cd-git-ops/manifests/`
+* Airflow DAGs are in: `demos/argo-cd-git-ops/dags/`
 
-=== Increase Airflow webserver replicas
+==== Increase Airflow webserver replicas
 
 Assuming your working directory is the root of the forked demo repository, try to increase the `spec.webservers.roleGroups.<role-group>.replicas` in the folder `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml`.
 Once this is pushed / merged, ArgoCD should sync the changes and you should see more webserver pods.
 
-=== Add new Airflow DAGs
+==== Add new Airflow DAGs
 
 In the `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml` manifest you have to adapt the git-sync configuration for DAGs to the forked repository:
 
 [source,yaml]
 ----
     dagsGitSync:
-      - repo: <my-demo-fork-url>
+      - repo: https://github.com/<your-username>/demos/
         branch: <my-custom-branch-with-changes>
         [...]
 ----
 
-Similar to ArgoCD, after adding a new DAG to the folder `demos/argo-cd-git-ops/dags`, Airflow should pick up the new DAG via git-sync and display it in the UI.
+After adding a new DAG to the folder `demos/argo-cd-git-ops/dags/`, Airflow should pick up the new DAG via git-sync and display it in the UI.
+This may take a while for the syncing process. Refreshing the Airflow UI might help if no DAGs show up.
+
+The synchronisation status of the DAGs can be monitored in via the Airflow scheduler:
+
+[source,console]
+----
+kubectl logs -n stackable-airflow -c airflow -f svc/airflow-scheduler-default-headless
+----
+
+which should show output the DAG processing stats:
+
+[source,console]
+----
+================================================================================
+DAG File Processing Stats
+
+File Path                                                               PID  Runtime      # DAGs    # Errors  Last Runtime    Last Run      Last # of DB Queries
+--------------------------------------------------------------------  -----  ---------  --------  ----------  --------------  ----------  ----------------------
+/stackable/app/git-0/current/demos/argo-cd-git-ops/dags/date_demo.py     51  0.03s             0           0                                                   0
+================================================================================
+[2025-08-06T15:32:23.182+0000] {kubernetes_executor_utils.py:95} INFO - Kubernetes watch timed out waiting for events. Restarting watch.
+[2025-08-06T15:32:23.345+0000] {manager.py:997} INFO - 
+================================================================================
+----
+
+==== Commit and push changes
+
+[source,console]
+----
+git checkout -b <my-custom-branch-with-changes>
+git add .
+git commit -m "Update Airflow configuration and add new DAG"
+git push origin <my-custom-branch-with-changes>
+----
+
+Now ArgoCD and Airflow should sync the respective changes into the cluster.