Merge branch 'main' into docs/demu-upgrade-issue-template

Author: sbernauer

demos/airflow-scheduled-job/create-trino-tables.yaml

@@ -9,7 +9,7 @@ spec:
       serviceAccountName: demo-serviceaccount
       containers:
         - name: create-tables-in-trino
-          image: oci.stackable.tech/sdp/testing-tools:0.3.0-stackable0.0.0-dev
+          image: oci.stackable.tech/sdp/testing-tools/trino:0.3.0-stackable0.0.0-dev
           command: ["bash", "-c", "python -u /tmp/script/script.py"]
           volumeMounts:
             - name: script

docs/modules/demos/pages/airflow-scheduled-job.adoc

@@ -3,12 +3,11 @@
 
 Install this demo on an existing Kubernetes cluster:
 
-[source,console]
-----
-$ stackablectl demo install airflow-scheduled-job
-----
-
-You can also deploy it to a specific namespace (the namespace `airflow-demo` will be assumed in this guide):
+[NOTE]
+====
+The namespace `airflow-demo` will be assumed in this guide.
+It will be created if it doesn't exist.
+====
 
 [source,console]
 ----
@@ -225,7 +224,7 @@ image::airflow-scheduled-job/airflow_17.png[]
 
 Click on the DAG, switching to the task view.
 This DAG uses a deferrable operator which, in conjunction with the triggerer process, "offloads" the DAG from its worker for a specfic period of time, before being picked up and again and executed.
-You will see the task cycle through the following states:
+From the Task Instances view (under Dags), you will see the task cycle through the following states:
 
 image::airflow-scheduled-job/deferrable_01_running.png[]
 image::airflow-scheduled-job/deferrable_02_queued.png[]
@@ -284,7 +283,7 @@ The patch also sets some environment variables that can be used to change the fr
 [source,yaml]
 ----
 ---
-apiVersion: airflow.stackable.tech/v1alpha1
+apiVersion: airflow.stackable.tech/v1alpha2
 kind: AirflowCluster
 metadata:
   name: airflow
@@ -294,8 +293,6 @@ spec:
       - name: airflow-dags
         mountPath: /dags/dag_factory.py
         subPath: dag_factory.py
-      - name: kafka-tls-pem
-        mountPath: /stackable/kafka-tls-pem
   webservers:
     roleGroups:
       default:
@@ -315,13 +312,15 @@ spec:
         envOverrides: *envOverrides
 ----
 
-THe patch can be applied like this:
+The patch can be applied like this:
 
 [source,console]
 ----
 kubectl patch airflowcluster airflow --type="merge" --patch-file stacks/airflow/patch_airflow.yaml -n airflow-demo
 ----
 
+Wait for Airflow to come back up, and you should now see the generated DAGs.
+
 [NOTE]
 ====
 The scheduled job runs every minute and so an instance of it may be running while the scheduler is being re-started as a result of the patch, in which case that instance may fail.

stacks/airflow/airflow.yaml

@@ -1,6 +1,6 @@
 ---
 # {% raw %}
-apiVersion: airflow.stackable.tech/v1alpha1
+apiVersion: airflow.stackable.tech/v1alpha2
 kind: AirflowCluster
 metadata:
   name: airflow
@@ -74,7 +74,7 @@ spec:
       spec:
         containers:
           - name: airflow
-            image: oci.stackable.tech/sdp/airflow:3.0.6-stackable0.0.0-dev
+            image: oci.stackable.tech/sdp/airflow:3.1.6-stackable0.0.0-dev
             imagePullPolicy: IfNotPresent
             env:
               - name: NAMESPACE
@@ -102,7 +102,7 @@ spec:
       spec:
         containers:
           - name: base
-            image: oci.stackable.tech/sdp/airflow:3.0.6-stackable0.0.0-dev
+            image: oci.stackable.tech/sdp/airflow:3.1.6-stackable0.0.0-dev
             imagePullPolicy: IfNotPresent
             env:
               - name: AWS_SECRET_ACCESS_KEY

stacks/airflow/patch_airflow.yaml

@@ -1,5 +1,5 @@
 ---
-apiVersion: airflow.stackable.tech/v1alpha1
+apiVersion: airflow.stackable.tech/v1alpha2
 kind: AirflowCluster
 metadata:
   name: airflow
@@ -9,8 +9,6 @@ spec:
       - name: airflow-dags
         mountPath: /dags/dag_factory.py
         subPath: dag_factory.py
-      - name: kafka-tls-pem
-        mountPath: /stackable/kafka-tls-pem
   webservers:
     roleGroups:
       default: