Skip to content

Commit 74a54c7

Browse files
authored
Merge branch 'main' into fix/demos-in-namespaces-with-labels
2 parents decf874 + c6c4d89 commit 74a54c7

25 files changed

Lines changed: 43 additions & 103 deletions

demos/data-lakehouse-iceberg-trino-spark/create-spark-ingestion-job.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,12 +145,13 @@ data:
145145
stackable.tech/demo: {{ DEMO }}
146146
spec:
147147
sparkImage:
148+
# Iceberg 1.10.1 only supports Spark 4.0.x
148149
productVersion: 4.0.1
149150
mode: cluster
150151
mainApplicationFile: local:///stackable/spark/jobs/spark-ingest-into-lakehouse.py
151152
deps:
152153
packages:
153-
- org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.0
154+
- org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.1
154155
- org.apache.spark:spark-sql-kafka-0-10_2.13:4.0.1
155156
s3connection:
156157
reference: minio

demos/end-to-end-security/create-spark-report.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,12 +58,13 @@ data:
5858
stackable.tech/demo: {{ DEMO }}
5959
spec:
6060
sparkImage:
61+
# Iceberg 1.10.1 only supports Spark 4.0.x
6162
productVersion: 4.0.1
6263
mode: cluster
6364
mainApplicationFile: local:///stackable/spark/jobs/spark-report.py
6465
deps:
6566
packages:
66-
- org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.0
67+
- org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.1
6768
sparkConf:
6869
spark.driver.extraClassPath: /stackable/config/hdfs
6970
spark.executor.extraClassPath: /stackable/config/hdfs

demos/spark-k8s-anomaly-detection-taxi-data/create-spark-anomaly-detection-job.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,12 +51,13 @@ data:
5151
name: spark-ad
5252
spec:
5353
sparkImage:
54+
# Iceberg 1.10.1 only supports Spark 4.0.x
5455
productVersion: 4.0.1
5556
mode: cluster
5657
mainApplicationFile: local:///spark-scripts/spark-ad.py
5758
deps:
5859
packages:
59-
- org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.0
60+
- org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.1
6061
requirements:
6162
- scikit-learn==1.4.0
6263
s3connection:

docs/modules/demos/pages/airflow-scheduled-job.adoc

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,11 @@
33

44
Install this demo on an existing Kubernetes cluster:
55

6-
[source,console]
7-
----
8-
$ stackablectl demo install airflow-scheduled-job
9-
----
10-
11-
You can also deploy it to a specific namespace (the namespace `airflow-demo` will be assumed in this guide):
6+
[NOTE]
7+
====
8+
The namespace `airflow-demo` will be assumed in this guide.
9+
It will be created if it doesn't exist.
10+
====
1211

1312
[source,console]
1413
----
@@ -225,7 +224,7 @@ image::airflow-scheduled-job/airflow_17.png[]
225224

226225
Click on the DAG, switching to the task view.
227226
This DAG uses a deferrable operator which, in conjunction with the triggerer process, "offloads" the DAG from its worker for a specfic period of time, before being picked up and again and executed.
228-
You will see the task cycle through the following states:
227+
From the Task Instances view (under Dags), you will see the task cycle through the following states:
229228

230229
image::airflow-scheduled-job/deferrable_01_running.png[]
231230
image::airflow-scheduled-job/deferrable_02_queued.png[]
@@ -315,13 +314,15 @@ spec:
315314
envOverrides: *envOverrides
316315
----
317316

318-
THe patch can be applied like this:
317+
The patch can be applied like this:
319318

320319
[source,console]
321320
----
322321
kubectl patch airflowcluster airflow --type="merge" --patch-file stacks/airflow/patch_airflow.yaml -n airflow-demo
323322
----
324323

324+
Wait for Airflow to come back up, and you should now see the generated DAGs.
325+
325326
[NOTE]
326327
====
327328
The scheduled job runs every minute and so an instance of it may be running while the scheduler is being re-started as a result of the patch, in which case that instance may fail.

stacks/_templates/argo-cd.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ name: argo-cd
44
repo:
55
name: argo-cd
66
url: https://argoproj.github.io/argo-helm
7-
version: 9.0.5 # v3.1.9
7+
version: 9.4.10 # v3.3.3
88
options:
99
global:
1010
additionalLabels:

stacks/_templates/keycloak.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ spec:
1717
spec:
1818
containers:
1919
- name: keycloak
20-
image: quay.io/keycloak/keycloak:26.1.4
20+
image: quay.io/keycloak/keycloak:26.5.5
2121
args:
2222
- start
2323
- --hostname-strict=false

stacks/_templates/minio-distributed-small.yaml

Lines changed: 0 additions & 31 deletions
This file was deleted.

stacks/_templates/minio-distributed.yaml

Lines changed: 0 additions & 33 deletions
This file was deleted.

stacks/_templates/postgresql-airflow.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ name: postgresql
44
repo:
55
name: bitnami
66
url: https://charts.bitnami.com/bitnami/
7-
version: 18.1.3 # 18.0.0
7+
version: 18.5.6 # 18.3.0
88
options:
99
commonLabels:
1010
stackable.tech/vendor: Stackable

stacks/_templates/postgresql-druid.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ name: postgresql
44
repo:
55
name: bitnami
66
url: https://charts.bitnami.com/bitnami/
7-
version: 18.1.3 # 18.0.0
7+
version: 18.5.6 # 18.3.0
88
options:
99
commonLabels:
1010
stackable.tech/vendor: Stackable

0 commit comments

Comments
 (0)