feat: Replace ubi9 with ubi10 (#1353)

* feat(ubi-rust-builder): replace ubi9 with ubi10

* feat: Use ubi10 in base images

Note: This required an adjustment to the opa daemonset patch, otherwise compilation fails due to gcc-15 requirements.
Along the road to fixing this, we found some relevant links which might be useful in future:
- [Github Repo](https://github.com/daemontools/daemontools) (shows daemontools 0.76 is 12 years old)
- [daemontools Homepage](https://cr.yp.to/daemontools.html)
- [Debian bug report](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1066623)
  - [Debian patches)(https://salsa.debian.org/debian/daemontools/-/tree/debian/latest/debian/patches?ref_type=heads)
  - [Reverted patch and GCC-15 fix](https://salsa.debian.org/debian/daemontools/-/commit/3e20831dfabe9dac4c5b6de3ae3945491280c80b#8756c63497c8dc39f7773438edf53b220c773f67)

* feat(airflow/wip): use a newer uv version, and install the correct python version using it

WIP: Still need to fix the build layers, but this appears to work fine (tested the oldest and newest versions via the getting_started guide)

* chore: fix typo in upload script

* chore(wip): Upload script for nvm

WIP: We might not need this, but I made it and it works.

* chore: Move lint config into the right place

* chore(airflow): replace python/pip package installations with uv for consistency

* chore(airflow): Replace source with .

* chore(airflow): Remove unneeded comments

* uvx migrate-to-uv

* chore(superset/opa-authorizer): Add description and format the file

* chore(superset): Upgrade uv version

* chore(superset): Use uv instead of pip

* fix(superset): Add patch to pin the sax package to 1.2.4

* chore(mirror): Allow uv image to be mirrored

* chore: Use mirrored uv image

* chore(airflow): Add hint about updating git-sync

* chore(airflow):Remove unused args

* chore(superset):Remove unused args

* chore: Update changelog

* chore: Remove unused upload script

Note: this came from 01d1d59

* Apply suggestions from code review

---------

Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
Co-authored-by: Techassi <git@techassi.dev>

Author: 3 people

.github/ISSUE_TEMPLATE/update-base-ubi-rust-builders.md

@@ -51,15 +51,15 @@ Add/Change/Remove anything that isn't applicable anymore
 > This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been
 > checked, the issue can be moved into _Development: Done_.
 
-- [ ] Done for [ubi9-rust-builder/Dockerfile](https://github.com/stackabletech/docker-images/blob/main/ubi9-rust-builder/Dockerfile)
+- [ ] Done for [ubi10-rust-builder/Dockerfile](https://github.com/stackabletech/docker-images/blob/main/ubi10-rust-builder/Dockerfile)
 - [ ] Can build the image locally
 - [ ] Can build an operator image
 
 <details>
 <summary>Testing instructions</summary>
 
 ```shell
-docker build -t oci.stackable.tech/sdp/ubi9-rust-builder . -f ubi9-rust-builder/Dockerfile
+docker build -t oci.stackable.tech/sdp/ubi10-rust-builder . -f ubi10-rust-builder/Dockerfile
 
 # Change directory into the an operator repository and ensure the image can build
 docker build . -f docker/Dockerfile

.github/workflows/mirror.yaml

@@ -15,6 +15,7 @@ on:
           - registry.k8s.io/sig-storage/csi-provisioner
           - registry.k8s.io/git-sync/git-sync
           - registry-1.docker.io/library/golang
+          - ghcr.io/astral-sh/uv
       image-index-manifest-tag:
         description: |
           The image index manifest tag, like 1.0.14 or v1.0.14

.github/workflows/ubi-rust-builder.yml

@@ -8,7 +8,6 @@ on:
     branches:
       - main
     paths:
-      - ubi9-rust-builder/**
       - ubi10-rust-builder/**
       - .github/actions/**
       - .github/workflows/ubi-rust-builder.yml
@@ -25,7 +24,7 @@ jobs:
       fail-fast: false
       matrix:
         runner: ["ubuntu-latest", "ubicloud-standard-8-arm-ubuntu-2404"]
-        ubi-version: ["ubi9"]
+        ubi-version: ["ubi10"]
     runs-on: ${{ matrix.runner }}
     steps:
       - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
@@ -65,7 +64,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        ubi-version: ["ubi9"]
+        ubi-version: ["ubi10"]
     runs-on: ubuntu-latest
     needs: ["build"]
     steps:

.scripts/upload_new_maven_version.sh

@@ -71,4 +71,4 @@ curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --uploa
 
 echo "Successfully uploaded Maven $VERSION to Nexus"
 echo "https://repo.stackable.tech/service/rest/repository/browse/packages/maven/"
-echo "https://github.com/prometheus/maven/releases/tag/$VERSION"
+echo "https://github.com/apache/maven/releases/tag/maven-$VERSION"

CHANGELOG.md

@@ -17,6 +17,7 @@ All notable changes to this project will be documented in this file.
 - airflow: Bump celery version to 5.5.3 for Airflow 3.x ([#1343]).
 - testing-tools: refactoring: Split image into multiple images, remove unnecessary components and switch to UBI as base image ([#1354]).
 - hive: fixed 4.0.1 shaded hive-metastore-opa-authorizer jar by relocating dependencies ([#1356]).
+- Use ubi10 as the base image for all products ([#1353]).
 
 ### Removed
 
@@ -30,6 +31,7 @@ All notable changes to this project will be documented in this file.
 [#1356]: https://github.com/stackabletech/docker-images/pull/1356
 [#1357]: https://github.com/stackabletech/docker-images/pull/1357
 [#1358]: https://github.com/stackabletech/docker-images/pull/1358
+[#1353]: https://github.com/stackabletech/docker-images/pull/1353
 
 ## [25.11.0] - 2025-11-07
 

airflow/Dockerfile

@@ -1,33 +1,32 @@
 # syntax=docker/dockerfile:1.16.0@sha256:e2dd261f92e4b763d789984f6eab84be66ab4f5f08052316d8eb8f173593acf7
+# check=error=true;skip=InvalidDefaultArgInFrom,SecretsUsedInArgOrEnv
 # Disabled error checks:
 # - SecretsUsedInArgOrEnv : OPA_AUTH_MANAGER is a false positive and breaks the build.
-# check=error=true;skip=InvalidDefaultArgInFrom,SecretsUsedInArgOrEnv
 
 ARG GIT_SYNC_VERSION
+ARG UV_VERSION
 
 # For updated versions check https://github.com/kubernetes/git-sync/releases
 # which should contain a image location (e.g. registry.k8s.io/git-sync/git-sync:v3.6.8)
 FROM oci.stackable.tech/sdp/git-sync/git-sync:${GIT_SYNC_VERSION} AS gitsync-image
 
+FROM oci.stackable.tech/sdp/astral-sh/uv:${UV_VERSION} AS uv-image
+
 FROM local-image/shared/statsd-exporter AS statsd_exporter-builder
 
 FROM local-image/vector AS opa-auth-manager-builder
 
 ARG OPA_AUTH_MANAGER
 ARG PYTHON_VERSION
-ARG UV_VERSION
+ARG STACKABLE_USER_UID
 
 COPY airflow/opa-auth-manager/${OPA_AUTH_MANAGER} /tmp/opa-auth-manager
 
 WORKDIR /tmp/opa-auth-manager
 
-RUN <<EOF
-microdnf update
-microdnf install python${PYTHON_VERSION}-pip
-microdnf clean all
-
-pip${PYTHON_VERSION} install --no-cache-dir uv==${UV_VERSION}
+COPY --from=uv-image --chown=${STACKABLE_USER_UID}:0 /uv /uvx /bin/
 
+RUN <<EOF
 # This folder is required by the tests to set up an sqlite database
 mkdir /root/airflow
 
@@ -43,10 +42,8 @@ ARG PRODUCT_VERSION
 ARG PYTHON_VERSION
 ARG TARGETARCH
 ARG STACKABLE_USER_UID
-ARG NODEJS_VERSION
 ARG S3FS_VERSION
 ARG CYCLONEDX_BOM_VERSION
-ARG UV_VERSION
 
 # Airflow "extras" packages are listed here: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html
 # They evolve over time and thus belong to the version-specific arguments.
@@ -62,8 +59,9 @@ ARG AIRFLOW_EXTRAS_EXTERNAL_SERVICES=""
 ARG AIRFLOW_EXTRAS_LOCALLY_INSTALLED_SOFTWARE=""
 ARG AIRFLOW_EXTRAS_OTHER=""
 
-RUN microdnf module enable -y nodejs:${NODEJS_VERSION} && \
-    microdnf update && \
+COPY --from=uv-image --chown=${STACKABLE_USER_UID}:0 /uv /uvx /bin/
+
+RUN microdnf update && \
     microdnf install \
         cyrus-sasl-devel \
         # Needed for kerberos
@@ -78,10 +76,6 @@ RUN microdnf module enable -y nodejs:${NODEJS_VERSION} && \
         libpq-devel \
         openldap-devel \
         openssl-devel \
-        python${PYTHON_VERSION} \
-        python${PYTHON_VERSION}-devel \
-        python${PYTHON_VERSION}-pip \
-        python${PYTHON_VERSION}-wheel \
         # The airflow odbc provider can compile without the development files (headers and libraries) (see https://github.com/stackabletech/docker-images/pull/683)
         unixODBC \
         # Needed for Airflow UI assets
@@ -111,14 +105,10 @@ AIRFLOW_EXTRAS="$AIRFLOW_EXTRAS_CORE,$AIRFLOW_EXTRAS_META,$AIRFLOW_EXTRAS_PROVID
 # Removing duplicates
 AIRFLOW_EXTRAS=$(echo "$AIRFLOW_EXTRAS" | tr ',' '\n' | awk 'NF > 0 {if (!seen[$0]++) print $0}' | tr '\n' ',' | sed 's/,$//')
 
-python${PYTHON_VERSION} -m venv --system-site-packages /stackable/app
-
-source /stackable/app/bin/activate
+uv venv --python "${PYTHON_VERSION}" --system-site-packages /stackable/app
+. /stackable/app/bin/activate
 
-# Upgrade pip to the latest version
-# Also install uv to get support for build constraints
-pip install --no-cache-dir --upgrade pip
-pip install --no-cache-dir uv==${UV_VERSION}
+# Another project manager
 uv tool install hatch
 
 cd "$(/stackable/patchable --images-repo-root=src checkout airflow ${PRODUCT_VERSION})"
@@ -225,14 +215,15 @@ COPY --from=gitsync-image --chown=${STACKABLE_USER_UID}:0 /git-sync ${HOME}/git-
 
 COPY airflow/licenses /licenses
 
+COPY --from=uv-image --chown=${STACKABLE_USER_UID}:0 /uv /uvx /bin/
+
 # Update image and install needed packages
 RUN <<EOF
 microdnf update
 
 # git: Needed for the gitsync functionality
 # openldap: Needed for authentication of clients against LDAP servers
 # openssh-clients: We need the openssh libs for the gitsync functionality (the clone target could be e.g. git@github.com:org/repo.git)
-# python: Airflow needs Python
 microdnf install \
   ca-certificates \
   cyrus-sasl \
@@ -242,8 +233,6 @@ microdnf install \
   openldap-clients \
   openssh-clients \
   openssl-libs \
-  openssl-pkcs11 \
-  python${PYTHON_VERSION} \
   socat \
   unixODBC
 microdnf clean all
@@ -253,6 +242,13 @@ rm -rf /var/cache/yum
 # It is used as an init alternative in the entrypoint
 curl -o /usr/bin/tini "https://repo.stackable.tech/repository/packages/tini/tini-${TINI_VERSION}-${TARGETARCH}"
 
+# There is currently no way to tell `uv` where to write to, and it assumes `$HOME` for the XDG paths.
+export XDG_DATA_HOME=/usr/local/share
+export XDG_CACHE_HOME=/var/cache
+uv venv --python "${PYTHON_VERSION}" --system-site-packages /stackable/app
+chown -R "${STACKABLE_USER_UID}:0" /stackable/app/bin
+. /stackable/app/bin/activate
+
 # fix missing permissions
 chmod a+x /entrypoint.sh
 chmod a+x /run-airflow.sh

airflow/boil-config.toml

@@ -5,14 +5,13 @@ stackable-devel = "1.0.0"
 
 [versions."2.9.3".build-arguments]
 python-version = "3.9"
-git-sync-version = "v4.4.1"
+git-sync-version = "v4.4.1" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 s3fs-version = "2024.9.0"
 cyclonedx-bom-version = "6.0.0"
 tini-version = "0.19.0"
-uv-version = "0.7.8"
+uv-version = "0.9.10" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 airflow-extras-other = "async,amazon,celery,cncf.kubernetes,docker,dask,elasticsearch,ftp,grpc,hashicorp,http,ldap,google,google_auth,microsoft.azure,odbc,pandas,postgres,redis,sendgrid,sftp,slack,ssh,statsd,virtualenv,trino"
 opa-auth-manager = "airflow-2"
-nodejs-version = "20"
 
 [versions."2.10.5".local-images]
 "shared/statsd-exporter" = "0.28.0"
@@ -21,14 +20,13 @@ stackable-devel = "1.0.0"
 
 [versions."2.10.5".build-arguments]
 python-version = "3.12"
-git-sync-version = "v4.4.1"
+git-sync-version = "v4.4.1" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 s3fs-version = "2024.9.0"
 cyclonedx-bom-version = "6.0.0"
 tini-version = "0.19.0"
-uv-version = "0.7.8"
+uv-version = "0.9.10" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 airflow-extras-other = "async,amazon,celery,cncf.kubernetes,docker,dask,elasticsearch,ftp,grpc,hashicorp,http,ldap,google,google_auth,microsoft.azure,odbc,pandas,postgres,redis,sendgrid,sftp,slack,ssh,statsd,virtualenv,trino"
 opa-auth-manager = "airflow-2"
-nodejs-version = "20"
 
 [versions."3.0.1".local-images]
 "shared/statsd-exporter" = "0.28.0"
@@ -37,14 +35,13 @@ stackable-devel = "1.0.0"
 
 [versions."3.0.1".build-arguments]
 python-version = "3.12"
-git-sync-version = "v4.4.1"
+git-sync-version = "v4.4.1" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 s3fs-version = "2024.9.0"
 cyclonedx-bom-version = "6.0.0"
 tini-version = "0.19.0"
-uv-version = "0.7.8"
+uv-version = "0.9.10" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 airflow-extras-other = "async,amazon,celery,cncf-kubernetes,docker,elasticsearch,fab,ftp,grpc,hashicorp,http,ldap,google,microsoft-azure,odbc,pandas,postgres,redis,sendgrid,sftp,slack,ssh,statsd,trino"
 opa-auth-manager = "airflow-3"
-nodejs-version = "20"
 
 [versions."3.0.6".local-images]
 "shared/statsd-exporter" = "0.28.0"
@@ -53,30 +50,29 @@ stackable-devel = "1.0.0"
 
 [versions."3.0.6".build-arguments]
 python-version = "3.12"
-git-sync-version = "v4.4.1"
+git-sync-version = "v4.4.1" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 s3fs-version = "2024.9.0"
 cyclonedx-bom-version = "6.0.0"
 tini-version = "0.19.0"
-uv-version = "0.7.8"
+uv-version = "0.9.10" # When changing this version, you will need to use the mirror workflow to get the image into our registry
 
 # Airflow extras are defined in separate lists to make them easier to check against the links below. The lists will be concatenated and duplicates removed in the dockerfile.
 # See https://airflow.apache.org/docs/apache-airflow/3.0.6/extra-packages-ref.html#core-airflow-extras
-airflow-extras-core="async,graphviz,kerberos,otel,sentry,standard,statsd"
+airflow-extras-core = "async,graphviz,kerberos,otel,sentry,standard,statsd"
 
 # See https://airflow.apache.org/docs/apache-airflow/3.0.6/extra-packages-ref.html#meta-airflow-package-extras
-airflow-extras-meta="aiobotocore,cloudpickle,github-enterprise,google-auth,graphviz,ldap,leveldb,pandas,polars,rabbitmq,s3fs,saml,uv"
+airflow-extras-meta = "aiobotocore,cloudpickle,github-enterprise,google-auth,graphviz,ldap,leveldb,pandas,polars,rabbitmq,s3fs,saml,uv"
 
 # See https://airflow.apache.org/docs/apache-airflow/3.0.6/extra-packages-ref.html#apache-software-extras
-airflow-extras-provider-apache="apache-beam,apache-cassandra,apache-drill,apache-druid,apache-flink,apache-hdfs,apache-hive,apache-iceberg,apache-impala,apache-kafka,apache-kylin,apache-livy,apache-pig,apache-pinot"
+airflow-extras-provider-apache = "apache-beam,apache-cassandra,apache-drill,apache-druid,apache-flink,apache-hdfs,apache-hive,apache-iceberg,apache-impala,apache-kafka,apache-kylin,apache-livy,apache-pig,apache-pinot"
 
 # See https://airflow.apache.org/docs/apache-airflow/3.0.6/extra-packages-ref.html#external-services-extras
-airflow-extras-external-services="airbyte,alibaba,apprise,amazon,asana,atlassian-jira,microsoft-azure,cloudant,cohere,databricks,datadog,dbt-cloud,dingding,discord,facebook,github,google,hashicorp,openai,opsgenie,pagerduty,pgvector,pinecone,qdrant,salesforce,sendgrid,segment,slack,snowflake,tableau,tabular,telegram,vertica,weaviate,yandex,ydb,zendesk"
+airflow-extras-external-services = "airbyte,alibaba,apprise,amazon,asana,atlassian-jira,microsoft-azure,cloudant,cohere,databricks,datadog,dbt-cloud,dingding,discord,facebook,github,google,hashicorp,openai,opsgenie,pagerduty,pgvector,pinecone,qdrant,salesforce,sendgrid,segment,slack,snowflake,tableau,tabular,telegram,vertica,weaviate,yandex,ydb,zendesk"
 
 # See https://airflow.apache.org/docs/apache-airflow/3.0.6/extra-packages-ref.html#locally-installed-software-extras
-airflow-extras-locally-installed-software="arangodb,celery,cncf-kubernetes,docker,edge3,elasticsearch,exasol,fab,git,github,influxdb,jenkins,mongo,microsoft-mssql,neo4j,odbc,openfaas,oracle,postgres,presto,redis,samba,singularity,teradata,trino"
+airflow-extras-locally-installed-software = "arangodb,celery,cncf-kubernetes,docker,edge3,elasticsearch,exasol,fab,git,github,influxdb,jenkins,mongo,microsoft-mssql,neo4j,odbc,openfaas,oracle,postgres,presto,redis,samba,singularity,teradata,trino"
 
 # See https://airflow.apache.org/docs/apache-airflow/3.0.6/extra-packages-ref.html#other-extras
-airflow-extras-other="common-compat,common-io,common-messaging,common-sql,ftp,grpc,http,imap,jdbc,microsoft-psrp,microsoft-winrm,openlineage,opensearch,papermill,sftp,smtp,sqlite,ssh"
+airflow-extras-other = "common-compat,common-io,common-messaging,common-sql,ftp,grpc,http,imap,jdbc,microsoft-psrp,microsoft-winrm,openlineage,opensearch,papermill,sftp,smtp,sqlite,ssh"
 
 opa-auth-manager = "airflow-3"
-nodejs-version = "20"

druid/Dockerfile

@@ -131,7 +131,7 @@ LABEL org.opencontainers.image.description="${DESCRIPTION}"
 
 # https://docs.openshift.com/container-platform/4.16/openshift_images/create-images.html#defining-image-metadata
 # https://github.com/projectatomic/ContainerApplicationGenericLabels/blob/master/vendor/redhat/labels.md
-LABEL io.openshift.tags="ubi9,stackable,druid,sdp"
+LABEL io.openshift.tags="ubi10,stackable,druid,sdp"
 LABEL io.k8s.description="${DESCRIPTION}"
 LABEL io.k8s.display-name="${NAME}"
 

hbase/Dockerfile

@@ -92,7 +92,7 @@ LABEL org.opencontainers.image.description="${DESCRIPTION}"
 
 # https://docs.openshift.com/container-platform/4.16/openshift_images/create-images.html#defining-image-metadata
 # https://github.com/projectatomic/ContainerApplicationGenericLabels/blob/master/vendor/redhat/labels.md
-LABEL io.openshift.tags="ubi9,stackable,hbase,sdp,nosql"
+LABEL io.openshift.tags="ubi10,stackable,hbase,sdp,nosql"
 LABEL io.k8s.description="${DESCRIPTION}"
 LABEL io.k8s.display-name="${NAME}"
 

hive/Dockerfile

@@ -173,7 +173,7 @@ LABEL org.opencontainers.image.description="${DESCRIPTION}"
 
 # https://docs.openshift.com/container-platform/4.16/openshift_images/create-images.html#defining-image-metadata
 # https://github.com/projectatomic/ContainerApplicationGenericLabels/blob/master/vendor/redhat/labels.md
-LABEL io.openshift.tags="ubi9,stackable,hive,sdp"
+LABEL io.openshift.tags="ubi10,stackable,hive,sdp"
 LABEL io.k8s.description="${DESCRIPTION}"
 LABEL io.k8s.display-name="${NAME}"