Apply suggestions from code review

Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>

Author: sweb

CHANGELOG.md

@@ -24,7 +24,7 @@ All notable changes to this project will be documented in this file.
 - druid: Add `37.0.0` ([#1535]).
 - hbase: Add `2.6.6` ([#1547]).
 - spark: Add `4.1.2` ([#1550]).
-- spark: Bake the OpenLineage Spark listener (`1.51.0`) into the Spark images under the dedicated directory `/stackable/spark/openlineage/` (off the `extra-jars` classpath), enabling the OpenLineage MVP in the operator. The Scala 2.13 build (`openlineage-spark_2.13`) is used for the `4.1.x` images and the Scala 2.12 build (`openlineage-spark_2.12`) for the `3.5.x` images, both exposed through a stable `openlineage-spark.jar` symlink so the operator need not track the version or Scala suffix ([#XXXX]).
+- spark: Add the OpenLineage listener under the dedicated directory `/stackable/spark/openlineage/` ([#1577]).
 - opensearch: Add `3.6.0` ([#1549]).
 - opensearch-dashboards: Add `3.6.0` ([#1551]).
 
@@ -96,7 +96,7 @@ All notable changes to this project will be documented in this file.
 [#1548]: https://github.com/stackabletech/docker-images/pull/1548
 [#1549]: https://github.com/stackabletech/docker-images/pull/1549
 [#1550]: https://github.com/stackabletech/docker-images/pull/1550
-[#XXXX]: https://github.com/stackabletech/docker-images/pull/XXXX
+[#1577]: https://github.com/stackabletech/docker-images/pull/1577
 [#1551]: https://github.com/stackabletech/docker-images/pull/1551
 [#1559]: https://github.com/stackabletech/docker-images/pull/1559
 [#1561]: https://github.com/stackabletech/docker-images/pull/1561

spark-k8s/Dockerfile.3

@@ -154,14 +154,12 @@ cp /root/.m2/repository/com/fasterxml/woodstox/woodstox-core/${WOODSTOX_CORE_VER
 # so it loads on Spark's child/user classloader, sharing it with connectors delivered via `--packages`
 # (e.g. Iceberg). If it landed on extra-jars it would be added to the system classloader via
 # extraClassPath, OpenLineage's connector probe would fail, and no datasets would be emitted.
-# Spark 3.5.x is a Scala 2.12 build, so the 2.12 artifact is used (Spark 4.x uses 2.13).
 mkdir -p /stackable/spark-${PRODUCT_VERSION}-stackable${RELEASE_VERSION}/dist/openlineage
 mvn dependency:get -Dartifact=io.openlineage:openlineage-spark_2.12:${OPENLINEAGE_SPARK_VERSION}
 cp /root/.m2/repository/io/openlineage/openlineage-spark_2.12/${OPENLINEAGE_SPARK_VERSION}/openlineage-spark_2.12-${OPENLINEAGE_SPARK_VERSION}.jar \
   /stackable/spark-${PRODUCT_VERSION}-stackable${RELEASE_VERSION}/dist/openlineage/openlineage-spark_2.12-${OPENLINEAGE_SPARK_VERSION}.jar
 # Symlink to a stable, version- and Scala-independent name (mirroring the jmx_prometheus_javaagent.jar
 # pattern) so the operator can reference the jar without knowing the version or Scala build baked here.
-# The relative target keeps the link valid after `dist` is copied into the final image.
 ln -s openlineage-spark_2.12-${OPENLINEAGE_SPARK_VERSION}.jar \
   /stackable/spark-${PRODUCT_VERSION}-stackable${RELEASE_VERSION}/dist/openlineage/openlineage-spark.jar
 

spark-k8s/Dockerfile.4

@@ -132,9 +132,8 @@ mkdir -p /stackable/spark-${PRODUCT_VERSION}-stackable${RELEASE_VERSION}/dist/op
 mvn dependency:get -Dartifact=io.openlineage:openlineage-spark_2.13:${OPENLINEAGE_SPARK_VERSION}
 cp /root/.m2/repository/io/openlineage/openlineage-spark_2.13/${OPENLINEAGE_SPARK_VERSION}/openlineage-spark_2.13-${OPENLINEAGE_SPARK_VERSION}.jar \
   /stackable/spark-${PRODUCT_VERSION}-stackable${RELEASE_VERSION}/dist/openlineage/openlineage-spark_2.13-${OPENLINEAGE_SPARK_VERSION}.jar
-# Symlink to a stable, version- and Scala-independent name (mirroring the jmx_prometheus_javaagent.jar
-# pattern) so the operator can reference the jar without knowing the version or Scala build baked here.
-# The relative target keeps the link valid after `dist` is copied into the final image.
+# Symlink to a stable, version- and Scala-independent name
+# so the operator can reference the jar without knowing the version or Scala build baked here.
 ln -s openlineage-spark_2.13-${OPENLINEAGE_SPARK_VERSION}.jar \
   /stackable/spark-${PRODUCT_VERSION}-stackable${RELEASE_VERSION}/dist/openlineage/openlineage-spark.jar