[IcebergIO] Raise Java 17 floor for IcebergIO's Java 11 dependents (#39064)

* [IcebergIO] Bump Java floor to 17 (prep for Iceberg 1.11.0)

Iceberg 1.11.0 is published as Java 17 bytecode (class file version 61);
Gradle module metadata declares "org.gradle.jvm.version": 17. The current
Java 11 floor on sdks/java/io/iceberg causes Iceberg 1.11.0 artifacts
to fail resolution. Raise the floor independently so the version bump
is a clean diff.

Iceberg 1.10.0 still resolves under Java 17 (10.0 only requires Java 11+).

Tracks #38925.

* [IcebergIO] Run IcebergIO CI workflows on Java 17

Iceberg 1.11.0 dropped Java 11 support and this branch raised the iceberg
module floor to Java 17 (requireJavaVersion VERSION_17). The IO_Iceberg_*
workflows call setup-environment-action without a java-version, which
defaults to Java 11, so the module would fail to build or be disabled on
CI. Pin these workflows to Java 17, matching the Delta and Debezium IO
workflows.

For #38925

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [IcebergIO] Raise Java 17 floor for IcebergIO's Java 11 dependents

Bumping the iceberg module to Java 17 (Iceberg 1.11.0 dropped Java 11)
breaks the Java 11 modules that depend on it, because Gradle's
JVM-version variant resolution refuses to let a Java 11 consumer depend
on a Java 17 library. Raise requireJavaVersion 11 -> 17 in:

  - sdks/java/extensions/sql/iceberg
  - examples/java/iceberg

The Java IO expansion service (sdks/java/io/expansion-service) also
depends on IcebergIO but was already raised to Java 17 on master in

For #38925

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [IcebergIO] Fix Checker skipUses regex dropped on forked Java 17 compile

Raising the Java 17 floor routes IcebergIO through a forked compile: on an older host JDK (e.g. Java 11 CI) the module is compiled by a separate Java 17 javac launched via java17Home, and Gradle passes its arguments through an @argfile. The backslash-escaped '\.' in Beam's Checker Framework -AskipUses and -AskipDefs regexes does not survive that @argfile round-trip, so '^org\.slf4j\.Logger.*' becomes a literal-backslash regex that matches nothing and the org.slf4j.Logger nullness suppression is silently dropped. Two latent Logger.info(@nullable) calls in IcebergIO then hard-error under the NullnessChecker.

Use the backslash-free character class '[.]' (semantically identical to '\.') so the regexes survive the @argfile round-trip. This fixes the suppression for every module that forks to a newer JDK, with no behavior change on the in-process compile path.

* [IcebergIO] Run xlang wrapper validation expansion service on Java 17

The cross-language wrapper validation launches the io expansion-service jar from Python via JAVA_HOME. That jar bundles IcebergIO, now Java 17 bytecode, so launching it on the default Java 11 fails with UnsupportedClassVersionError.

createCrossLanguageUsingJavaExpansionTask was missing the JAVA_HOME redirect that #38974 added to the other cross-language task factories. Add it, driven by -PtestJavaVersion (resolving java\Home), and pass -PtestJavaVersion=17 in the Xlang_Generated_Transforms precommit so the expansion service runs on a JDK that can load the bundled Java 17 IOs.

* Trigger Python Xlang IO postcommits

* [IcebergIO] Run Python Xlang IO postcommits on Java 17

* [IcebergIO] Pin Java 17 on IO expansion service test suites

* [IcebergIO] Address Java 17 review comments

* [IcebergIO] Raise Iceberg integration tests to Java 17

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: peterphitran

.github/trigger_files/beam_PostCommit_Python_Xlang_IO_Dataflow.json

@@ -1,4 +1,4 @@
 {
   "comment": "Modify this file in a trivial way to cause this test suite to run",
-  "modification": 1
+  "modification": 2
 }

.github/trigger_files/beam_PostCommit_Python_Xlang_IO_Direct.json

@@ -1,4 +1,4 @@
 {
   "comment": "Modify this file in a trivial way to cause this test suite to run",
-  "modification": 6
+  "modification": 7
 }

.github/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml

@@ -75,7 +75,7 @@ jobs:
       - name: Setup environment
         uses: ./.github/actions/setup-environment-action
         with:
-          java-version: default
+          java-version: '17'
           python-version: default
       - name: Set k8s access
         uses: ./.github/actions/setup-k8s-access

.github/workflows/beam_PostCommit_Python_Xlang_IO_Dataflow.yml

@@ -75,7 +75,7 @@ jobs:
       - name: Setup environment
         uses: ./.github/actions/setup-environment-action
         with:
-          java-version: default
+          java-version: '17'
           python-version: |
             3.10
             3.14

.github/workflows/beam_PostCommit_Python_Xlang_IO_Direct.yml

@@ -75,7 +75,7 @@ jobs:
       - name: Setup environment
         uses: ./.github/actions/setup-environment-action
         with:
-          java-version: default
+          java-version: '17'
           python-version: |
             3.10
             3.14

CHANGES.md

@@ -79,6 +79,10 @@
 * (Python) Removed `google-perftools` from the SDK container images. Users who wish to use `--profiler_agent=tcmalloc` should install google-perftools APT package in their custom container images separately ([#39323](https://github.com/apache/beam/issues/39323)).
 * `DoFn.process` returning a `str`, `bytes`, or `dict` (instead of an iterable wrapping one) now raises a `TypeError` rather than silently iterating per-character/byte/key (Python) ([#18712](https://github.com/apache/beam/issues/18712)).
 * (Java) Added `DRAINING` and `DRAINED` states to `PipelineResult`, including runner state mappings and Dataflow update handling ([#39020](https://github.com/apache/beam/issues/39020)).
+* (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any,
+  use pipeline option `--exclude_infer_dataclass_field_type` ([#38797](https://github.com/apache/beam/issues/38797)).
+  However fixing forward is recommended.
+* (Java) IcebergIO now requires Java 17 at runtime. This raises the floor in preparation for the Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)).
 
 ## Deprecations
 

buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

@@ -1257,20 +1257,26 @@ class BeamModulePlugin implements Plugin<Project> {
         maxHeapSize = '2g'
       }
 
+      // NOTE: Use the character class "[.]" instead of an escaped "\\." to match a literal dot in
+      // these Checker Framework -AskipDefs/-AskipUses regexes. When a module is compiled on an older
+      // host JDK and forked to a newer JDK via javaXXHome (e.g. iceberg's requireJavaVersion 17 on a
+      // Java 11 CI host), Gradle passes the javac arguments through an @argfile. Backslash escapes do
+      // not survive that round-trip intact, so "\\." becomes a literal-backslash regex that matches
+      // nothing and the suppression is silently dropped. "[.]" is backslash-free and survives.
       List<String> skipDefRegexes = []
       skipDefRegexes << "AutoValue_.*"
       skipDefRegexes << "AutoBuilder_.*"
       skipDefRegexes << "AutoOneOf_.*"
-      skipDefRegexes << ".*\\.jmh_generated\\..*"
+      skipDefRegexes << ".*[.]jmh_generated[.].*"
       skipDefRegexes += configuration.generatedClassPatterns
       skipDefRegexes += configuration.classesTriggerCheckerBugs.keySet()
       String skipDefCombinedRegex = skipDefRegexes.collect({ regex -> "(${regex})"}).join("|")
 
       List<String> skipUsesRegexes = []
       // zstd-jni is not annotated, handles Zstd(De)CompressCtx.loadDict(null) just fine
-      skipUsesRegexes << "^com\\.github\\.luben\\.zstd\\..*"
+      skipUsesRegexes << "^com[.]github[.]luben[.]zstd[.].*"
       // SLF4J logger handles null log message parameters
-      skipUsesRegexes << "^org\\.slf4j\\.Logger.*"
+      skipUsesRegexes << "^org[.]slf4j[.]Logger.*"
       String skipUsesCombinedRegex = skipUsesRegexes.collect({ regex -> "(${regex})"}).join("|")
 
       project.apply plugin: 'org.checkerframework'

examples/java/iceberg/build.gradle

@@ -26,8 +26,8 @@ plugins {
 applyJavaNature(
   exportJavadoc: false,
   automaticModuleName: 'org.apache.beam.examples.iceberg',
-  // iceberg requires Java11+
-  requireJavaVersion: JavaVersion.VERSION_11
+  // iceberg requires Java17+
+  requireJavaVersion: JavaVersion.VERSION_17
 )
 
 description = "Apache Beam :: Examples :: Java :: Iceberg"

it/iceberg/build.gradle

@@ -22,8 +22,8 @@ plugins { id 'org.apache.beam.module' }
 applyJavaNature(
   automaticModuleName: 'org.apache.beam.it.iceberg',
   exportJavadoc: false,
-  // iceberg ended support for Java 8 in 1.7.0
-  requireJavaVersion: JavaVersion.VERSION_11,
+  // iceberg ended support for Java 11 in 1.11.0
+  requireJavaVersion: JavaVersion.VERSION_17,
 )
 
 description = "Apache Beam :: IT :: Iceberg"

sdks/java/extensions/sql/iceberg/build.gradle

@@ -22,8 +22,8 @@ plugins { id 'org.apache.beam.module' }
 
 applyJavaNature(
   automaticModuleName: 'org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog',
-  // iceberg requires Java11+
-  requireJavaVersion: JavaVersion.VERSION_11,
+  // iceberg requires Java17+
+  requireJavaVersion: JavaVersion.VERSION_17,
 )
 
 dependencies {