From fe31164228c5ad2670a6328c11ad6f82183e5350 Mon Sep 17 00:00:00 2001 From: Peter-Phi-Tran Date: Sat, 20 Jun 2026 03:22:35 -0500 Subject: [PATCH 1/3] [IcebergIO] Upgrade Iceberg dependency to 1.11.0 Bumps iceberg-core/api/parquet/orc/data and the GCP/AWS/Azure runtime modules from 1.10.0 to 1.11.0. Iceberg 1.11.0 pulls parquet-avro/parquet-hadoop 1.17.1 transitively; pin them at 1.16.0 (current Beam parquet_version) so this PR stays zero-behavior-change. A separate PR can bump parquet_version once 1.17.x is vetted across the rest of Beam. The old per-format static builders (Parquet.read(), Avro.writeData(), ORC.write(), etc.) are only deprecated in 1.11.0, not removed, so RecordWriter / ScanTaskReader / ReadUtils still compile unchanged. A follow-up PR will migrate them to the new FormatModelRegistry SPI. Rebased on merged #39064 (Java 17 floor). Tracks #38925. --- CHANGES.md | 1 + sdks/java/io/iceberg/build.gradle | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index c5da08bd1a5f..1d7a775eab6d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -64,6 +64,7 @@ ## I/Os +* Upgraded Iceberg dependency to 1.11.0 (Java) ([#38925](https://github.com/apache/beam/issues/38925)). * Support for X source added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)). * Add ArrowFlight IO (Java) ([#20116](https://github.com/apache/beam/issues/20116)). diff --git a/sdks/java/io/iceberg/build.gradle b/sdks/java/io/iceberg/build.gradle index 7cdd32ed90e4..983ebd07fefc 100644 --- a/sdks/java/io/iceberg/build.gradle +++ b/sdks/java/io/iceberg/build.gradle @@ -37,7 +37,7 @@ def hadoopVersions = [ hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")} -def iceberg_version = "1.10.0" +def iceberg_version = "1.11.0" def parquet_version = "1.16.0" def orc_version = "1.9.6" def hive_version = "3.1.3" @@ -118,6 +118,10 @@ dependencies { configurations.all { // iceberg-core needs avro:1.12.0 resolutionStrategy.force 'org.apache.avro:avro:1.12.0' + // Iceberg 1.11.0 pulls parquet 1.17.1 transitively; hold at 1.16.0 to keep + // this PR zero-behavior-change. Bump parquet_version in a separate PR. + resolutionStrategy.force 'org.apache.parquet:parquet-avro:1.16.0' + resolutionStrategy.force 'org.apache.parquet:parquet-hadoop:1.16.0' // TODO(https://github.com/apache/beam/issues/38515): // Remove below pins when parquet-hadoop upgrades to hadoop-common:3.4.2 resolutionStrategy.force 'org.apache.hadoop:hadoop-common:3.3.6' From 980fe23cbfa25bd2ae94bb89126f727d1b91417b Mon Sep 17 00:00:00 2001 From: Peter-Phi-Tran Date: Thu, 30 Jul 2026 11:33:45 -0500 Subject: [PATCH 2/3] Update changelog --- CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index 1d7a775eab6d..ff3985239c24 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -83,7 +83,7 @@ * (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any, use pipeline option `--exclude_infer_dataclass_field_type` ([#38797](https://github.com/apache/beam/issues/38797)). However fixing forward is recommended. -* (Java) IcebergIO now requires Java 17 at runtime. This raises the floor in preparation for the Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)). +* (Java) IcebergIO and projects that use it must now be built with Java 17 or later. This raises the floor in preparation for the Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)). ## Deprecations From c46251bf6464167a4771ca18f6628e869d34a54f Mon Sep 17 00:00:00 2001 From: Yi Hu Date: Thu, 30 Jul 2026 15:22:42 -0400 Subject: [PATCH 3/3] Apply suggestions from code review Co-authored-by: Yi Hu --- CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index ff3985239c24..1ede8102b2de 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -83,7 +83,7 @@ * (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any, use pipeline option `--exclude_infer_dataclass_field_type` ([#38797](https://github.com/apache/beam/issues/38797)). However fixing forward is recommended. -* (Java) IcebergIO and projects that use it must now be built with Java 17 or later. This raises the floor in preparation for the Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)). +* (Java) IcebergIO and projects that use it must now be built with Java 17 or later as a result of Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)). ## Deprecations