Skip to content

Commit 60fd893

Browse files
authored
[AMORO-4128] Drop mixed Flink 1.16 support (#4132)
* Drop mixed Flink 1.16 support (#4128) Remove mixed Flink 1.16 modules and update documentation to reflect the supported Flink versions (1.17, 1.18). This unblocks future Iceberg upgrades that no longer support Flink 1.16. - Remove v1.16/amoro-mixed-flink-1.16 and v1.16/amoro-mixed-flink-runtime-1.16 modules - Remove Flink 1.16 entries from parent pom.xml module list - Update version matrices in README and docs - Update Flink documentation links from 1.16 to 1.17 - Clean up stale Flink 1.15 references in docs Signed-off-by: Jiwon Park <jpark92@outlook.kr> * Address review comment: sync MixedCatalog method with Iceberg 1.6.x source Signed-off-by: Jiwon Park <jpark92@outlook.kr> * Fix stale Spark 3.2 path reference in deployment docs Update the example path from the non-existent amoro-format-mixed-spark/v3.2 to amoro-mixed-spark/v3.3 to match the current project structure. Also remove a trailing parenthesis typo in the jar filename comment. Signed-off-by: Jiwon Park <jpark92@outlook.kr> --------- Signed-off-by: Jiwon Park <jpark92@outlook.kr>
1 parent 7cebea4 commit 60fd893

12 files changed

Lines changed: 31 additions & 358 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ Amoro support multiple processing engines for Mixed format as below:
8282

8383
| Processing Engine | Version | Batch Read | Batch Write | Batch Overwrite | Streaming Read | Streaming Write | Create Table | Alter Table |
8484
|-------------------|------------------------|-------------|-------------|-----------------|----------------|-----------------|--------------|-------------|
85-
| Flink | 1.16.x, 1.17.x, 1.18.x | &#x2714; | &#x2714; | &#x2716; | &#x2714; | &#x2714; | &#x2714; | &#x2716; |
85+
| Flink | 1.17.x, 1.18.x | &#x2714; | &#x2714; | &#x2716; | &#x2714; | &#x2714; | &#x2714; | &#x2716; |
8686
| Spark | 3.3, 3.4, 3.5 | &#x2714; | &#x2714; | &#x2714; | &#x2716; | &#x2716; | &#x2714; | &#x2714; |
8787
| Hive | 2.x, 3.x | &#x2714; | &#x2716; | &#x2714; | &#x2716; | &#x2716; | &#x2716; | &#x2714; |
8888
| Trino | 406 | &#x2714; | &#x2716; | &#x2714; | &#x2716; | &#x2716; | &#x2716; | &#x2714; |

amoro-format-mixed/amoro-mixed-flink/amoro-mixed-flink-common/src/main/java/org/apache/amoro/flink/catalog/MixedCatalog.java

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -665,7 +665,7 @@ private void validateColumnOrder(CatalogBaseTable table) {
665665

666666
/**
667667
* copy from
668-
* https://github.com/apache/iceberg/blob/main/flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java#L425C23-L425C54
668+
* https://github.com/apache/iceberg/blob/1.6.x/flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java#L425-L448
669669
*
670670
* @param ct1 CatalogTable before
671671
* @param ct2 CatalogTable after
@@ -687,9 +687,15 @@ private static void validateTableSchemaAndPartition(CatalogTable ct1, CatalogTab
687687
if (!(Objects.equal(ts1.getTableColumns(), ts2.getTableColumns())
688688
&& Objects.equal(ts1.getWatermarkSpecs(), ts2.getWatermarkSpecs())
689689
&& equalsPrimary)) {
690-
throw new UnsupportedOperationException("Altering schema is not supported yet.");
690+
throw new UnsupportedOperationException(
691+
"Altering schema is not supported in the old alterTable API. "
692+
+ "To alter schema, use the other alterTable API and provide a list of TableChange's.");
691693
}
692694

695+
validateTablePartition(ct1, ct2);
696+
}
697+
698+
private static void validateTablePartition(CatalogTable ct1, CatalogTable ct2) {
693699
if (!ct1.getPartitionKeys().equals(ct2.getPartitionKeys())) {
694700
throw new UnsupportedOperationException("Altering partition keys is not supported yet.");
695701
}

amoro-format-mixed/amoro-mixed-flink/pom.xml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,6 @@
3636
<module>amoro-mixed-flink-common</module>
3737
<module>amoro-mixed-flink-common-format</module>
3838
<module>amoro-mixed-flink-common-iceberg-bridge</module>
39-
<module>v1.16/amoro-mixed-flink-1.16</module>
40-
<module>v1.16/amoro-mixed-flink-runtime-1.16</module>
4139
<module>v1.17/amoro-mixed-flink-1.17</module>
4240
<module>v1.17/amoro-mixed-flink-runtime-1.17</module>
4341
<module>v1.18/amoro-mixed-flink-1.18</module>

amoro-format-mixed/amoro-mixed-flink/v1.16/amoro-mixed-flink-1.16/pom.xml

Lines changed: 0 additions & 100 deletions
This file was deleted.

amoro-format-mixed/amoro-mixed-flink/v1.16/amoro-mixed-flink-runtime-1.16/pom.xml

Lines changed: 0 additions & 229 deletions
This file was deleted.

0 commit comments

Comments
 (0)