Skip to content

Commit e8176fa

Browse files
lintingbinclaude
andcommitted
[AMORO-4163][ams] Fix CommitFailedException when loading legacy mixed-iceberg tables on Iceberg 1.7.2
Iceberg 1.7.2 introduced a breaking change in HadoopTableOperations.commit(): it now uses reference equality (==) to compare the `base` parameter against the internally cached currentMetadata. If the references differ (e.g. because versionAndMetadata() refreshes the internal state), a CommitFailedException with no cause is thrown, which propagates through Thrift and causes table creation to fail with "update table meta failed". The commit in newTableOperations() is a best-effort migration step that backfills REST-based mixed-format properties into legacy table metadata (tables created before v0.7.0). It is not required for the table to function correctly; a failure here should not abort table loading. Wrap the call in try-catch and log a warning so the migration is retried on the next load instead of surfacing as a fatal error. This fixes the flaky TestInternalMixedCatalogService.CompatibilityCatalogTests #testNewCatalogLoadHistorical test observed after the Iceberg 1.7.2 upgrade. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 10d13d4 commit e8176fa

1 file changed

Lines changed: 15 additions & 2 deletions

File tree

amoro-ams/src/main/java/org/apache/amoro/server/table/internal/InternalMixedIcebergHandler.java

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,14 @@
3232
import org.apache.hadoop.fs.Path;
3333
import org.apache.iceberg.TableOperations;
3434
import org.apache.iceberg.catalog.TableIdentifier;
35+
import org.slf4j.Logger;
36+
import org.slf4j.LoggerFactory;
3537

3638
import java.util.Map;
3739

3840
/** Table handler for mixed-iceberg format */
3941
public class InternalMixedIcebergHandler extends InternalIcebergHandler {
42+
private static final Logger LOG = LoggerFactory.getLogger(InternalMixedIcebergHandler.class);
4043
private final boolean changeStore;
4144
private final CatalogMeta catalogMeta;
4245

@@ -75,8 +78,18 @@ private TableOperations newTableOperations(boolean changeStore) {
7578
}
7679
org.apache.iceberg.TableMetadata legacyCurrent = legacyTableMetadata(current, changeStore);
7780
if (!current.equals(legacyCurrent)) {
78-
// add rest based mixed-format table properties
79-
ops.commit(current, legacyCurrent);
81+
// add rest based mixed-format table properties to legacy table metadata
82+
try {
83+
ops.commit(current, legacyCurrent);
84+
} catch (Exception e) {
85+
// Commit may fail transiently (e.g. due to Iceberg internal stale-metadata checks).
86+
// This is a best-effort upgrade path: the properties will be retried on the next load.
87+
LOG.warn(
88+
"Failed to update legacy mixed-iceberg table {} with rest-based properties,"
89+
+ " will retry on next load",
90+
tableLocation,
91+
e);
92+
}
8093
}
8194
return ops;
8295
}

0 commit comments

Comments
 (0)