REST: treat HTTP 400 commit-validation failures as CommitFailedException

Some REST catalog implementations (e.g., Databricks Unity Catalog) return
HTTP 400 with a "commit validation failed" message for concurrent-write
conflicts instead of the spec-mandated HTTP 409. Because CommitErrorHandler
previously mapped all 400 responses to BadRequestException, these conflicts
escaped SnapshotProducer's retry-with-refresh loop entirely and propagated
as fatal errors.

Add a case 400 check in CommitErrorHandler that recognises the conflict
pattern and raises CommitFailedException instead, restoring normal
optimistic-concurrency retry behaviour for non-compliant catalogs. Responses
that do not match the pattern still fall through to the default 400 handler
and raise BadRequestException as before.

Author: martinskeem

core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java

@@ -18,6 +18,7 @@
  */
 package org.apache.iceberg.rest;
 
+import java.util.Locale;
 import java.util.function.Consumer;
 import org.apache.iceberg.exceptions.AlreadyExistsException;
 import org.apache.iceberg.exceptions.BadRequestException;
@@ -122,6 +123,18 @@ private static class CommitErrorHandler extends DefaultErrorHandler {
     @Override
     public void accept(ErrorResponse error) {
       switch (error.code()) {
+        case 400:
+          // Some REST catalog implementations (e.g., Databricks Unity Catalog) return HTTP 400
+          // with a commit-conflict message instead of the spec-mandated 409. Treat such responses
+          // as CommitFailedException so the standard retry-with-refresh loop can handle them.
+          // Responses that indicate a genuinely malformed request (no commit-related message) still
+          // fall through to the default 400 handler which raises BadRequestException.
+          if (error.message() != null
+              && error.message().toLowerCase(Locale.ROOT).contains("commit")
+              && error.message().toLowerCase(Locale.ROOT).contains("validation failed")) {
+            throw new CommitFailedException("Commit failed: %s", error.message());
+          }
+          break;
         case 404:
           throw new NoSuchTableException("%s", error.message());
         case 409: