Make admin tool bootstrap idempotent for already-bootstrapped realms (apache#4996)

* Make admin tool bootstrap idempotent for already-bootstrapped realms

Bootstrapping a realm that already has a root principal used to throw
IllegalArgumentException, which BootstrapCommand mapped to the generic
EXIT_CODE_BOOTSTRAP_ERROR. Automated deployments (e.g. Kubernetes Jobs
or ArgoCD hooks) that run bootstrap on every deploy therefore failed on
any redeploy, and callers could not distinguish the benign case from
real failures except by matching the error-message text.

AuthBootstrapUtil.createPolarisPrincipalForRealm now returns a
PrincipalSecretsResult with ENTITY_ALREADY_EXISTS instead of throwing,
PolarisMetaStoreManager.bootstrapPolarisService propagates that result,
and BootstrapCommand reports "Realm 'x' is already bootstrapped;
skipping." while keeping exit code 0. Existing credentials are never
returned or altered; realms that are not yet bootstrapped are still
bootstrapped normally.

Fixes apache#4995

* Address review feedback: exact status assertion, distinct skip-completion message

- testBootstrapAgainIsNoOp now asserts ReturnStatus.ENTITY_ALREADY_EXISTS
  directly instead of isSuccess() + alreadyExists()
- BootstrapCommand reports 'Bootstrap completed (N realm(s) already
  bootstrapped).' when realms were skipped, keeping 'Bootstrap completed
  successfully.' for full bootstraps

Author: sepandr

CHANGELOG.md

@@ -62,6 +62,12 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
 - Added optional `sessionPolicy` field to `SigV4AuthenticationParameters` for catalog federation. When set, the IAM session policy JSON is attached to the STS AssumeRole request, allowing administrators to restrict vended credentials to only the required AWS services and actions (Principle of Least Privilege).
 
 ### Changes
+- The admin tool's `bootstrap` command is now idempotent: bootstrapping a realm that is already
+  bootstrapped is reported as "Realm '<realm>' is already bootstrapped; skipping." and no longer
+  fails the command, making automated bootstrap jobs safe to re-run. Correspondingly,
+  `PolarisMetaStoreManager.bootstrapPolarisService` now returns a result with
+  `ReturnStatus.ENTITY_ALREADY_EXISTS` instead of throwing `IllegalArgumentException` when the
+  root principal already exists. Existing credentials are never returned or altered.
 - Added REPL support to Polaris CLI.
 - The `nosql maintenance-run` admin command now rejects a new run when the latest recorded maintenance run is still unfinished, unless the operator explicitly passes `--supersede-run=<run-id>`.
 - Added version option to Polaris CLI.

polaris-core/src/main/java/org/apache/polaris/core/auth/AuthBootstrapUtil.java

@@ -32,6 +32,7 @@
 import org.apache.polaris.core.entity.PrincipalEntity;
 import org.apache.polaris.core.entity.PrincipalRoleEntity;
 import org.apache.polaris.core.persistence.PolarisMetaStoreManager;
+import org.apache.polaris.core.persistence.dao.entity.BaseResult;
 import org.apache.polaris.core.persistence.dao.entity.CreatePrincipalResult;
 import org.apache.polaris.core.persistence.dao.entity.GenerateEntityIdResult;
 import org.apache.polaris.core.persistence.dao.entity.PrincipalSecretsResult;
@@ -54,11 +55,9 @@ public static PrincipalSecretsResult createPolarisPrincipalForRealm(
 
     Optional<PrincipalEntity> preliminaryRootPrincipal = metaStoreManager.findRootPrincipal(ctx);
     if (preliminaryRootPrincipal.isPresent()) {
-      String overrideMessage =
-          "It appears this metastore manager has already been bootstrapped. "
-              + "To continue bootstrapping, please first purge the metastore with the `purge` command.";
-      LOGGER.error("\n\n {} \n\n", overrideMessage);
-      throw new IllegalArgumentException(overrideMessage);
+      LOGGER.info("Realm is already bootstrapped (root principal exists); nothing to do.");
+      return new PrincipalSecretsResult(
+          BaseResult.ReturnStatus.ENTITY_ALREADY_EXISTS, "realm is already bootstrapped");
     }
 
     // Create a root container entity that can represent the securable for any top-level grants.

polaris-core/src/main/java/org/apache/polaris/core/persistence/LocalPolarisMetaStoreManagerFactory.java

@@ -195,11 +195,11 @@ private PrincipalSecretsResult bootstrapServiceAndCreatePolarisPrincipalForRealm
     Optional<PrincipalEntity> preliminaryRootPrincipal =
         metaStoreManager.findRootPrincipal(polarisContext);
     if (preliminaryRootPrincipal.isPresent()) {
-      String overrideMessage =
-          "It appears this metastore manager has already been bootstrapped. "
-              + "To continue bootstrapping, please first purge the metastore with the `purge` command.";
-      LOGGER.error("\n\n {} \n\n", overrideMessage);
-      throw new IllegalArgumentException(overrideMessage);
+      LOGGER.info(
+          "Realm {} is already bootstrapped (root principal exists); nothing to do.",
+          realmContext.getRealmIdentifier());
+      return new PrincipalSecretsResult(
+          BaseResult.ReturnStatus.ENTITY_ALREADY_EXISTS, "realm is already bootstrapped");
     }
 
     metaStoreManager.bootstrapPolarisService(polarisContext);

polaris-core/src/main/java/org/apache/polaris/core/persistence/PolarisMetaStoreManager.java

@@ -66,15 +66,16 @@ public interface PolarisMetaStoreManager
 
   /**
    * Bootstrap the Polaris service, creating the root catalog, root principal, and associated
-   * service admin role. Will fail if the service has already been bootstrapped.
+   * service admin role. If the service has already been bootstrapped, this is a no-op that returns
+   * a result with {@link BaseResult.ReturnStatus#ENTITY_ALREADY_EXISTS}.
    *
    * @param callCtx call context
-   * @return the result of the bootstrap attempt
+   * @return the result of the bootstrap attempt; {@code alreadyExists()} is true if the service was
+   *     already bootstrapped
    */
   @NonNull
   default BaseResult bootstrapPolarisService(@NonNull PolarisCallContext callCtx) {
-    AuthBootstrapUtil.createPolarisPrincipalForRealm(this, callCtx);
-    return new BaseResult(BaseResult.ReturnStatus.SUCCESS);
+    return AuthBootstrapUtil.createPolarisPrincipalForRealm(this, callCtx);
   }
 
   /**

polaris-core/src/testFixtures/java/org/apache/polaris/core/persistence/BasePolarisMetaStoreManagerTest.java

@@ -44,6 +44,7 @@
 import org.apache.polaris.core.entity.PrincipalEntity;
 import org.apache.polaris.core.entity.TaskEntity;
 import org.apache.polaris.core.exceptions.AlreadyExistsException;
+import org.apache.polaris.core.persistence.dao.entity.BaseResult;
 import org.apache.polaris.core.persistence.pagination.PageToken;
 import org.assertj.core.api.Assertions;
 import org.assertj.core.api.InstanceOfAssertFactories;
@@ -83,6 +84,19 @@ protected void validateBootstrap() {
     polarisTestMetaStoreManager.validateBootstrap();
   }
 
+  /** re-bootstrapping an already-bootstrapped service must be a graceful no-op */
+  @Test
+  protected void testBootstrapAgainIsNoOp() {
+    PolarisMetaStoreManager metaStoreManager = polarisTestMetaStoreManager.polarisMetaStoreManager;
+    // the test driver bootstrapped the service already; bootstrap a second time
+    BaseResult secondBootstrap =
+        metaStoreManager.bootstrapPolarisService(polarisTestMetaStoreManager.polarisCallContext);
+    Assertions.assertThat(secondBootstrap.getReturnStatus())
+        .isEqualTo(BaseResult.ReturnStatus.ENTITY_ALREADY_EXISTS);
+    // the existing realm must be left fully intact
+    polarisTestMetaStoreManager.validateBootstrap();
+  }
+
   @Test
   protected void testCreateTestCatalog() {
     // allocate test driver

runtime/admin/src/main/java/org/apache/polaris/admintool/BootstrapCommand.java

@@ -155,6 +155,7 @@ public Integer call() {
 
       // Log any errors:
       boolean success = true;
+      int alreadyBootstrappedCount = 0;
       for (Map.Entry<String, PrincipalSecretsResult> result : results.entrySet()) {
         if (result.getValue().isSuccess()) {
           String realm = result.getKey();
@@ -169,6 +170,15 @@ public Integer call() {
                     result.getValue().getPrincipalSecrets().getMainSecret());
             spec.commandLine().getOut().println(msg);
           }
+        } else if (result.getValue().alreadyExists()) {
+          // Re-running bootstrap on an existing realm is a no-op, not an error; this
+          // keeps automated bootstrap jobs idempotent. Existing credentials are never
+          // returned or altered.
+          alreadyBootstrappedCount++;
+          String realm = result.getKey();
+          spec.commandLine()
+              .getOut()
+              .printf("Realm '%s' is already bootstrapped; skipping.%n", realm);
         } else {
           String realm = result.getKey();
           spec.commandLine()
@@ -181,7 +191,15 @@ public Integer call() {
       }
 
       if (success) {
-        spec.commandLine().getOut().println("Bootstrap completed successfully.");
+        if (alreadyBootstrappedCount > 0) {
+          spec.commandLine()
+              .getOut()
+              .printf(
+                  "Bootstrap completed (%d realm(s) already bootstrapped).%n",
+                  alreadyBootstrappedCount);
+        } else {
+          spec.commandLine().getOut().println("Bootstrap completed successfully.");
+        }
         return 0;
       } else {
         spec.commandLine().getErr().println("Bootstrap encountered errors during operation.");