bundle migrate: handle added/removed resources gracefully (#5463)

## Changes

- Update "bundle deployment migrate" to support added/removed resources:
- If resource is present in databricks.yml but not in terraform state
the migrate command no longer fails, it skips the resource, it'll be
deployed by direct engine.
- If resource is present in the state but not in databricks.yml,
migration preserves their ID in the state so that it can be deleted by
direct engine.

## Why
Towards auto-migration: we should migrate any situation without errors.

## Tests
New acceptance tests.

Author: denik

NEXT_CHANGELOG.md

@@ -9,6 +9,7 @@
 ### Bundles
 * Set the default `data_security_mode` to `DATA_SECURITY_MODE_AUTO` in bundle templates ([#5452](https://github.com/databricks/cli/pull/5452)).
 * Mark vector search index index_subtype as backend_default to prevent drift after deployment ([#5454](https://github.com/databricks/cli/pull/5454)).
+* `bundle deployment migrate`: handle resources added to or removed from `databricks.yml` since the last Terraform deploy ([#5463](https://github.com/databricks/cli/pull/5463)).
 
 ### Dependency updates
 

acceptance/bundle/migrate/added/databricks.yml

@@ -0,0 +1,7 @@
+bundle:
+  name: migrate-added-test
+
+resources:
+  jobs:
+    job_a: {name: "Job A"}
+    #job_b: {name: "Job B"}

acceptance/bundle/migrate/added/out.test.toml

@@ -0,0 +1,61 @@
+
+>>> [CLI] bundle deploy
+Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/migrate-added-test/default/files...
+Deploying resources...
+Updating deployment state...
+Deployment complete!
+
+>>> musterr [CLI] bundle deployment migrate
+Note: Migration should be done after a full deploy. Running plan now to verify that deployment was done:
+create jobs.job_b
+
+Plan: 1 to add, 0 to change, 0 to delete, 1 unchanged
+Error: 'databricks bundle plan' shows actions planned, aborting migration. Please run 'databricks bundle deploy' first to ensure your bundle is up to date, If actions persist after deploy, skip plan check with --noplancheck option
+
+>>> [CLI] bundle deployment migrate --noplancheck
+Success! Migrated 1 resources to direct engine state file: [TEST_TMP_DIR]/.databricks/bundle/default/resources.json
+
+Validate the migration by running "databricks bundle plan", there should be no actions planned.
+
+The state file is not synchronized to the workspace yet. To do that and finalize the migration, run "bundle deploy".
+
+To undo the migration, remove [TEST_TMP_DIR]/.databricks/bundle/default/resources.json and rename [TEST_TMP_DIR]/.databricks/bundle/default/terraform/terraform.tfstate.backup to [TEST_TMP_DIR]/.databricks/bundle/default/terraform/terraform.tfstate
+
+
+>>> DATABRICKS_BUNDLE_ENGINE=direct [CLI] bundle plan
+create jobs.job_b
+
+Plan: 1 to add, 0 to change, 0 to delete, 1 unchanged
+
+>>> DATABRICKS_BUNDLE_ENGINE=direct [CLI] bundle deploy
+Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/migrate-added-test/default/files...
+Deploying resources...
+Updating deployment state...
+Deployment complete!
+
+>>> print_requests.py //jobs/create
+{
+  "headers": {
+    "User-Agent": [
+      "cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_deploy cmd-exec-id/[UUID] interactive/none engine/direct auth/pat"
+    ]
+  },
+  "method": "POST",
+  "path": "/api/2.2/jobs/create",
+  "body": {
+    "deployment": {
+      "kind": "BUNDLE",
+      "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/migrate-added-test/default/state/metadata.json"
+    },
+    "edit_mode": "UI_LOCKED",
+    "format": "MULTI_TASK",
+    "max_concurrent_runs": 1,
+    "name": "Job B",
+    "queue": {
+      "enabled": true
+    }
+  }
+}
+
+>>> DATABRICKS_BUNDLE_ENGINE=direct [CLI] bundle plan
+Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged

acceptance/bundle/migrate/added/output.txt

@@ -0,0 +1,26 @@
+export DATABRICKS_BUNDLE_ENGINE=terraform
+
+# Deploy with terraform (only job_a; job_b is commented out)
+trace $CLI bundle deploy
+
+# Uncomment job_b (add it to config without deploying)
+update_file.py databricks.yml "#job_b" "job_b"
+
+# Should fail at plan check: job_b is "1 to add"
+trace musterr $CLI bundle deployment migrate
+
+# Should succeed: job_b skipped, will be created on next deploy
+trace $CLI bundle deployment migrate --noplancheck
+
+# After migration: plan shows job_b as "to add"
+trace DATABRICKS_BUNDLE_ENGINE=direct $CLI bundle plan | contains.py "1 to add"
+
+# Deploy creates job_b; verify via recorded requests
+rm out.requests.txt
+trace DATABRICKS_BUNDLE_ENGINE=direct $CLI bundle deploy
+trace print_requests.py //jobs/create
+
+# No further actions planned
+trace DATABRICKS_BUNDLE_ENGINE=direct $CLI bundle plan | contains.py "2 unchanged"
+
+rm out.requests.txt