Skip to content

Commit 3857f0f

Browse files
committed
Add postgres_catalogs bundle resource
## Changes New `postgres_catalogs` resource binding a Unity Catalog catalog to a Postgres database on a Lakebase Autoscaling branch. Supported on both direct and terraform deployment engines. The spec fields are classified as both `recreate_on_changes` and `ignore_remote_changes: input_only`. The two cover orthogonal diffs the planner runs — recreate fires on local edits to an immutable field, and ignore_remote silences the phantom drift from GET not echoing spec back today. Lift the `input_only` entries once the backend starts returning spec. ## Tests Acceptance coverage: `basic` and `recreate` exercise each engine, plus the existing `no_drift` and `migrate` invariants pick up the new resource. Both engines produce identical human-readable output and identical wire bodies; only the captured request streams diverge by filename (`out.requests.{direct,terraform}.json`). Verified end to end on a live workspace: the bundle deploys a project and catalog, a row written directly into the bound Postgres database becomes visible through the UC federated view, and a follow-up write shows up on re-read. This pull request and its description were written by Isaac.
1 parent d8bda62 commit 3857f0f

46 files changed

Lines changed: 1003 additions & 34 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agent/skills/pr-checklist/SKILL.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,9 @@ Before submitting a PR, run these commands to match what CI checks. CI uses the
1616
# 3. Tests (CI runs with both deployment engines)
1717
./task test
1818

19-
# 4. If you changed bundle config structs or schema-related code:
19+
# 4. If you changed bundle config structs, schema, or direct-engine resource code:
2020
./task generate-schema
21+
./task generate-direct
2122

2223
# 5. If you changed files in python/:
2324
./task pydabs-codegen pydabs-test pydabs-lint pydabs-docs

NEXT_CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,6 @@
1010

1111
### Bundles
1212
* Make sure warnings asking for approval are understood by agents ([#5239](https://github.com/databricks/cli/pull/5239))
13+
* Add `postgres_catalogs` resource to bind a Unity Catalog catalog to a Postgres database on a Lakebase Autoscaling branch.
1314

1415
### Dependency updates
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
bundle:
2+
name: test-bundle-$UNIQUE_NAME
3+
4+
resources:
5+
postgres_projects:
6+
project:
7+
project_id: test-pg-project-$UNIQUE_NAME
8+
display_name: Test Postgres Project
9+
10+
postgres_catalogs:
11+
catalog:
12+
catalog_id: test_pg_catalog_$UNIQUE_NAME
13+
branch: ${resources.postgres_projects.project.name}/branches/production
14+
postgres_database: appdb
15+
create_database_if_missing: true

acceptance/bundle/invariant/continue_293/out.test.toml

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

acceptance/bundle/invariant/migrate/out.test.toml

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

acceptance/bundle/invariant/no_drift/out.test.toml

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

acceptance/bundle/invariant/test.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ EnvMatrix.INPUT_CONFIG = [
4444
"pipeline.yml.tmpl",
4545
"pipeline_config_dots.yml.tmpl",
4646
"postgres_branch.yml.tmpl",
47+
"postgres_catalog.yml.tmpl",
4748
"postgres_endpoint.yml.tmpl",
4849
"postgres_project.yml.tmpl",
4950
"registered_model.yml.tmpl",
@@ -67,6 +68,7 @@ no_alert_on_cloud = ["CONFIG_Cloud=true", "INPUT_CONFIG=alert.yml.tmpl"]
6768
no_postgres_project_on_cloud = ["CONFIG_Cloud=true", "INPUT_CONFIG=postgres_project.yml.tmpl"]
6869
no_postgres_branch_on_cloud = ["CONFIG_Cloud=true", "INPUT_CONFIG=postgres_branch.yml.tmpl"]
6970
no_postgres_endpoint_on_cloud = ["CONFIG_Cloud=true", "INPUT_CONFIG=postgres_endpoint.yml.tmpl"]
71+
no_postgres_catalog_on_cloud = ["CONFIG_Cloud=true", "INPUT_CONFIG=postgres_catalog.yml.tmpl"]
7072

7173
# External locations require actual storage credentials with cloud IAM setup
7274
# which are environment-specific, so we only test locally with the mock server

acceptance/bundle/refschema/out.fields.txt

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2733,6 +2733,28 @@ resources.postgres_branches.*.ttl *duration.Duration INPUT STATE
27332733
resources.postgres_branches.*.uid string REMOTE
27342734
resources.postgres_branches.*.update_time *time.Time REMOTE
27352735
resources.postgres_branches.*.url string INPUT
2736+
resources.postgres_catalogs.*.branch string INPUT STATE
2737+
resources.postgres_catalogs.*.catalog_id string INPUT STATE
2738+
resources.postgres_catalogs.*.create_database_if_missing bool INPUT STATE
2739+
resources.postgres_catalogs.*.create_time *time.Time REMOTE
2740+
resources.postgres_catalogs.*.id string INPUT
2741+
resources.postgres_catalogs.*.lifecycle resources.Lifecycle INPUT
2742+
resources.postgres_catalogs.*.lifecycle.prevent_destroy bool INPUT
2743+
resources.postgres_catalogs.*.modified_status string INPUT
2744+
resources.postgres_catalogs.*.name string REMOTE
2745+
resources.postgres_catalogs.*.postgres_database string INPUT STATE
2746+
resources.postgres_catalogs.*.spec *postgres.CatalogCatalogSpec REMOTE
2747+
resources.postgres_catalogs.*.spec.branch string REMOTE
2748+
resources.postgres_catalogs.*.spec.create_database_if_missing bool REMOTE
2749+
resources.postgres_catalogs.*.spec.postgres_database string REMOTE
2750+
resources.postgres_catalogs.*.status *postgres.CatalogCatalogStatus REMOTE
2751+
resources.postgres_catalogs.*.status.branch string REMOTE
2752+
resources.postgres_catalogs.*.status.catalog_id string REMOTE
2753+
resources.postgres_catalogs.*.status.postgres_database string REMOTE
2754+
resources.postgres_catalogs.*.status.project string REMOTE
2755+
resources.postgres_catalogs.*.uid string REMOTE
2756+
resources.postgres_catalogs.*.update_time *time.Time REMOTE
2757+
resources.postgres_catalogs.*.url string INPUT
27362758
resources.postgres_endpoints.*.autoscaling_limit_max_cu float64 INPUT STATE
27372759
resources.postgres_endpoints.*.autoscaling_limit_min_cu float64 INPUT STATE
27382760
resources.postgres_endpoints.*.create_time *time.Time REMOTE
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
bundle:
2+
name: deploy-postgres-catalog-$UNIQUE_NAME
3+
4+
sync:
5+
paths: []
6+
7+
resources:
8+
postgres_projects:
9+
my_project:
10+
project_id: test-pg-proj-$UNIQUE_NAME
11+
display_name: "Test Project for Catalog"
12+
pg_version: 16
13+
14+
postgres_catalogs:
15+
my_catalog:
16+
catalog_id: lakebase_test_$UNIQUE_NAME
17+
branch: ${resources.postgres_projects.my_project.id}/branches/production
18+
postgres_database: appdb
19+
create_database_if_missing: true
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
{
2+
"method": "POST",
3+
"path": "/api/2.0/postgres/projects",
4+
"q": {
5+
"project_id": "test-pg-proj-[UNIQUE_NAME]"
6+
},
7+
"body": {
8+
"spec": {
9+
"display_name": "Test Project for Catalog",
10+
"pg_version": 16
11+
}
12+
}
13+
}
14+
{
15+
"method": "POST",
16+
"path": "/api/2.0/postgres/catalogs",
17+
"q": {
18+
"catalog_id": "lakebase_test_[UNIQUE_NAME]"
19+
},
20+
"body": {
21+
"spec": {
22+
"branch": "projects/test-pg-proj-[UNIQUE_NAME]/branches/production",
23+
"create_database_if_missing": true,
24+
"postgres_database": "appdb"
25+
}
26+
}
27+
}
28+
{
29+
"method": "GET",
30+
"path": "/api/2.0/postgres/catalogs/lakebase_test_[UNIQUE_NAME]"
31+
}

0 commit comments

Comments
 (0)