Skip to content

Expose stage/release status in schema#5443

Draft
janniklasrose wants to merge 3 commits into
cligenfrom
janniklasrose/sdk-field-status-in-schema
Draft

Expose stage/release status in schema#5443
janniklasrose wants to merge 3 commits into
cligenfrom
janniklasrose/sdk-field-status-in-schema

Conversation

@janniklasrose

@janniklasrose janniklasrose commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Stacked on #5484

Changes

Expose launch stage/release status in the bundle schema

Why

Non-GA fields should be annotated as such when viewing a bundle in IDEs.

Tests

Example vector search endpoint:

Enum with one GA, one PuPr:
Screenshot 2026-06-04 at 14 29 40

PuPr field without any annotation:
Screenshot 2026-06-04 at 14 29 49

Comment thread bundle/docsgen/main.go Outdated
@@ -139,13 +139,51 @@ func assignAnnotation(s *jsonschema.Schema, a annotation.Descriptor) {
s.Deprecated = true
s.DeprecationMessage = a.DeprecationMessage
}
if a.Preview == "PRIVATE" {
s.DoNotSuggest = true
if a.Preview != "" {

@ilyakuz-db ilyakuz-db Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shreyas-goenka do we still use docsgen?

@janniklasrose janniklasrose marked this pull request as ready for review June 8, 2026 11:25
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/bundle/ - needs approval

14 files changed
Suggested: @pietern
Also eligible: @shreyas-goenka, @andrewnester, @denik, @lennartkats-db, @anton-107

General files (require maintainer)

66 files changed
Based on git history:

  • @pietern -- recent work in bundle/internal/schema/, python/databricks/bundles/pipelines/_models/, bundle/schema/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Commit: e6b64cd

Run: 27259369137

Env 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 7 15 261 926 6:41
💚​ aws windows 7 15 263 924 13:08
💚​ aws-ucws linux 7 15 357 840 6:48
💚​ aws-ucws windows 7 15 359 838 11:52
💚​ azure linux 1 17 264 924 6:14
💚​ azure windows 1 17 266 922 10:57
💚​ azure-ucws linux 1 17 362 836 10:06
💚​ azure-ucws windows 1 17 364 834 11:58
💚​ gcp linux 1 17 260 927 7:41
💚​ gcp windows 1 17 262 925 11:25
22 interesting tests: 15 SKIP, 7 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/grants/select 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
Top 31 slowest tests (at least 2 minutes):
duration env testname
6:15 aws windows TestAccept
6:08 gcp windows TestAccept
6:01 aws-ucws windows TestAccept
5:57 azure windows TestAccept
5:50 azure-ucws windows TestAccept
4:32 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:26 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:19 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:17 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:35 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:24 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:24 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:18 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:12 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:08 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:00 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:58 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:56 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:53 gcp linux TestAccept
2:53 aws linux TestAccept
2:52 azure-ucws linux TestAccept
2:51 aws-ucws linux TestAccept
2:50 azure linux TestAccept
2:46 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:45 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:45 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:43 azure-ucws linux TestSQLExecScalar
2:37 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:36 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:30 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:29 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@janniklasrose janniklasrose enabled auto-merge June 8, 2026 12:35
@janniklasrose janniklasrose force-pushed the janniklasrose/sdk-field-status-in-schema branch from a4089fd to 1a4d7eb Compare June 9, 2026 23:03
@janniklasrose janniklasrose changed the base branch from main to cligen June 9, 2026 23:04
auto-merge was automatically disabled June 9, 2026 23:04

Merge commits are not allowed on this repository

Comment thread bundle/docsgen/main.go Outdated
Comment thread bundle/internal/annotation/descriptor.go Outdated
Comment thread bundle/internal/schema/annotations.go Outdated
Comment on lines +224 to +229
case "PRIVATE_PREVIEW":
return "[PrPr]"
case "PUBLIC_BETA":
return "[Beta]"
case "PUBLIC_PREVIEW":
return "[PuPr]"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PrPr and PuPr aren't used in Databricks public documentation, but for space-constrained values I think it's still more appropriate. wdyt @juliacrawf-db ?

@juliacrawf-db juliacrawf-db Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly I think those are very Databricks-internally-specific acronyms and we shouldn't be putting them in public facing content/APIs/source because the meaning isn't super obvious. Ideally it's PublicPreview and PrivatePreview, but if it's constrained maybe PrivatePr and PublicPr (imo the private/public part seems more important to expand)? (Or do you need it to be even shorter?)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... now I'm having second thoughts about that...because "Pr" likely will get translated as PR in this context...

Comment thread bundle/internal/schema/annotations.go Outdated
Comment thread bundle/internal/schema/annotations.go Outdated
// field's description. cli.json is filtered at min-stage=PRIVATE_PREVIEW
// upstream, so DEVELOPMENT never reaches here and GA yields no tag.
//
// Keep this in sync with previewTag in bundle/docsgen/main.go.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why two sources of truth?

Read launch_stage, enum_launch_stages and enum_descriptions from the
checked-in .codegen/cli.json and surface them in the generated bundle
schema, docs and Python models:

- Prefix field descriptions with [Public Preview] / [Beta] /
  [Private Preview] so the stage shows in editor hover tooltips and the
  generated docstrings.
- Emit the parallel enumDescriptions array so VSCode renders a per-value
  stage label and description in autocomplete dropdowns.
- Carry the raw x-databricks-launch-stage alongside the existing
  x-databricks-preview, and hide PRIVATE_PREVIEW fields via doNotSuggest.

PreviewTag lives in bundle/internal/annotation as the single source of
truth shared by the schema and docs generators. The PUBLIC_BETA label is
"[Beta]" to match Databricks public documentation.

cli.json is filtered upstream at min-stage=PRIVATE_PREVIEW, so no
DEVELOPMENT surface reaches the extractor and launch_stage is already
clean (GA plus the three preview stages). The OpenAPI-spec-based
DEVELOPMENT filtering and preview normalization the earlier version
carried are no longer needed.

assignAnnotation flags an open question (in a comment) on whether either
x-databricks-preview or x-databricks-launch-stage is consumed downstream
of the published schema, pending confirmation with the team.

Regenerated jsonschema.json, jsonschema_for_docs.json,
annotations_openapi.yml, the bundle docs and the Python (pydabs) models.

Co-authored-by: Isaac
Map ./bundle/schema/jsonschema.json onto databricks.yml / bundle.yml in
.vscode/settings.json so the schema generated in this repo is the one the
editor validates and autocompletes against.

Co-authored-by: Isaac
@janniklasrose janniklasrose force-pushed the janniklasrose/sdk-field-status-in-schema branch from 1a4d7eb to e69435c Compare June 10, 2026 06:56
Comment thread bundle/internal/schema/annotations.go Outdated

// previewTagShort is the compact counterpart to previewTag, used for per-enum-
// value labels where vertical space in the dropdown is tighter.
func previewTagShort(launchStage string) string {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to annotation.PreviewTagShort

// derivable from it (PRIVATE iff PRIVATE_PREVIEW) and is kept only for
// backwards compatibility, so PRIVATE_PREVIEW must hide the field here too.
//
// OPEN QUESTION (confirm with the team before finalizing this PR): is

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open question

Addresses review feedback: PreviewTagShort now lives alongside PreviewTag
in bundle/internal/annotation, keeping the launch-stage label vocabulary
in a single place. Pure refactor, no change to generated output.

Co-authored-by: Isaac
@janniklasrose janniklasrose marked this pull request as draft June 10, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants