Skip to content

[WIP] VPEX: one-click Databricks Connect environment setup#1924

Draft
rugpanov wants to merge 1 commit into
databricks:mainfrom
rugpanov:feature/vpex-environment-setup
Draft

[WIP] VPEX: one-click Databricks Connect environment setup#1924
rugpanov wants to merge 1 commit into
databricks:mainfrom
rugpanov:feature/vpex-environment-setup

Conversation

@rugpanov

Copy link
Copy Markdown
Contributor

⚠️ WIP / DO-NOT-MERGE — demo / proof-of-concept. Not ready to merge. See the caveat at the bottom.

Summary

Adds a guided one-click "VPEX" flow to set up a Databricks Connect Python environment matched to the selected compute target, with real UI feedback, and auto-adopt the resulting .venv.

It runs the real CLI in the open project folder:

databricks dbconnect init --serverless v4 --profile <detected>
databricks dbconnect sync  --serverless v4 --profile <detected>

parses the CLI's === Phase N: <name> === output to narrate each step, then selects .venv/bin/python automatically.

What's included

  • VpexEnvironmentSetup — pre-flight confirmation, phase-aware progress notification, dedicated "VPEX Demo" output channel, and success/failure pop-ups. The run is cancellable (SIGTERM → SIGKILL).
  • Auto-adopt interpreter — on success, refreshEnvironments() + updateActiveEnvironmentPath(".venv/bin/python", <project>) via the MS Python extension API; degrades gracefully if the extension is absent.
  • Three entry points, all triggering the same flow and reflecting the matched state on success:
    • command databricks.environment.setupVpex (+ databricks.environment.showVpexVersions)
    • a $(rocket) status-bar button
    • a node under the Python Environment tree section
  • MsPythonExtensionWrapper — resolve uv from ~/.local/bin / ~/.cargo/bin / $XDG_BIN_HOME when it isn't on the extension-host PATH, so uv venvs no longer fall back to the (absent) native pip.

This is intentionally kept parallel to the existing databricks.environment.setup (FeatureManager) flow rather than merged into it.

Verification

  • tsc --noEmit, eslint src, prettier: clean
  • Unit suite (VS Code test host): 237 passing, 0 failing
  • Ran the real databricks dbconnect init against a demo project: phases preflight → resolve → fetch → parse-python-version → plan → apply → ensure-python all ok; the final uv sync (PyPI-dependent) was blocked only by no network access in the test environment.

Backward compatibility

Additive: new commands, a new status-bar button, and a new tree node. The ConfigurationDataProvider constructor gains one parameter (internal). No changes to existing commands, settings, or persisted state/formats.

⚠️ Why this is do-not-merge

  • The dbconnect subcommand currently exists only in the dev CLI build, so CLI_PATH is hardcoded to a local absolute path (marked DEMO/POC in code). This must switch to CliWrapper.cliPath once the bundled CLI ships dbconnect.
  • serverless-v4 / Python 3.12 / databricks-connect 17.3 and macOS-only .venv/bin/python are hardcoded for the demo.

This pull request and its description were written by Isaac.

*Why*
Setting up a Databricks Connect Python environment that matches the selected
compute target is a manual, error-prone flow. This adds a guided one-click
path (the "VPEX" demo) that provisions a matched .venv and adopts it
automatically, with real UI feedback.

*What*
- New `VpexEnvironmentSetup` flow: pre-flight confirmation, phase-aware
  progress notification, dedicated "VPEX Demo" output channel, and
  success/failure pop-ups. Runs the real CLI in the project folder:
  `databricks dbconnect init` then `dbconnect sync` (--serverless v4
  --profile <detected>), parsing the CLI's `=== Phase N: name ===` output to
  narrate each step. Cancellable (SIGTERM/SIGKILL).
- On success, auto-adopts `.venv/bin/python` via the MS Python extension API
  (refresh + updateActiveEnvironmentPath), degrading gracefully if absent.
- New commands `databricks.environment.setupVpex` /
  `.showVpexVersions`, a `$(rocket)` status-bar button, and a node under the
  "Python Environment" tree section — all triggering the same flow and
  reflecting the matched state on success.
- MsPythonExtensionWrapper: resolve `uv` from ~/.local/bin / ~/.cargo/bin /
  $XDG_BIN_HOME when it isn't on the extension-host PATH, so uv venvs no
  longer fall back to the (absent) native pip.

Note: the `dbconnect` subcommand currently exists only in the dev CLI build,
so CLI_PATH is hardcoded to it pending a bundled CLI that ships `dbconnect`
(marked DEMO/POC in code). Not for merge.

*Verification*
- tsc --noEmit, eslint src, prettier: clean
- Unit suite (VS Code test host): 237 passing, 0 failing
- Ran the real `databricks dbconnect init` against the demo project: phases
  preflight→resolve→fetch→parse-python-version→plan→apply→ensure-python all
  ok; final `uv sync` (PyPI-dependent) blocked only by no network access in
  this environment.

Co-authored-by: Isaac
@github-actions

Copy link
Copy Markdown
Contributor

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

  • PR number: 1924
  • Commit SHA: 6852bc9312d9b8d4b3992b8b53c8248aec84c40e

Checks will be approved automatically on success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant