[telemetry] Detect Python package manager(s) at project setup (#1918)

## Changes

Measurement-only telemetry to learn which Python package manager(s) our
users' projects actually use (pip / conda / uv / poetry), so the VPEX
setup-flow investment can be prioritized from first-party data instead
of public-survey estimates. No setup behavior changes — this is
detection only.

The work splits cleanly into three layers so each is independently
testable and the dependency direction stays correct (high-level →
low-level):

- **Pure classifier** (`packageManagerDetection.ts`): given a set of
already-collected signals, reports every applicable manager, a
best-guess primary (priority `uv > poetry > conda > pip`), the firing
signals, `hasLockfile`, and interpreter source. Side-effect free and
total.
- **Emit** (`telemetry/packageManagerExtensions.ts`): adds
`recordPackageManagerDetection` to the existing `Telemetry` class via
the same `declare module` pattern as `commandExtensions.ts`. Keeps
disk/Python-extension dependencies out of the telemetry client.
- **Collection** (`PackageManagerTelemetry.ts`): a best-effort,
non-blocking collector that reads disk and already-resolved interpreter
metadata, runs the pure classifier, and calls the emit method.
Deduplicated per session on `(trigger, projectRoot)`; any failure
degrades to `unknown` and is swallowed so it never disrupts setup.

Emission is wired into three setup touchpoints: project-open environment
check (`auto_open`), the set-up-environment command
(`explicit_command`), and first Run/Debug with Databricks Connect
(`run`/`debug`).

A new `Events.PYTHON_ENV_SETUP_DETECTED` event carries a typed,
documented schema (reuses the existing telemetry transport; opt-out
honored; categorical data only — no paths, package names, or cluster
names). A handoff note for the analytics/dashboard owner is included at
`src/telemetry/PACKAGE_MANAGER_DETECTION.md`.

**Detection correctness** (the parts most worth reviewing):
- `interpreterSource` is derived from the active interpreter alone,
never from project files. A `uv.lock` project running a
conda/venv/system interpreter reports that interpreter's real source,
keeping the "uv project, interpreter not uv-managed yet" setup-flow gap
visible. A genuinely uv-provisioned venv is identified by the `uv =`
marker in `pyvenv.cfg`, not by `uv.lock`.
- conda is attributed only when the active interpreter resides under
`CONDA_PREFIX` (path-boundary checked), not on the bare env var — which
is session-global in the extension host (launching VS Code from an
activated conda shell) and would otherwise over-count conda for
uv/poetry/pip projects.
- `pyproject` `[tool.uv]`/`[tool.poetry]` detection uses a bounded
table-header scan, not substring matching: ignores comments and in-value
mentions, rejects prefix collisions (e.g. `tool.uvicorn`), and matches
subtable and array-of-table headers (`[tool.uv.sources]`,
`[[tool.poetry.source]]`).
- No external executable is run for telemetry: the uv-on-PATH probe was
removed (it spawned a PATH-resolved `uv` for a weak, non-attributing
signal). Detection reads only disk and already-resolved interpreter
metadata.

**Scope / privacy:** measurement only — no changes to setup behavior
(the VPEX flows are a separate effort). Only enum/categorical data and a
closed set of signal identifiers are emitted; the existing telemetry
opt-out (`telemetry.telemetryLevel`) is respected by the transport.

## Tests

- [x] `yarn run test:unit`: 202 passing, 0 failing — includes the pure
classifier (each manager, interpreter sources, overlaps like uv+pip /
conda+pip / poetry+uv, weak signals, none) and pure helpers
(`pyprojectHasToolSection`, `pyvenvCfgMarksUv`,
`interpreterUnderCondaPrefix`), covering the conda-prefix boundary and
shell-global false-positive cases.
- [x] `yarn run build` (typecheck) passes.
- [x] `eslint` clean; `prettier` formatted.

Reviewer can validate with:

```bash
cd packages/databricks-vscode
yarn run build
yarn run test:unit
npx eslint src --ext ts && npx prettier . -c
```

Author: rugpanov

packages/databricks-vscode/src/extension.ts

@@ -75,6 +75,7 @@ import {BundleVariableTreeDataProvider} from "./ui/bundle-variables/BundleVariab
 import {ConfigurationTreeViewManager} from "./ui/configuration-view/ConfigurationTreeViewManager";
 import {getCLIDependenciesEnvVars} from "./utils/envVarGenerators";
 import {EnvironmentCommands} from "./language/EnvironmentCommands";
+import {PackageManagerTelemetry} from "./language/PackageManagerTelemetry";
 import {WorkspaceFolderManager} from "./vscode-objs/WorkspaceFolderManager";
 import {SyncCommands} from "./sync/SyncCommands";
 import {CodeSynchronizer} from "./sync";
@@ -335,6 +336,24 @@ export async function activate(
         customWhenContext,
         telemetry
     );
+    const packageManagerTelemetry = new PackageManagerTelemetry(
+        telemetry,
+        pythonExtensionWrapper,
+        () => {
+            try {
+                return workspaceFolderManager.activeProjectUri.fsPath;
+            } catch (e) {
+                return undefined;
+            }
+        },
+        () => {
+            if (connectionManager.serverless) {
+                return "serverless";
+            }
+            return connectionManager.cluster ? "cluster" : "none";
+        },
+        () => connectionManager.state === "CONNECTED"
+    );
     context.subscriptions.push(
         bundleFileWatcher,
         bundleValidateModel,
@@ -619,13 +638,15 @@ export async function activate(
                 connectionManager,
                 pythonExtensionWrapper,
                 environmentDependenciesInstaller,
-                configureAutocomplete
+                configureAutocomplete,
+                packageManagerTelemetry
             )
     );
     const environmentCommands = new EnvironmentCommands(
         featureManager,
         pythonExtensionWrapper,
-        environmentDependenciesInstaller
+        environmentDependenciesInstaller,
+        packageManagerTelemetry
     );
     context.subscriptions.push(
         telemetry.registerCommand(
@@ -1003,7 +1024,8 @@ export async function activate(
         featureManager,
         context,
         customWhenContext,
-        telemetry
+        telemetry,
+        packageManagerTelemetry
     );
     const debugFactory = new DatabricksDebugAdapterFactory(
         connectionManager,

packages/databricks-vscode/src/language/EnvironmentCommands.ts

@@ -5,16 +5,19 @@ import {Cluster} from "../sdk-extensions";
 import {EnvironmentDependenciesInstaller} from "./EnvironmentDependenciesInstaller";
 import {Environment} from "./MsPythonExtensionApi";
 import {environmentName} from "../utils/environmentUtils";
+import {PackageManagerTelemetry} from "./PackageManagerTelemetry";
 
 export class EnvironmentCommands {
     constructor(
         private featureManager: FeatureManager,
         private pythonExtension: MsPythonExtensionWrapper,
-        private installer: EnvironmentDependenciesInstaller
+        private installer: EnvironmentDependenciesInstaller,
+        private packageManagerTelemetry: PackageManagerTelemetry
     ) {}
 
     async setup(stepId?: string) {
         commands.executeCommand("configurationView.focus");
+        void this.packageManagerTelemetry.emitDetection("explicit_command");
         await window.withProgress(
             {location: {viewId: "configurationView"}},
             () => this._setup(stepId)

packages/databricks-vscode/src/language/EnvironmentDependenciesVerifier.ts

@@ -10,6 +10,7 @@ import {ResolvedEnvironment} from "./MsPythonExtensionApi";
 import {NamedLogger} from "@databricks/sdk-experimental/dist/logging";
 import {ConfigureAutocomplete} from "./ConfigureAutocomplete";
 import {workspaceConfigs} from "../vscode-objs/WorkspaceConfigs";
+import {PackageManagerTelemetry} from "./PackageManagerTelemetry";
 
 export class EnvironmentDependenciesVerifier extends MultiStepAccessVerifier {
     private readonly logger = NamedLogger.getOrCreate(Loggers.Extension);
@@ -18,7 +19,8 @@ export class EnvironmentDependenciesVerifier extends MultiStepAccessVerifier {
         private readonly connectionManager: ConnectionManager,
         private readonly pythonExtension: MsPythonExtensionWrapper,
         private readonly installer: EnvironmentDependenciesInstaller,
-        private readonly configureAutocomplete: ConfigureAutocomplete
+        private readonly configureAutocomplete: ConfigureAutocomplete,
+        private readonly packageManagerTelemetry: PackageManagerTelemetry
     ) {
         super([
             "checkCluster",
@@ -404,6 +406,10 @@ export class EnvironmentDependenciesVerifier extends MultiStepAccessVerifier {
 
     override async check() {
         await this.connectionManager.waitForConnect();
+        // Emit package-manager detection only once connected (waitForConnect
+        // resolves on CONNECTED), so unauthenticated sessions are not reported.
+        // Deduplicated per session; never throws.
+        void this.packageManagerTelemetry.emitDetection("auto_open");
         await Promise.all([
             this.checkCluster(this.connectionManager.cluster),
             this.checkWorkspaceHasUc(),

packages/databricks-vscode/src/language/PackageManagerTelemetry.test.ts

@@ -0,0 +1,209 @@
+import {expect} from "chai";
+import * as tmp from "tmp";
+import path from "node:path";
+import {writeFileSync} from "node:fs";
+import {Telemetry} from "../telemetry";
+import {MsPythonExtensionWrapper} from "./MsPythonExtensionWrapper";
+import {PackageManagerTelemetry, SetupTrigger} from "./PackageManagerTelemetry";
+
+type RecordedEvent = {
+    name: string;
+    props: Record<string, string>;
+    metrics: Record<string, number>;
+};
+
+/** A Telemetry backed by a fake reporter that captures sent events. */
+function makeTelemetry(level: "all" | "error" | "crash" | "off" = "all") {
+    const events: RecordedEvent[] = [];
+    const reporter = {
+        telemetryLevel: level,
+        sendTelemetryEvent: (
+            name: string,
+            props?: Record<string, string>,
+            metrics?: Record<string, number>
+        ) => {
+            events.push({name, props: props ?? {}, metrics: metrics ?? {}});
+        },
+        sendTelemetryErrorEvent: () => {},
+        sendDangerousTelemetryEvent: () => {},
+        sendDangerousTelemetryErrorEvent: () => {},
+        dispose: () => Promise.resolve(),
+    };
+    return {telemetry: new Telemetry(reporter as any), events};
+}
+
+describe(__filename, () => {
+    const cleanups: Array<() => void> = [];
+
+    afterEach(() => {
+        while (cleanups.length) {
+            cleanups.pop()!();
+        }
+    });
+
+    /**
+     * Create a throwaway project dir populated with the given files, passed as
+     * [name, contents] tuples (file names aren't valid identifiers, so a tuple
+     * list avoids object-literal key lint noise).
+     */
+    function makeProject(files: Array<[string, string]>): string {
+        const dir = tmp.dirSync({unsafeCleanup: true});
+        cleanups.push(dir.removeCallback);
+        for (const [name, contents] of files) {
+            writeFileSync(path.join(dir.name, name), contents);
+        }
+        return dir.name;
+    }
+
+    // Interpreter is irrelevant to these disk-signal tests; report none.
+    const noInterpreter = {
+        get pythonEnvironment() {
+            return Promise.resolve(undefined);
+        },
+    } as unknown as MsPythonExtensionWrapper;
+
+    function makePmt(
+        telemetry: Telemetry,
+        opts: {
+            projectRoot: string;
+            compute?: "cluster" | "serverless" | "none";
+            connected?: boolean;
+        }
+    ) {
+        return new PackageManagerTelemetry(
+            telemetry,
+            noInterpreter,
+            () => opts.projectRoot,
+            () => opts.compute ?? "none",
+            () => opts.connected ?? true
+        );
+    }
+
+    const emit = async (pmt: PackageManagerTelemetry, t: SetupTrigger) =>
+        pmt.emitDetection(t);
+
+    it("emits a detection event for a connected project (uv + pip)", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        const projectRoot = makeProject([
+            ["uv.lock", "version = 1\n"],
+            ["pyproject.toml", "[project]\nname='x'\n[tool.uv]\n"],
+            ["requirements-dev.txt", "requests\n"],
+        ]);
+        const pmt = makePmt(telemetry, {projectRoot, compute: "cluster"});
+
+        await emit(pmt, "explicit_command");
+
+        expect(events).to.have.length(1);
+        const e = events[0];
+        expect(e.name).to.equal("python_env.setup.detected");
+        expect(e.props["event.primaryManager"]).to.equal("uv");
+        expect(e.props["event.managersDetected"]).to.equal('["uv","pip"]');
+        expect(e.props["event.hasLockfile"]).to.equal("true");
+        expect(e.props["event.targetCompute"]).to.equal("cluster");
+        expect(e.props["event.setupTrigger"]).to.equal("explicit_command");
+        expect(e.props["event.interpreterSource"]).to.equal("unknown");
+    });
+
+    it("deduplicates per (trigger, projectRoot) within a session", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        const projectRoot = makeProject([["uv.lock", "version = 1\n"]]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+        await emit(pmt, "auto_open");
+
+        expect(events).to.have.length(1);
+    });
+
+    it("does not emit while disconnected, and does not burn the dedupe slot", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        const projectRoot = makeProject([["uv.lock", "version = 1\n"]]);
+
+        const disconnected = makePmt(telemetry, {
+            projectRoot,
+            connected: false,
+        });
+        await emit(disconnected, "auto_open");
+        expect(events).to.have.length(0);
+
+        // A later connected emit for the same (trigger, project) still fires --
+        // i.e. the disconnected attempt did not consume the dedupe key.
+        const connected = makePmt(telemetry, {projectRoot, connected: true});
+        await emit(connected, "auto_open");
+        expect(events).to.have.length(1);
+    });
+
+    it("does not emit when telemetry is disabled", async () => {
+        const {telemetry, events} = makeTelemetry("error");
+        const projectRoot = makeProject([["uv.lock", "version = 1\n"]]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+
+        expect(events).to.have.length(0);
+    });
+
+    it("reports unknown for a project with no recognizable signals", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        // `requirementsfoo.txt` (no separator) is NOT a requirements file, so
+        // pip must not be attributed.
+        const projectRoot = makeProject([["requirementsfoo.txt", "x\n"]]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+
+        expect(events).to.have.length(1);
+        expect(events[0].props["event.managersDetected"]).to.equal("[]");
+        expect(events[0].props["event.primaryManager"]).to.equal("unknown");
+    });
+
+    it("attributes pip from a separator-suffixed requirements file", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        const projectRoot = makeProject([
+            ["requirements_test.txt", "pytest\n"],
+        ]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+
+        expect(events[0].props["event.managersDetected"]).to.equal('["pip"]');
+        expect(events[0].props["event.primaryManager"]).to.equal("pip");
+    });
+
+    it("does not attribute pip for a tool-only pyproject", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        // Only linter config, no [project]/[build-system] -- not a pip signal.
+        const projectRoot = makeProject([
+            ["pyproject.toml", "[tool.ruff]\nline-length = 88\n"],
+        ]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+
+        expect(events[0].props["event.managersDetected"]).to.equal("[]");
+        expect(events[0].props["event.primaryManager"]).to.equal("unknown");
+    });
+
+    it("attributes pip for a pyproject with [project] and no uv/poetry", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        const projectRoot = makeProject([
+            ["pyproject.toml", '[project]\nname = "x"\n'],
+        ]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+
+        expect(events[0].props["event.managersDetected"]).to.equal('["pip"]');
+    });
+
+    it("omits pythonVersion from the event when the interpreter is unknown", async () => {
+        const {telemetry, events} = makeTelemetry("all");
+        const projectRoot = makeProject([["uv.lock", "version = 1\n"]]);
+        const pmt = makePmt(telemetry, {projectRoot});
+
+        await emit(pmt, "auto_open");
+
+        // The key must be absent, not the string "undefined".
+        expect(events[0].props).to.not.have.property("event.pythonVersion");
+    });
+});