Fix testing perf telemetry: send measurements via measures arg (microsoft#26028)

## Problem

The discovery/run testing telemetry added in microsoft#25980 emitted its numeric
performance fields — `totalDurationMs`, `testCount`
(`UNITTEST.DISCOVERY.DONE`) and `durationMs`, `requestedCount`
(`UNITTEST.RUN.DONE`) — in the **properties** bag instead of the
**measures** argument of `sendTelemetryEvent`.

These fields are annotated `isMeasurement: true`, and telemetry
ingestion drops measurement-annotated fields that arrive as properties.
Verified against production telemetry: across ~46M
`unittest.discovery.done` events in the last 14 days, **zero** carried
`totalDurationMs` or `testCount` (in either `Properties` or `Measures`),
while the string categoricals (`mode`, `trigger`, `failureCategory`)
came through fine. The `mode`/`trigger` fields survive because they're
strings; the numeric ones silently vanish.

This means there is currently **no duration signal** to evaluate the
recent pytest discovery performance work (microsoft#25974, microsoft#25982) — the
measurements were never actually recorded.

The correct pattern is confirmed by `NATIVE_FINDER_PERF`, which passes
its numbers via the measures (2nd) argument and lands them cleanly in
the `Measures` column.

## Fix

Move the numeric measurement fields to the `measures` (2nd) argument of
`sendTelemetryEvent` at all five emit sites:

- `resultResolver.ts` — `DISCOVERY.DONE` success path
(`totalDurationMs`, `testCount`)
- `controller.ts` — `DISCOVERY.DONE` project-error path
(`totalDurationMs`) and `RUN.DONE` legacy path (`durationMs`,
`requestedCount`)
- `workspaceTestAdapter.ts` — `DISCOVERY.DONE` legacy-error path
(`totalDurationMs`)
- `projectTestExecution.ts` — `RUN.DONE` project path (`durationMs`,
`requestedCount`)

Also removed these fields from the event **property** type definitions
(the `__GDPR__` annotations are unchanged and already correctly mark
them `isMeasurement: true`) and added notes so they aren't reintroduced
as properties.

## Verification

- `tsc`: no new type errors from these changes.
- `eslint` + `prettier`: clean on all changed files.
- Unit tests pass: `resolveDiscovery` (5), project `RUN.DONE` telemetry
(2), and workspace discovery (5).

## Impact

Once a build with this fix ships,
`totalDurationMs`/`testCount`/`durationMs`/`requestedCount` will
populate in the `Measures` column, enabling the before/after
discovery-performance comparison (sliced by `testCount` bucket ×
`mode`). No user-facing behavior change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: eleanorjboyd

src/client/telemetry/index.ts

@@ -2209,14 +2209,10 @@ export interface IEventNamePropertyMapping {
             | 'env-resolution'
             | 'cancelled'
             | 'unknown';
-        /**
-         * Wall-clock duration of the discovery cycle in milliseconds.
-         */
-        totalDurationMs?: number;
-        /**
-         * Number of test items discovered (leaf nodes).
-         */
-        testCount?: number;
+        // NOTE: `totalDurationMs` and `testCount` are measurements (see the __GDPR__ block
+        // above) and MUST be passed via the `measures` argument of sendTelemetryEvent, not
+        // as properties. Fields annotated `isMeasurement: true` are dropped by telemetry
+        // ingestion when sent in the properties bag.
     };
     /**
      * Telemetry event sent when cancelling discovering tests
@@ -2286,14 +2282,10 @@ export interface IEventNamePropertyMapping {
          * Coarse failure category when `failed` is true.
          */
         failureCategory?: UnitTestRunFailureCategory;
-        /**
-         * Wall-clock duration of the run in milliseconds.
-         */
-        durationMs?: number;
-        /**
-         * Number of test items the user asked to run.
-         */
-        requestedCount?: number;
+        // NOTE: `durationMs` and `requestedCount` are measurements (see the __GDPR__ block
+        // above) and MUST be passed via the `measures` argument of sendTelemetryEvent, not
+        // as properties. Fields annotated `isMeasurement: true` are dropped by telemetry
+        // ingestion when sent in the properties bag.
     };
     /**
      * Telemetry event sent when testing is disabled for a workspace.

src/client/testing/testController/common/projectTestExecution.ts

@@ -85,15 +85,17 @@ export async function executeTestsForProjects(
                 traceError(`[test-by-project] Execution failed for project ${project.projectName}:`, error);
             }
         } finally {
-            sendTelemetryEvent(EventName.UNITTEST_RUN_DONE, undefined, {
-                tool: project.testProvider,
-                debugging: isDebugMode,
-                mode: 'project',
-                failed,
-                failureCategory,
-                durationMs: stopWatch.elapsedTime,
-                requestedCount: items.length,
-            });
+            sendTelemetryEvent(
+                EventName.UNITTEST_RUN_DONE,
+                { durationMs: stopWatch.elapsedTime, requestedCount: items.length },
+                {
+                    tool: project.testProvider,
+                    debugging: isDebugMode,
+                    mode: 'project',
+                    failed,
+                    failureCategory,
+                },
+            );
         }
     });
 

src/client/testing/testController/common/resultResolver.ts

@@ -93,14 +93,20 @@ export class PythonResultResolver implements ITestResultResolver {
         const cycle = this.discoveryTelemetry.complete();
         const mode = cycle?.mode ?? this.discoveryTelemetry.defaultMode;
         const failed = payload?.status === 'error';
-        sendTelemetryEvent(EventName.UNITTEST_DISCOVERY_DONE, undefined, {
+        // Numeric fields must be sent as measures (2nd arg), not properties. Fields
+        // annotated `isMeasurement: true` are dropped by telemetry ingestion when they
+        // arrive in the properties bag, so totalDurationMs/testCount would otherwise never
+        // be recorded.
+        const measures: Record<string, number> = { testCount };
+        if (cycle?.stopWatch.elapsedTime !== undefined) {
+            measures.totalDurationMs = cycle.stopWatch.elapsedTime;
+        }
+        sendTelemetryEvent(EventName.UNITTEST_DISCOVERY_DONE, measures, {
             tool: this.testProvider,
             failed,
             mode,
             trigger: cycle?.trigger,
             failureCategory: failed ? (token?.isCancellationRequested ? 'cancelled' : 'unknown') : undefined,
-            totalDurationMs: cycle?.stopWatch.elapsedTime,
-            testCount,
         });
     }
 

src/client/testing/testController/controller.ts

@@ -716,13 +716,16 @@ export class PythonTestController implements ITestController, IExtensionSingleAc
             // Individual project failures don't block others
             projectsCompleted.add(project.projectUri.toString()); // Still mark as completed
             const cycle = (project.resultResolver as Partial<PythonResultResolver>).discoveryTelemetry?.complete();
-            sendTelemetryEvent(EventName.UNITTEST_DISCOVERY_DONE, undefined, {
+            const measures: Record<string, number> = {};
+            if (cycle?.stopWatch.elapsedTime !== undefined) {
+                measures.totalDurationMs = cycle.stopWatch.elapsedTime;
+            }
+            sendTelemetryEvent(EventName.UNITTEST_DISCOVERY_DONE, measures, {
                 tool: project.testProvider,
                 failed: true,
                 mode: 'project',
                 trigger: cycle?.trigger ?? this.currentDiscoveryTrigger,
                 failureCategory: this.refreshCancellation.token.isCancellationRequested ? 'cancelled' : 'unknown',
-                totalDurationMs: cycle?.stopWatch.elapsedTime,
             });
         } finally {
             project.isDiscovering = false;
@@ -1048,15 +1051,17 @@ export class PythonTestController implements ITestController, IExtensionSingleAc
             failureCategory = token.isCancellationRequested ? 'cancelled' : 'unknown';
             throw ex;
         } finally {
-            sendTelemetryEvent(EventName.UNITTEST_RUN_DONE, undefined, {
-                tool: provider,
-                debugging,
-                mode: 'legacy',
-                failed,
-                failureCategory,
-                durationMs: stopWatch.elapsedTime,
-                requestedCount: testItems.length,
-            });
+            sendTelemetryEvent(
+                EventName.UNITTEST_RUN_DONE,
+                { durationMs: stopWatch.elapsedTime, requestedCount: testItems.length },
+                {
+                    tool: provider,
+                    debugging,
+                    mode: 'legacy',
+                    failed,
+                    failureCategory,
+                },
+            );
         }
     }
 

src/client/testing/testController/workspaceTestAdapter.ts

@@ -153,14 +153,17 @@ export class WorkspaceTestAdapter {
             deferred.resolve();
         } catch (ex) {
             const cycle = (this.resultResolver as Partial<PythonResultResolver>).discoveryTelemetry?.complete();
-            sendTelemetryEvent(EventName.UNITTEST_DISCOVERY_DONE, undefined, {
-                tool: this.testProvider,
-                failed: true,
-                mode: 'legacy',
-                trigger: cycle?.trigger ?? trigger,
-                failureCategory: token?.isCancellationRequested ? 'cancelled' : 'unknown',
-                totalDurationMs: cycle?.stopWatch.elapsedTime ?? stopWatch.elapsedTime,
-            });
+            sendTelemetryEvent(
+                EventName.UNITTEST_DISCOVERY_DONE,
+                { totalDurationMs: cycle?.stopWatch.elapsedTime ?? stopWatch.elapsedTime },
+                {
+                    tool: this.testProvider,
+                    failed: true,
+                    mode: 'legacy',
+                    trigger: cycle?.trigger ?? trigger,
+                    failureCategory: token?.isCancellationRequested ? 'cancelled' : 'unknown',
+                },
+            );
 
             let cancel = token?.isCancellationRequested
                 ? Testing.cancelUnittestDiscovery

src/test/testing/testController/resultResolver.unit.test.ts

@@ -155,7 +155,7 @@ suite('Result Resolver tests', () => {
 
             resultResolver.resolveDiscovery(payload, cancelationToken);
 
-            sinon.assert.calledWithMatch(sendTelemetryStub, EventName.UNITTEST_DISCOVERY_DONE, undefined, {
+            sinon.assert.calledWithMatch(sendTelemetryStub, EventName.UNITTEST_DISCOVERY_DONE, sinon.match.any, {
                 failed: true,
                 failureCategory: 'unknown',
             });