test(profiler): fix Profiler telemetry flake (#8114)

* test(profiler): fix Profiler telemetry flake

The integration test `profiler > Profiler telemetry > sends profiler API
telemetry` accounts for the bulk of recent Profiling flakes. Two issues:

1. `assertTelemetryReceived` in fake-agent.js filters only by
   `request_type`, but the tracer's NamespaceManager emits a
   `generate-metrics` payload per namespace (`tracers`, `profilers`,
   `appsec`, ...). Tracer-namespace messages tripped the inner
   `assert.strictEqual(pp.namespace, 'profilers')` check, polluting
   the collected `errors[]` with confusing `'tracers' !== 'profilers'`
   noise on any failure.

2. The `requestCount <= 3` upper bound was off-by-one: with
   DD_PROFILING_UPLOAD_PERIOD=1 and TEST_DURATION_MS=2500, the
   periodic-vs-shutdown race the comment already calls out can yield
   4 uploads.

Add an optional `namespace` parameter to `assertTelemetryReceived` that
filters at the same point as `requestType` (backward-compatible, no
existing callers pass a 6th argument). Use it from the three profiler
telemetry assertions and bump the bound to `<= 4`.

* refactor(test): convert assertTelemetryReceived to a single options arg

The signature had grown to six positional parameters with a built-in
overload that shifted them when `fn` was omitted, which was hard to
read and easy to get wrong. Convert to a single options object with
defaulted properties (`fn`, `requestType`, `timeout`,
`expectedMessageCount`, `resolveAtFirstSuccess`, `namespace`) and
update all 16 call sites.

No behavior change; only the call shape.

* fix(test): preserve never-resolve semantics for SSI-absent telemetry check

The `records profile on process exit` test wraps a telemetry assertion
in `expectTimeout`, which requires the inner promise to reject with a
timeout. The original 2-arg call relied on a quirk in the helper's
positional overload that left `expectedMessageCount` at 30000, making
the promise effectively unresolvable.

The options-object refactor restored the documented default of 1, so
the first matching `generate-metrics` message resolved the promise and
broke the test. Make the intent explicit with `expectedMessageCount:
Infinity` and a comment explaining why.

Author: szegedi

integration-tests/appsec/endpoints-collection.spec.js

@@ -138,21 +138,26 @@ describe('Endpoints collection', () => {
 
       const expectedMessageCount = framework === 'express' ? 7 : 4
 
-      const telemetryPromise = agent.assertTelemetryReceived(({ payload }) => {
-        isFirstFlags.push(Boolean(payload.payload.is_first))
-
-        if (payload.payload.endpoints) {
-          payload.payload.endpoints.forEach(endpoint => {
-            endpointsFound.push({
-              method: endpoint.method,
-              path: endpoint.path,
-              type: endpoint.type,
-              operation_name: endpoint.operation_name,
-              resource_name: endpoint.resource_name,
+      const telemetryPromise = agent.assertTelemetryReceived({
+        fn: ({ payload }) => {
+          isFirstFlags.push(Boolean(payload.payload.is_first))
+
+          if (payload.payload.endpoints) {
+            payload.payload.endpoints.forEach(endpoint => {
+              endpointsFound.push({
+                method: endpoint.method,
+                path: endpoint.path,
+                type: endpoint.type,
+                operation_name: endpoint.operation_name,
+                resource_name: endpoint.resource_name,
+              })
             })
-          })
-        }
-      }, 'app-endpoints', 5_000, expectedMessageCount)
+          }
+        },
+        requestType: 'app-endpoints',
+        timeout: 5_000,
+        expectedMessageCount,
+      })
 
       proc = await spawnProc(appFile, {
         cwd,

integration-tests/bun/bun.spec.js

@@ -106,7 +106,7 @@ describe('Bun runtime smoke tests', function () {
   })
 
   it('should emit app-started telemetry under Bun', async () => {
-    const telemetryPromise = agent.assertTelemetryReceived('app-started', 20_000, 1)
+    const telemetryPromise = agent.assertTelemetryReceived({ requestType: 'app-started', timeout: 20_000 })
 
     const [bunResult] = await Promise.all([
       runBun('bun/init-telemetry.js', {

integration-tests/helpers/fake-agent.js

@@ -213,30 +213,25 @@ module.exports = class FakeAgent extends EventEmitter {
   /**
    * Assert that a telemetry message is received.
    *
-   * @overload
-   * @param {string} requestType - The request type to assert.
-   * @param {number} [timeout=30_000] - The timeout in milliseconds.
-   * @param {number} [expectedMessageCount=1] - The number of messages to expect.
-   * @returns {Promise<void>} A promise that resolves when the telemetry message of type `requestType` is received.
+   * @param {object} options
+   * @param {Function} [options.fn] - Function called with each matching telemetry message. If it throws,
+   *     the error is collected and the listener stays alive for the next message. Defaults to a no-op.
+   * @param {string} options.requestType - The telemetry request type to match.
+   * @param {number} [options.timeout=30_000] - Timeout in milliseconds before the promise rejects.
+   * @param {number} [options.expectedMessageCount=1] - Number of matching messages to wait for.
+   * @param {boolean} [options.resolveAtFirstSuccess=false] - Resolve as soon as `fn` first runs without throwing.
+   * @param {string} [options.namespace] - If set, only consider messages whose payload namespace equals this value.
+   * @returns {Promise<void>} A promise that resolves when the expected telemetry messages are received and `fn` has
+   *     run successfully. If `fn` throws on every matching message, the promise rejects once `timeout` is reached.
    */
-  /**
-   * @overload
-   * @param {Function} fn - The function to call with the telemetry message of type `requestType`.
-   * @param {string} requestType - The request type to assert.
-   * @param {number} [timeout=30_000] - The timeout in milliseconds.
-   * @param {number} [expectedMessageCount=1] - The number of messages to expect.
-   * @returns {Promise<void>} A promise that resolves when the telemetry message of type `requestType` is received and
-   *     the function `fn` has finished running. If `fn` throws an error, the promise will be rejected once `timeout`
-   *     is reached.
-   */
-  assertTelemetryReceived (fn, requestType, timeout = 30_000, expectedMessageCount = 1, resolveAtFirstSuccess = false) {
-    if (typeof fn !== 'function') {
-      expectedMessageCount = timeout
-      timeout = requestType
-      requestType = fn
-      fn = noop
-    }
-
+  assertTelemetryReceived ({
+    fn = noop,
+    requestType,
+    timeout = 30_000,
+    expectedMessageCount = 1,
+    resolveAtFirstSuccess = false,
+    namespace,
+  }) {
     let resultResolve
     let resultReject
     let msgCount = 0
@@ -260,6 +255,7 @@ module.exports = class FakeAgent extends EventEmitter {
 
     const messageHandler = msg => {
       if (msg.payload.request_type !== requestType) return
+      if (namespace !== undefined && msg.payload.payload?.namespace !== namespace) return
       msgCount += 1
       try {
         fn(msg)

integration-tests/opentelemetry.spec.js

@@ -9,7 +9,7 @@ const { FakeAgent, sandboxCwd, useSandbox, stopProc } = require('./helpers')
 
 async function check (agent, proc, timeout, onMessage = () => { }, isMetrics) {
   const messageReceiver = isMetrics
-    ? agent.assertTelemetryReceived(onMessage, 'generate-metrics', timeout)
+    ? agent.assertTelemetryReceived({ fn: onMessage, requestType: 'generate-metrics', timeout })
     : agent.assertMessageReceived(onMessage, timeout)
 
   const [res] = await Promise.all([

integration-tests/profiler/profiler.spec.js

@@ -579,9 +579,14 @@ describe('profiler', () => {
           DD_PROFILING_ENABLED: '1',
         },
       })
-      const checkTelemetry = agent.assertTelemetryReceived('generate-metrics', 1000)
       // SSI telemetry is not supposed to have been emitted when DD_INJECTION_ENABLED is absent,
-      // so expect telemetry callback to time out
+      // so expect telemetry callback to time out. expectedMessageCount: Infinity keeps the listener
+      // from ever resolving, so the only outcome is the timeout that expectTimeout requires.
+      const checkTelemetry = agent.assertTelemetryReceived({
+        requestType: 'generate-metrics',
+        timeout: 1000,
+        expectedMessageCount: Infinity,
+      })
       await Promise.all([checkProfiles(agent, proc, timeout), expectTimeout(checkTelemetry)])
     })
 
@@ -703,38 +708,47 @@ describe('profiler', () => {
       let requestCount = 0
       let pointsCount = 0
 
-      const checkMetrics = agent.assertTelemetryReceived(({ _, payload }) => {
-        const pp = payload.payload
-        assert.strictEqual(pp.namespace, 'profilers')
-        const series = pp.series
-        const requests = series.find(s => s.metric === 'profile_api.requests')
-        assert.strictEqual(requests.type, 'count')
-        // There's a race between metrics and on-shutdown profile, so metric
-        // value will be between 1 and 3
-        requestCount = requests.points[0][1]
-        assert.ok(requestCount >= 1)
-        assert.ok(requestCount <= 3)
-
-        const responses = series.find(s => s.metric === 'profile_api.responses')
-        assert.strictEqual(responses.type, 'count')
-        assert.deepStrictEqual(responses.tags, ['status_code:200'])
-
-        // Same number of requests and responses
-        assert.strictEqual(responses.points[0][1], requestCount)
-      }, 'generate-metrics', timeout, 1, true)
-
-      const checkDistributions = agent.assertTelemetryReceived(({ _, payload }) => {
-        const pp = payload.payload
-        assert.strictEqual(pp.namespace, 'profilers')
-        const series = pp.series
-        assert.strictEqual(series.length, 2)
-        assert.strictEqual(series[0].metric, 'profile_api.bytes')
-        assert.strictEqual(series[1].metric, 'profile_api.ms')
-
-        // Same number of points
-        pointsCount = series[0].points.length
-        assert.strictEqual(pointsCount, series[1].points.length)
-      }, 'distributions', timeout)
+      const checkMetrics = agent.assertTelemetryReceived({
+        fn: ({ _, payload }) => {
+          const pp = payload.payload
+          const series = pp.series
+          const requests = series.find(s => s.metric === 'profile_api.requests')
+          assert.strictEqual(requests.type, 'count')
+          // There's a race between the periodic uploader and the on-shutdown
+          // upload, so the count can include up to one extra request.
+          requestCount = requests.points[0][1]
+          assert.ok(requestCount >= 1)
+          assert.ok(requestCount <= 4)
+
+          const responses = series.find(s => s.metric === 'profile_api.responses')
+          assert.strictEqual(responses.type, 'count')
+          assert.deepStrictEqual(responses.tags, ['status_code:200'])
+
+          // Same number of requests and responses
+          assert.strictEqual(responses.points[0][1], requestCount)
+        },
+        requestType: 'generate-metrics',
+        timeout,
+        resolveAtFirstSuccess: true,
+        namespace: 'profilers',
+      })
+
+      const checkDistributions = agent.assertTelemetryReceived({
+        fn: ({ _, payload }) => {
+          const pp = payload.payload
+          const series = pp.series
+          assert.strictEqual(series.length, 2)
+          assert.strictEqual(series[0].metric, 'profile_api.bytes')
+          assert.strictEqual(series[1].metric, 'profile_api.ms')
+
+          // Same number of points
+          pointsCount = series[0].points.length
+          assert.strictEqual(pointsCount, series[1].points.length)
+        },
+        requestType: 'distributions',
+        timeout,
+        namespace: 'profilers',
+      })
 
       await Promise.all([checkProfiles(agent, proc, timeout), checkMetrics, checkDistributions])
 
@@ -761,16 +775,21 @@ describe('profiler', () => {
         },
       })
 
-      const checkMetrics = agent.assertTelemetryReceived(({ _, payload }) => {
-        const pp = payload.payload
-        assert.strictEqual(pp.namespace, 'profilers');
-        ['live', 'used'].forEach(metricName => {
-          const sampleContexts = pp.series.find(s => s.metric === `wall.async_contexts_${metricName}`)
-          assert.notStrictEqual(sampleContexts, undefined)
-          assert.strictEqual(sampleContexts.type, 'gauge')
-          assert.ok(sampleContexts.points[0][1] >= 1)
-        })
-      }, 'generate-metrics', timeout, 1, true)
+      const checkMetrics = agent.assertTelemetryReceived({
+        fn: ({ _, payload }) => {
+          const pp = payload.payload;
+          ['live', 'used'].forEach(metricName => {
+            const sampleContexts = pp.series.find(s => s.metric === `wall.async_contexts_${metricName}`)
+            assert.notStrictEqual(sampleContexts, undefined)
+            assert.strictEqual(sampleContexts.type, 'gauge')
+            assert.ok(sampleContexts.points[0][1] >= 1)
+          })
+        },
+        requestType: 'generate-metrics',
+        timeout,
+        resolveAtFirstSuccess: true,
+        namespace: 'profilers',
+      })
 
       await Promise.all([checkProfiles(agent, proc, timeout), checkMetrics])
     })

integration-tests/telemetry.spec.js

@@ -41,36 +41,44 @@ describe('telemetry', () => {
       let ddTraceFound = false
       let importInTheMiddleFound = false
 
-      await agent.assertTelemetryReceived(msg => {
-        const { payload } = msg
+      await agent.assertTelemetryReceived({
+        fn: msg => {
+          const { payload } = msg
 
-        if (payload.request_type === 'app-dependencies-loaded') {
-          if (payload.payload.dependencies) {
-            payload.payload.dependencies.forEach(dependency => {
-              if (dependency.name === 'dd-trace') {
-                ddTraceFound = true
-              }
-              if (dependency.name === 'import-in-the-middle') {
-                importInTheMiddleFound = true
-              }
-            })
+          if (payload.request_type === 'app-dependencies-loaded') {
+            if (payload.payload.dependencies) {
+              payload.payload.dependencies.forEach(dependency => {
+                if (dependency.name === 'dd-trace') {
+                  ddTraceFound = true
+                }
+                if (dependency.name === 'import-in-the-middle') {
+                  importInTheMiddleFound = true
+                }
+              })
+            }
           }
-        }
-      }, 'app-dependencies-loaded', 5_000, 1)
+        },
+        requestType: 'app-dependencies-loaded',
+        timeout: 5_000,
+      })
 
       assert.strictEqual(ddTraceFound, true)
       assert.strictEqual(importInTheMiddleFound, true)
     })
 
     it('Assert configuration chaining data is sent', async () => {
-      await agent.assertTelemetryReceived(msg => {
-        const { configuration } = msg.payload.payload
-        assertObjectContains(configuration, [
-          { name: 'DD_LOGS_INJECTION', value: true, origin: 'default' },
-          { name: 'DD_LOGS_INJECTION', value: true, origin: 'env_var' },
-          { name: 'DD_LOGS_INJECTION', value: false, origin: 'code' },
-        ])
-      }, 'app-started', 5_000, 1)
+      await agent.assertTelemetryReceived({
+        fn: msg => {
+          const { configuration } = msg.payload.payload
+          assertObjectContains(configuration, [
+            { name: 'DD_LOGS_INJECTION', value: true, origin: 'default' },
+            { name: 'DD_LOGS_INJECTION', value: true, origin: 'env_var' },
+            { name: 'DD_LOGS_INJECTION', value: false, origin: 'code' },
+          ])
+        },
+        requestType: 'app-started',
+        timeout: 5_000,
+      })
     })
   })
 })