fix(data-pipeline): allow opting into agent-info wait before first flush by bm1549 · Pull Request #2233 · DataDog/libdatadog

bm1549 · 2026-07-13T17:22:53Z

What does this PR do?

Makes the agent-info /info wait an opt-in production capability, so a tracer can ensure agent-driven policy (client-side stats / p0 dropping) is active before its first trace flush.

TraceExporter::wait_agent_info_ready is no longer #[cfg(feature = "test-utils")]. It is now a public, timeout-bounded primitive, and it sleeps via SleepCapability instead of tokio::time::sleep so it works off a tokio reactor (e.g. wasm), matching the agent-info fetcher.
Export::wait_ready and its initial_trigger invocation are no longer test-utils-only. The default impl is a no-op, so exporters that do not override it behave exactly as before. initial_trigger logs and proceeds on error instead of unwrapping, so a slow or unavailable agent never blocks or panics the export loop.
DefaultExport drops its test-utils-only wait_ready override (dead scaffolding) and documents that consumers needing the wait override Export::wait_ready.

Default behavior is unchanged: nothing waits for /info unless a caller opts in.

Motivation

Client-side stats and p0 dropping only turn on after the async /info fetch confirms the agent supports them: the exporter starts in StatsComputationStatus::DisabledByAgent and flips to Enabled from the background fetcher. Any trace flushed during that startup window is forwarded to the agent undropped (carrying its negative sampling priority) instead of being dropped client-side.

The only mechanism that made the first flush wait for /info was gated behind test-utils, so production builds could not close the race. This is the mechanism behind the flaky system-test test_entire_trace_dropped_when_dropping_policy_is_active018[parametric-rust] in dd-trace-rs, which forwards one trace instead of dropping it whenever the first span is flushed before /info lands.

Additional Notes

Readiness observes the process-global agent-info cache, which is consistent with the existing check_agent_info behavior on the send path (one agent per process). Making it endpoint-scoped is a separate, pre-existing concern.

Follow-up in a separate dd-trace-rs PR: opt into this wait when client-side stats are enabled, which closes the parametric-rust flake end to end.

How to test the change?

Two new tests in libdd-data-pipeline (trace_exporter, single_threaded_tests group), run with cargo nextest run -p libdd-data-pipeline:

test_p0_dropping_startup_race — with a delayed /info, a p0 trace flushed at startup is forwarded undropped (x-datadog-trace-count: 1); once /info is cached, the same trace is dropped (count: 0).
test_p0_dropping_startup_race_fixed_by_waiting_for_agent_info — opting into wait_agent_info_ready before the first send drops the p0 trace even when it is flushed immediately.

Client-side stats / p0 dropping only activate after the async `/info` fetch confirms agent support: the exporter starts in `DisabledByAgent` and flips to `Enabled` from the background fetcher. Any trace flushed during that startup window is forwarded to the agent undropped instead of being dropped client-side. The only mechanism that made the first flush wait for `/info` was gated behind `#[cfg(feature = "test-utils")]`, so production (and non-test-utils) builds could never close the race. This is the mechanism behind the flaky system-tests `test_entire_trace_dropped_when_dropping_policy_is_active018[parametric-rust]`. Make the wait an opt-in production capability without changing default behavior: - `TraceExporter::wait_agent_info_ready` is no longer test-utils-only; it is a public, timeout-bounded primitive. It sleeps via `SleepCapability` (not `tokio::time::sleep`) so it stays usable off a tokio reactor (e.g. wasm), matching the agent-info fetcher. Default behavior is unchanged (no wait); only callers that opt in pay the bounded cost. - `Export::wait_ready` and its `initial_trigger` invocation are no longer test-utils-only. The default impl is a no-op, so exporters that do not override it behave exactly as before. `initial_trigger` no longer unwraps the result: a failed/timed-out wait is logged and the worker proceeds, never blocking the loop. - `DefaultExport` no longer carries a test-utils-only `wait_ready` override (it was misleading dead scaffolding); it uses the no-op default and documents that consumers needing the wait override `Export::wait_ready`. Add `test_p0_dropping_startup_race` (deterministic repro of the forwarded-during- startup bug) and `test_p0_dropping_startup_race_fixed_by_waiting_for_agent_info` (opt-in wait drops the immediate p0 send), in the `single_threaded_tests` group since they depend on the process-global agent-info cache. All trace_exporter / trace_buffer tests pass under `cargo nextest`. Note: readiness observes the process-global agent-info cache, consistent with the existing `check_agent_info` send-path behavior (one agent per process); making it endpoint-scoped is a separate, pre-existing concern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

github-actions · 2026-07-13T17:24:54Z

📚 Documentation Check Results

⚠️ 1202 documentation warning(s) found

📦 `libdd-data-pipeline` - 1202 warning(s)

Updated: 2026-07-13 17:25:12 UTC | Commit: 5bcd134 | missing-docs job results

github-actions · 2026-07-13T17:26:32Z

Clippy Allow Annotation Report

Tracked Clippy allow annotations changed vs main: ✅ -1 (3 → 2)

Rule	Base	PR	Δ
unwrap_used	3	2	✅ -1

By file and crate

By file

File	Base	PR	Δ
`libdd-data-pipeline/src/trace_buffer/mod.rs`	1	0	✅ -1

By crate

Crate	Base	PR	Δ
`libdd-data-pipeline`	6	5	✅ -1

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. Panic-inducing macros in particular should be avoided. In the future, this report may become a PR-blocking quality gate.

github-actions · 2026-07-13T17:26:37Z

🔒 Cargo Deny Results

⚠️ 3 issue(s) found, showing only errors (advisories, bans, sources)

📦 `libdd-data-pipeline` - 3 error(s)

Show output

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:75:1
   │
75 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:218:1
    │
218 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by @lopopolo) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:287:1
    │
287 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0

advisories FAILED, bans ok, sources ok

Updated: 2026-07-13 17:26:52 UTC | Commit: 5bcd134 | dependency-check job results

datadog-datadog-us1-prod · 2026-07-13T17:37:07Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 99.12%
• Overall Coverage: 74.51% (+0.03%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f9b26b9 | Docs | Datadog PR Page | Give us feedback!}

pr-commenter · 2026-07-13T17:39:49Z

Benchmarks

Comparison

Benchmark execution time: 2026-07-13 17:37:39

Comparing candidate commit f9b26b9 in PR branch brian.marks/p0-drop-startup-race with baseline commit 85ce322 in branch main.

Found 2 performance improvements and 0 performance regressions! Performance is the same for 22 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:trace_buffer/4_senders/no_delay

🟩 execution_time [-218.604µs; -195.271µs] or [-8.687%; -7.760%]
🟩 throughput [+121813.731op/s; +137448.641op/s] or [+8.513%; +9.606%]

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`f9b26b9`	1783963283	brian.marks/p0-drop-startup-race

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
trace_buffer/1_senders/10us_delay	execution_time	56.291ms	56.647ms ± 0.283ms	56.546ms ± 0.088ms	56.717ms	57.280ms	57.521ms	57.683ms	2.01%	1.726	2.514	0.50%	0.020ms	1	200
trace_buffer/1_senders/10us_delay	throughput	15602.453op/s	15888.210op/s ± 78.577op/s	15916.192op/s ± 24.790op/s	15936.444op/s	15961.328op/s	15975.903op/s	15988.322op/s	0.45%	-1.703	2.429	0.49%	5.556op/s	1	200
trace_buffer/1_senders/1us_delay	execution_time	50.023ms	50.089ms ± 0.037ms	50.082ms ± 0.012ms	50.093ms	50.172ms	50.198ms	50.216ms	0.27%	1.258	1.642	0.07%	0.003ms	1	200
trace_buffer/1_senders/1us_delay	throughput	17922.630op/s	17967.921op/s ± 13.298op/s	17970.359op/s ± 4.310op/s	17974.801op/s	17985.877op/s	17989.941op/s	17991.791op/s	0.12%	-1.254	1.632	0.07%	0.940op/s	1	200
trace_buffer/1_senders/no_delay	execution_time	334.516µs	341.784µs ± 5.251µs	340.858µs ± 2.048µs	343.337µs	348.089µs	357.876µs	394.119µs	15.63%	5.483	48.668	1.53%	0.371µs	1	200
trace_buffer/1_senders/no_delay	throughput	2283571.428op/s	2633817.259op/s ± 37564.275op/s	2640397.412op/s ± 15861.991op/s	2654276.567op/s	2670922.034op/s	2679749.223op/s	2690453.650op/s	1.90%	-4.631	37.323	1.42%	2656.195op/s	1	200
trace_buffer/2_senders/10us_delay	execution_time	56.813ms	57.211ms ± 0.207ms	57.200ms ± 0.162ms	57.366ms	57.542ms	57.647ms	57.764ms	0.99%	0.199	-0.694	0.36%	0.015ms	1	200
trace_buffer/2_senders/10us_delay	throughput	31161.350op/s	31463.116op/s ± 113.874op/s	31468.720op/s ± 88.957op/s	31548.874op/s	31640.739op/s	31673.156op/s	31683.045op/s	0.68%	-0.185	-0.703	0.36%	8.052op/s	1	200
trace_buffer/2_senders/1us_delay	execution_time	50.070ms	50.120ms ± 0.038ms	50.110ms ± 0.019ms	50.133ms	50.202ms	50.221ms	50.227ms	0.23%	1.050	0.351	0.08%	0.003ms	1	200
trace_buffer/2_senders/1us_delay	throughput	35837.324op/s	35914.158op/s ± 27.070op/s	35921.154op/s ± 13.702op/s	35933.195op/s	35945.416op/s	35948.972op/s	35949.731op/s	0.08%	-1.047	0.344	0.08%	1.914op/s	1	200
trace_buffer/2_senders/no_delay	execution_time	846.454µs	907.620µs ± 17.529µs	908.040µs ± 10.203µs	919.243µs	932.891µs	942.143µs	950.891µs	4.72%	-0.491	0.561	1.93%	1.239µs	1	200
trace_buffer/2_senders/no_delay	throughput	1892961.333op/s	1983953.632op/s ± 38734.190op/s	1982292.014op/s ± 22288.125op/s	2002940.774op/s	2058619.433op/s	2096305.406op/s	2126519.370op/s	7.28%	0.629	0.815	1.95%	2738.921op/s	1	200
trace_buffer/4_senders/10us_delay	execution_time	57.185ms	57.516ms ± 0.150ms	57.497ms ± 0.081ms	57.594ms	57.782ms	57.943ms	58.091ms	1.03%	0.848	1.162	0.26%	0.011ms	1	200
trace_buffer/4_senders/10us_delay	throughput	61972.101op/s	62591.639op/s ± 163.077op/s	62611.557op/s ± 88.416op/s	62697.437op/s	62827.161op/s	62896.554op/s	62953.494op/s	0.55%	-0.828	1.118	0.26%	11.531op/s	1	200
trace_buffer/4_senders/1us_delay	execution_time	50.104ms	50.169ms ± 0.042ms	50.157ms ± 0.022ms	50.183ms	50.256ms	50.294ms	50.342ms	0.37%	1.399	1.959	0.08%	0.003ms	1	200
trace_buffer/4_senders/1us_delay	throughput	71511.295op/s	71757.115op/s ± 60.359op/s	71774.139op/s ± 31.325op/s	71798.284op/s	71821.022op/s	71832.610op/s	71849.874op/s	0.11%	-1.394	1.939	0.08%	4.268op/s	1	200
trace_buffer/4_senders/no_delay	execution_time	2.009ms	2.309ms ± 0.076ms	2.316ms ± 0.052ms	2.363ms	2.424ms	2.465ms	2.490ms	7.54%	-0.488	0.714	3.29%	0.005ms	1	200
trace_buffer/4_senders/no_delay	throughput	1445575.658op/s	1560546.966op/s ± 52607.007op/s	1554547.739op/s ± 34102.425op/s	1595067.300op/s	1650000.029op/s	1685914.997op/s	1792189.749op/s	15.29%	0.753	1.438	3.36%	3719.877op/s	1	200
trace_buffer/8_senders/10us_delay	execution_time	57.624ms	57.987ms ± 0.266ms	57.880ms ± 0.126ms	58.195ms	58.512ms	58.634ms	58.751ms	1.51%	0.889	-0.330	0.46%	0.019ms	1	200
trace_buffer/8_senders/10us_delay	throughput	122550.307op/s	124168.067op/s ± 566.427op/s	124396.077op/s ± 270.928op/s	124617.302op/s	124789.985op/s	124911.162op/s	124946.997op/s	0.44%	-0.877	-0.358	0.46%	40.052op/s	1	200
trace_buffer/8_senders/1us_delay	execution_time	50.209ms	50.294ms ± 0.060ms	50.274ms ± 0.028ms	50.315ms	50.402ms	50.490ms	50.540ms	0.53%	1.556	2.617	0.12%	0.004ms	1	200
trace_buffer/8_senders/1us_delay	throughput	142461.174op/s	143159.717op/s ± 169.262op/s	143216.377op/s ± 79.991op/s	143270.342op/s	143341.670op/s	143371.172op/s	143401.805op/s	0.13%	-1.548	2.585	0.12%	11.969op/s	1	200
trace_buffer/8_senders/no_delay	execution_time	6.367ms	6.656ms ± 0.096ms	6.656ms ± 0.051ms	6.706ms	6.804ms	6.834ms	7.195ms	8.10%	0.631	4.466	1.43%	0.007ms	1	200
trace_buffer/8_senders/no_delay	throughput	1000628.867op/s	1082030.073op/s ± 15441.988op/s	1081702.762op/s ± 8352.983op/s	1090452.065op/s	1105869.644op/s	1119731.822op/s	1130794.971op/s	4.54%	-0.388	3.445	1.42%	1091.913op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
trace_buffer/1_senders/10us_delay	execution_time	[56.608ms; 56.686ms] or [-0.069%; +0.069%]	None	None	None
trace_buffer/1_senders/10us_delay	throughput	[15877.320op/s; 15899.100op/s] or [-0.069%; +0.069%]	None	None	None
trace_buffer/1_senders/1us_delay	execution_time	[50.084ms; 50.094ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/1_senders/1us_delay	throughput	[17966.078op/s; 17969.764op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/1_senders/no_delay	execution_time	[341.056µs; 342.512µs] or [-0.213%; +0.213%]	None	None	None
trace_buffer/1_senders/no_delay	throughput	[2628611.212op/s; 2639023.306op/s] or [-0.198%; +0.198%]	None	None	None
trace_buffer/2_senders/10us_delay	execution_time	[57.182ms; 57.239ms] or [-0.050%; +0.050%]	None	None	None
trace_buffer/2_senders/10us_delay	throughput	[31447.334op/s; 31478.898op/s] or [-0.050%; +0.050%]	None	None	None
trace_buffer/2_senders/1us_delay	execution_time	[50.114ms; 50.125ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/2_senders/1us_delay	throughput	[35910.406op/s; 35917.910op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/2_senders/no_delay	execution_time	[905.190µs; 910.049µs] or [-0.268%; +0.268%]	None	None	None
trace_buffer/2_senders/no_delay	throughput	[1978585.446op/s; 1989321.819op/s] or [-0.271%; +0.271%]	None	None	None
trace_buffer/4_senders/10us_delay	execution_time	[57.495ms; 57.537ms] or [-0.036%; +0.036%]	None	None	None
trace_buffer/4_senders/10us_delay	throughput	[62569.038op/s; 62614.240op/s] or [-0.036%; +0.036%]	None	None	None
trace_buffer/4_senders/1us_delay	execution_time	[50.163ms; 50.175ms] or [-0.012%; +0.012%]	None	None	None
trace_buffer/4_senders/1us_delay	throughput	[71748.750op/s; 71765.480op/s] or [-0.012%; +0.012%]	None	None	None
trace_buffer/4_senders/no_delay	execution_time	[2.299ms; 2.320ms] or [-0.458%; +0.458%]	None	None	None
trace_buffer/4_senders/no_delay	throughput	[1553256.141op/s; 1567837.791op/s] or [-0.467%; +0.467%]	None	None	None
trace_buffer/8_senders/10us_delay	execution_time	[57.950ms; 58.024ms] or [-0.063%; +0.063%]	None	None	None
trace_buffer/8_senders/10us_delay	throughput	[124089.565op/s; 124246.568op/s] or [-0.063%; +0.063%]	None	None	None
trace_buffer/8_senders/1us_delay	execution_time	[50.285ms; 50.302ms] or [-0.016%; +0.016%]	None	None	None
trace_buffer/8_senders/1us_delay	throughput	[143136.259op/s; 143183.175op/s] or [-0.016%; +0.016%]	None	None	None
trace_buffer/8_senders/no_delay	execution_time	[6.642ms; 6.669ms] or [-0.199%; +0.199%]	None	None	None
trace_buffer/8_senders/no_delay	throughput	[1079889.962op/s; 1084170.184op/s] or [-0.198%; +0.198%]	None	None	None

Baseline

Baseline benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`85ce322`	1783958688	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
trace_buffer/1_senders/10us_delay	execution_time	56.291ms	56.663ms ± 0.273ms	56.578ms ± 0.110ms	56.755ms	57.271ms	57.430ms	57.698ms	1.98%	1.375	1.312	0.48%	0.019ms	1	200
trace_buffer/1_senders/10us_delay	throughput	15598.421op/s	15883.696op/s ± 76.138op/s	15907.117op/s ± 30.913op/s	15936.108op/s	15962.158op/s	15976.190op/s	15988.319op/s	0.51%	-1.354	1.241	0.48%	5.384op/s	1	200
trace_buffer/1_senders/1us_delay	execution_time	50.024ms	50.083ms ± 0.029ms	50.081ms ± 0.008ms	50.088ms	50.134ms	50.200ms	50.248ms	0.33%	2.102	8.195	0.06%	0.002ms	1	200
trace_buffer/1_senders/1us_delay	throughput	17911.016op/s	17970.061op/s ± 10.432op/s	17970.834op/s ± 2.938op/s	17974.301op/s	17983.495op/s	17989.177op/s	17991.504op/s	0.12%	-2.092	8.143	0.06%	0.738op/s	1	200
trace_buffer/1_senders/no_delay	execution_time	337.333µs	347.065µs ± 5.500µs	346.657µs ± 3.104µs	349.780µs	352.308µs	362.679µs	396.250µs	14.31%	3.718	30.769	1.58%	0.389µs	1	200
trace_buffer/1_senders/no_delay	throughput	2271294.770op/s	2593785.891op/s ± 39067.109op/s	2596227.766op/s ± 23041.401op/s	2618938.187op/s	2647579.020op/s	2660463.991op/s	2667984.902op/s	2.76%	-2.922	22.116	1.50%	2762.462op/s	1	200
trace_buffer/2_senders/10us_delay	execution_time	56.484ms	57.051ms ± 0.258ms	57.016ms ± 0.178ms	57.225ms	57.535ms	57.639ms	57.683ms	1.17%	0.519	-0.473	0.45%	0.018ms	1	200
trace_buffer/2_senders/10us_delay	throughput	31205.060op/s	31551.291op/s ± 142.505op/s	31569.840op/s ± 98.790op/s	31659.884op/s	31740.600op/s	31798.934op/s	31867.183op/s	0.94%	-0.502	-0.486	0.45%	10.077op/s	1	200
trace_buffer/2_senders/1us_delay	execution_time	50.059ms	50.111ms ± 0.035ms	50.106ms ± 0.019ms	50.122ms	50.189ms	50.227ms	50.234ms	0.26%	1.434	2.209	0.07%	0.002ms	1	200
trace_buffer/2_senders/1us_delay	throughput	35832.153op/s	35920.479op/s ± 25.254op/s	35924.172op/s ± 13.372op/s	35937.864op/s	35949.988op/s	35955.194op/s	35957.227op/s	0.09%	-1.430	2.194	0.07%	1.786op/s	1	200
trace_buffer/2_senders/no_delay	execution_time	875.401µs	928.075µs ± 16.639µs	928.615µs ± 11.349µs	940.644µs	953.339µs	959.859µs	966.554µs	4.09%	-0.432	0.129	1.79%	1.177µs	1	200
trace_buffer/2_senders/no_delay	throughput	1862285.791op/s	1940125.211op/s ± 35088.288op/s	1938371.376op/s ± 23575.182op/s	1959899.748op/s	2000862.797op/s	2035695.540op/s	2056199.839op/s	6.08%	0.538	0.299	1.80%	2481.117op/s	1	200
trace_buffer/4_senders/10us_delay	execution_time	56.991ms	57.304ms ± 0.192ms	57.272ms ± 0.123ms	57.428ms	57.688ms	57.824ms	57.858ms	1.02%	0.763	0.022	0.33%	0.014ms	1	200
trace_buffer/4_senders/10us_delay	throughput	62221.339op/s	62823.832op/s ± 210.232op/s	62857.996op/s ± 135.142op/s	62977.962op/s	63100.818op/s	63124.286op/s	63167.339op/s	0.49%	-0.749	-0.008	0.33%	14.866op/s	1	200
trace_buffer/4_senders/1us_delay	execution_time	50.097ms	50.162ms ± 0.035ms	50.156ms ± 0.017ms	50.172ms	50.245ms	50.282ms	50.293ms	0.27%	1.530	2.816	0.07%	0.002ms	1	200
trace_buffer/4_senders/1us_delay	throughput	71580.376op/s	71767.874op/s ± 49.834op/s	71776.570op/s ± 24.058op/s	71800.754op/s	71825.961op/s	71835.607op/s	71860.509op/s	0.12%	-1.525	2.799	0.07%	3.524op/s	1	200
trace_buffer/4_senders/no_delay	execution_time	2.417ms	2.516ms ± 0.036ms	2.519ms ± 0.027ms	2.544ms	2.574ms	2.585ms	2.596ms	3.05%	-0.298	-0.263	1.41%	0.003ms	1	200
trace_buffer/4_senders/no_delay	throughput	1386931.102op/s	1430915.780op/s ± 20352.946op/s	1429298.010op/s ± 14951.108op/s	1445154.640op/s	1467860.681op/s	1479050.259op/s	1489295.423op/s	4.20%	0.369	-0.203	1.42%	1439.171op/s	1	200
trace_buffer/8_senders/10us_delay	execution_time	57.538ms	57.855ms ± 0.210ms	57.792ms ± 0.109ms	57.946ms	58.267ms	58.440ms	58.581ms	1.37%	1.171	0.691	0.36%	0.015ms	1	200
trace_buffer/8_senders/10us_delay	throughput	122906.285op/s	124450.350op/s ± 449.223op/s	124584.976op/s ± 236.249op/s	124753.981op/s	124948.971op/s	125046.396op/s	125135.257op/s	0.44%	-1.156	0.650	0.36%	31.765op/s	1	200
trace_buffer/8_senders/1us_delay	execution_time	50.197ms	50.285ms ± 0.051ms	50.274ms ± 0.026ms	50.302ms	50.372ms	50.485ms	50.508ms	0.47%	1.571	3.515	0.10%	0.004ms	1	200
trace_buffer/8_senders/1us_delay	throughput	142550.337op/s	143184.964op/s ± 145.063op/s	143215.476op/s ± 74.036op/s	143285.225op/s	143348.292op/s	143384.480op/s	143434.678op/s	0.15%	-1.562	3.472	0.10%	10.257op/s	1	200
trace_buffer/8_senders/no_delay	execution_time	6.354ms	6.671ms ± 0.089ms	6.683ms ± 0.052ms	6.732ms	6.796ms	6.837ms	6.861ms	2.67%	-0.802	0.965	1.33%	0.006ms	1	200
trace_buffer/8_senders/no_delay	throughput	1049461.408op/s	1079433.038op/s ± 14512.582op/s	1077431.191op/s ± 8402.710op/s	1086941.336op/s	1106879.240op/s	1126286.842op/s	1133168.611op/s	5.17%	0.897	1.211	1.34%	1026.195op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
trace_buffer/1_senders/10us_delay	execution_time	[56.625ms; 56.701ms] or [-0.067%; +0.067%]	None	None	None
trace_buffer/1_senders/10us_delay	throughput	[15873.144op/s; 15894.248op/s] or [-0.066%; +0.066%]	None	None	None
trace_buffer/1_senders/1us_delay	execution_time	[50.079ms; 50.087ms] or [-0.008%; +0.008%]	None	None	None
trace_buffer/1_senders/1us_delay	throughput	[17968.616op/s; 17971.507op/s] or [-0.008%; +0.008%]	None	None	None
trace_buffer/1_senders/no_delay	execution_time	[346.303µs; 347.828µs] or [-0.220%; +0.220%]	None	None	None
trace_buffer/1_senders/no_delay	throughput	[2588371.565op/s; 2599200.216op/s] or [-0.209%; +0.209%]	None	None	None
trace_buffer/2_senders/10us_delay	execution_time	[57.015ms; 57.087ms] or [-0.063%; +0.063%]	None	None	None
trace_buffer/2_senders/10us_delay	throughput	[31531.541op/s; 31571.041op/s] or [-0.063%; +0.063%]	None	None	None
trace_buffer/2_senders/1us_delay	execution_time	[50.106ms; 50.116ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/2_senders/1us_delay	throughput	[35916.979op/s; 35923.979op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/2_senders/no_delay	execution_time	[925.768µs; 930.381µs] or [-0.248%; +0.248%]	None	None	None
trace_buffer/2_senders/no_delay	throughput	[1935262.312op/s; 1944988.111op/s] or [-0.251%; +0.251%]	None	None	None
trace_buffer/4_senders/10us_delay	execution_time	[57.277ms; 57.330ms] or [-0.046%; +0.046%]	None	None	None
trace_buffer/4_senders/10us_delay	throughput	[62794.696op/s; 62852.968op/s] or [-0.046%; +0.046%]	None	None	None
trace_buffer/4_senders/1us_delay	execution_time	[50.157ms; 50.167ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/4_senders/1us_delay	throughput	[71760.967op/s; 71774.780op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/4_senders/no_delay	execution_time	[2.511ms; 2.521ms] or [-0.196%; +0.196%]	None	None	None
trace_buffer/4_senders/no_delay	throughput	[1428095.057op/s; 1433736.502op/s] or [-0.197%; +0.197%]	None	None	None
trace_buffer/8_senders/10us_delay	execution_time	[57.826ms; 57.884ms] or [-0.050%; +0.050%]	None	None	None
trace_buffer/8_senders/10us_delay	throughput	[124388.092op/s; 124512.608op/s] or [-0.050%; +0.050%]	None	None	None
trace_buffer/8_senders/1us_delay	execution_time	[50.278ms; 50.292ms] or [-0.014%; +0.014%]	None	None	None
trace_buffer/8_senders/1us_delay	throughput	[143164.860op/s; 143205.068op/s] or [-0.014%; +0.014%]	None	None	None
trace_buffer/8_senders/no_delay	execution_time	[6.659ms; 6.684ms] or [-0.184%; +0.184%]	None	None	None
trace_buffer/8_senders/no_delay	throughput	[1077421.733op/s; 1081444.342op/s] or [-0.186%; +0.186%]	None	None	None

dd-octo-sts · 2026-07-13T17:57:47Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	86.15 MB	86.15 MB	-0% (-32 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	7.88 MB	7.88 MB	0% (0 B) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.62 MB	10.62 MB	0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	97.40 MB	97.40 MB	-0% (-24 B) 👌

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	25.54 MB	25.54 MB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	88.44 KB	88.44 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	185.34 MB	185.33 MB	-0% (-8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	957.25 MB	957.25 MB	+0% (+3.46 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.35 MB	8.35 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	88.44 KB	88.44 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	24.72 MB	24.72 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	49.21 MB	49.21 MB	0% (0 B) 👌

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	22.19 MB	22.19 MB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	89.82 KB	89.82 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	189.79 MB	189.79 MB	0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	945.93 MB	945.93 MB	+0% (+3.38 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.46 MB	6.46 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	89.82 KB	89.82 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	26.54 MB	26.54 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	46.82 MB	46.82 MB	0% (0 B) 👌

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	76.96 MB	76.96 MB	-0% (-8 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.81 MB	8.81 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	92.37 MB	92.37 MB	-0% (-8 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.72 MB	10.72 MB	0% (0 B) 👌

bm1549 added the AI Generated PR largely written by AI tools label Jul 13, 2026

github-actions Bot added the data-pipeline label Jul 13, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(data-pipeline): allow opting into agent-info wait before first flush#2233

fix(data-pipeline): allow opting into agent-info wait before first flush#2233
bm1549 wants to merge 1 commit into
mainfrom
brian.marks/p0-drop-startup-race

bm1549 commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026 •

edited

Loading

Uh oh!

datadog-datadog-us1-prod Bot commented Jul 13, 2026

Uh oh!

pr-commenter Bot commented Jul 13, 2026 •

edited

Loading

Explanation

More details about the CI and significant changes

Group 1

Group 1

Uh oh!

dd-octo-sts Bot commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

bm1549 commented Jul 13, 2026

What does this PR do?

Motivation

Additional Notes

How to test the change?

Uh oh!

github-actions Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Documentation Check Results

📦 libdd-data-pipeline - 1202 warning(s)

Uh oh!

github-actions Bot commented Jul 13, 2026

Clippy Allow Annotation Report

About This Report

Uh oh!

github-actions Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Cargo Deny Results

📦 libdd-data-pipeline - 3 error(s)

Uh oh!

datadog-datadog-us1-prod Bot commented Jul 13, 2026

Uh oh!

pr-commenter Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Comparison

Explanation

More details about the CI and significant changes

scenario:trace_buffer/4_senders/no_delay

Candidate

Group 1

Baseline

Group 1

Uh oh!

dd-octo-sts Bot commented Jul 13, 2026

Artifact Size Benchmark Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Jul 13, 2026 •

edited

Loading

📦 `libdd-data-pipeline` - 1202 warning(s)

github-actions Bot commented Jul 13, 2026 •

edited

Loading

📦 `libdd-data-pipeline` - 3 error(s)

pr-commenter Bot commented Jul 13, 2026 •

edited

Loading