Skip to content

Commit f9ac7f2

Browse files
test(profiling): add profiles dictionary benchmarks (#2088)
## What does this PR do? Adds a focused Criterion benchmark for `ProfilesDictionary` unique string insertion. The benchmark uses `ProfilesDictionary` and covers 1, 2, 4, and 16 producer threads, so follow-up arena sizing/growth changes have a baseline in the GitLab benchmark job. #2048 changes the default configs for the ProfilesDictionary ## Why these thread counts? This benchmark is focused on dictionary string interning. It is not a full end-to-end profiler benchmark. | Thread count | Why it is useful | |---|---| | 1 | Approximates single-writer consumers such as ddprof. ddprof appears to intern from the worker thread while export happens from a separate inactive profile buffer. | | 2 | Approximates current dd-trace-py contention: one native stack sampler thread can intern off-GIL while one Python/Cython collector path is active under the GIL. | | 4 / 16 | Stress cases for future higher-concurrency scenarios, including possible free-threaded/no-GIL Python work. | In dd-trace-py, profile mutation and serialization are guarded by `profile_mtx`, but dictionary interning can happen before a sample is added to the profile. This means dictionary insertion can still be concurrent even when profile writes are serialized. ## What this does not cover This benchmark does not model every profiler behavior. In particular, it does not cover: - function or mapping insertion, - duplicate-heavy/cached lookup workloads, - profile serialization reading from the dictionary, - full profiler end-to-end behavior. ## Why only `ProfilesDictionary`? I originally tried an additional synthetic benchmark comparing 4 vs 16 shards. Local exploratory results suggested 16 shards helps once there is concurrent insertion: ```text 1 thread: 4 shards ~40 µs, 16 shards ~55 µs 2 threads: 4 shards ~157 µs, 16 shards ~129 µs 4 threads: 4 shards ~385 µs, 16 shards ~280 µs 16 threads: 4 shards ~2.64 ms, 16 shards ~1.61 ms ``` However, that synthetic comparison also changed total initial hash-table capacity because the capacity was applied per shard. Since the current follow-up keeps the production shard count at 16, this PR stays minimal and only adds the `ProfilesDictionary` benchmark. If we revisit shard count later, we should add a dedicated shard-count benchmark that holds total starting capacity constant across shard counts. ## How to test the change? - cargo +nightly-2026-02-08 fmt --all -- --check - cargo check -p libdd-profiling --benches - cargo +stable clippy -p libdd-profiling --benches -- -D warnings [PROF-14423](https://datadoghq.atlassian.net/browse/PROF-14423) [PROF-14423]: https://datadoghq.atlassian.net/browse/PROF-14423?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: morrisonlevi <levi.morrison@datadoghq.com> Co-authored-by: taegyun.kim <taegyun.kim@datadoghq.com>
1 parent 65d8d24 commit f9ac7f2

2 files changed

Lines changed: 109 additions & 1 deletion

File tree

libdd-profiling/benches/main.rs

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,5 +5,10 @@ use criterion::criterion_main;
55

66
mod add_samples;
77
mod interning_strings;
8+
mod profiles_dictionary;
89

9-
criterion_main!(interning_strings::benches, add_samples::benches);
10+
criterion_main!(
11+
interning_strings::benches,
12+
add_samples::benches,
13+
profiles_dictionary::benches
14+
);
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
// Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/
2+
// SPDX-License-Identifier: Apache-2.0
3+
4+
use criterion::{black_box, criterion_group, BatchSize, BenchmarkId, Criterion, Throughput};
5+
use libdd_profiling::profiles::datatypes::ProfilesDictionary;
6+
use std::sync::Barrier;
7+
use std::thread;
8+
use std::time::Duration;
9+
10+
// 1-2 threads matches expected profiler usage; higher counts stress contention behavior.
11+
const THREAD_COUNTS: [usize; 4] = [1, 2, 4, 16];
12+
const STRINGS_PER_THREAD: usize = 1024;
13+
// Bound one generated function-name component so the input has repeated
14+
// function-like fragments, with some full strings shared across workers.
15+
const FUNCTION_NAME_VARIANTS: usize = 97;
16+
const STRING_SHAPE_VARIANTS: usize = 4;
17+
// Knuth/Fibonacci multiplicative hash constant, used only to vary synthetic input.
18+
const KNUTH_MULTIPLICATIVE_HASH: usize = 2_654_435_761;
19+
20+
// The outer Vec partitions precomputed input by benchmark worker thread; each
21+
// inner Vec is the set of strings inserted by one worker.
22+
fn make_partitioned_strings(thread_count: usize) -> Vec<Vec<String>> {
23+
(0..thread_count)
24+
.map(|thread_id| {
25+
(0..STRINGS_PER_THREAD)
26+
.map(|string_id| {
27+
let function_id = string_id % FUNCTION_NAME_VARIANTS;
28+
let mixed_id = string_id.wrapping_mul(KNUTH_MULTIPLICATIVE_HASH);
29+
30+
match string_id % STRING_SHAPE_VARIANTS {
31+
0 => format!("function_{function_id}::{mixed_id}"),
32+
1 => format!("/src/thread_{thread_id}/module_{function_id}/file_{string_id:04}.rs"),
33+
2 => {
34+
format!("datadog::profiling::module_{function_id}::function_{mixed_id}")
35+
}
36+
_ => format!(
37+
"/opt/datadog/profiler/thread-{thread_id}/module-{function_id}/src/file_{string_id:04}.rs::function_{function_id}::{mixed_id}",
38+
),
39+
}
40+
})
41+
.collect()
42+
})
43+
.collect()
44+
}
45+
46+
fn insert_profile_strings(dict: &ProfilesDictionary, strings: &[String]) {
47+
for string in strings {
48+
black_box(dict.try_insert_str2(string.as_str()).unwrap());
49+
}
50+
}
51+
52+
fn insert_dictionary_strings_concurrently(strings: &[Vec<String>]) -> ProfilesDictionary {
53+
let dict = ProfilesDictionary::try_new().unwrap();
54+
55+
if let [strings] = strings {
56+
insert_profile_strings(&dict, strings);
57+
return dict;
58+
}
59+
60+
let barrier = Barrier::new(strings.len());
61+
thread::scope(|scope| {
62+
for thread_strings in strings {
63+
let dict = &dict;
64+
let barrier = &barrier;
65+
scope.spawn(move || {
66+
barrier.wait();
67+
insert_profile_strings(dict, thread_strings);
68+
});
69+
}
70+
});
71+
72+
dict
73+
}
74+
75+
pub fn bench_profile_string_inserts(c: &mut Criterion) {
76+
let mut group = c.benchmark_group("profiles_dictionary/profile_string_inserts");
77+
group.warm_up_time(Duration::from_secs(1));
78+
group.measurement_time(Duration::from_secs(5));
79+
group.sample_size(10);
80+
81+
for thread_count in THREAD_COUNTS {
82+
// Precompute input outside the measured closure so the benchmark measures
83+
// dictionary insertion rather than string formatting/allocation.
84+
let strings = make_partitioned_strings(thread_count);
85+
let total_strings = thread_count * STRINGS_PER_THREAD;
86+
group.throughput(Throughput::Elements(total_strings as u64));
87+
group.bench_with_input(
88+
BenchmarkId::new("threads", thread_count),
89+
&strings,
90+
|b, strings| {
91+
b.iter_batched(
92+
|| strings,
93+
|strings| black_box(insert_dictionary_strings_concurrently(strings)),
94+
BatchSize::LargeInput,
95+
);
96+
},
97+
);
98+
}
99+
100+
group.finish();
101+
}
102+
103+
criterion_group!(benches, bench_profile_string_inserts);

0 commit comments

Comments
 (0)