Skip to content

Commit 1223a7c

Browse files
authored
Merge branch 'main' into paul.fournillon/wallclock-taskblock
2 parents be21617 + 3a90147 commit 1223a7c

11 files changed

Lines changed: 1021 additions & 28 deletions

File tree

README.md

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -169,23 +169,40 @@ By default, the script clones the repository at the current commit for clean bui
169169
# Force rebuild of base image only (useful after Alpine/Ubuntu updates)
170170
./utils/run-containers-tests.sh --libc=musl --rebuild-base
171171

172+
# Preview all supported musl cells (sanitizer configs are skipped, matching CI policy)
173+
./utils/run-containers-tests.sh --matrix --libc=musl
174+
175+
# Run all supported musl cells without an interactive prompt
176+
./utils/run-containers-tests.sh --matrix --libc=musl --run
177+
178+
# Preview selected JDKs across every supported libc/architecture pair
179+
./utils/run-containers-tests.sh --libc=all --jdk=8,17,21 --arch=all
180+
181+
# Run all OpenJ9 cells
182+
./utils/run-containers-tests.sh --matrix --jdk=j9 --run
183+
172184
# Show options
173185
./utils/run-containers-tests.sh --help
174186
```
175187

176188
Supported options:
177-
- `--libc=glibc|musl` (default: glibc)
178-
- `--jdk=8|11|17|21|25|8-j9|11-j9|17-j9|21-j9|17-graal|21-graal|25-graal` (default: 21)
179-
- `--arch=x64|aarch64` (default: auto-detect)
180-
- `--config=debug|release|asan|tsan` (default: debug)
189+
- `--libc=glibc|musl|all[,..]` (default: glibc)
190+
- `--jdk=8|11|17|21|25|8-j9|11-j9|17-j9|21-j9|17-graal|21-graal|25-graal|regular|j9|graal|all[,..]` (default: 21)
191+
- `--arch=x64|aarch64|all[,..]` (default: auto-detect)
192+
- `--config=debug|release|asan|tsan|all[,..]` (default: debug)
181193
- `--container=podman|docker` (default: podman)
182194
- `--tests="TestPattern"`
183195
- `--gtest` (enable C++ gtests, disabled by default for faster runs)
184-
- `--gtest-task=Task` (run one C++ gtest task; accepts `elfparser_ut` or a full task path like `:ddprof-lib:gtestAsan_elfparser_ut`)
196+
- `--gtest-task=Task` (run one C++ gtest task; matrix runs require a short name like `elfparser_ut`, while single-cell runs also accept a full task path like `:ddprof-lib:gtestAsan_elfparser_ut`)
185197
- `--shell` (interactive shell instead of running tests)
186198
- `--mount` (mount local repo instead of cloning - faster but may have stale artifacts)
187199
- `--rebuild` (force rebuild of all container images)
188200
- `--rebuild-base` (force rebuild of base image only)
201+
- `--matrix` (preview a full test matrix; unset dimensions expand to `all`)
202+
- `--run` (execute an inferred matrix without prompting)
203+
- `--fail-fast` (stop matrix execution on first failure)
204+
205+
Single-value commands run one configuration immediately. When any dimension expands to multiple cells, the script prints a compact status table first; interactive terminals ask for confirmation, while non-interactive runs require `--run` to execute. Matrix execution prints the status table again after all cells finish and writes summaries to `build/reports/container-matrix/summary.md` and `build/reports/container-matrix/summary.json`. Cells not run because of `--fail-fast` are reported as cancelled separately from unsupported cells that are skipped.
189206

190207
## Unwinding Validation Tool
191208

ddprof-lib/src/test/fuzz/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,11 +154,35 @@ the 49152-entry expansion threshold, forcing the multi-node `_prev` chain that e
154154
**Regression guard — detects if these bugs are reintroduced**: heap-use-after-free (ASan), data race on `_table` (TSan), null-deref in
155155
`processTraces` from use-after-free when `_prev` chain is not fully disconnected.
156156

157+
### fuzz_threadLocal.cpp
158+
**Target**: `ThreadLocal<T, CREATE_FUNC, CLEAN_FUNC>` (and the `double`/generic-pointer
159+
specializations) - the pthread-TSD-based alternative to `thread_local` used because
160+
`pthread_(get/set)specific()` cannot be safely introduced mid-signal-handling on some
161+
platforms (see the comment block in `threadLocal.h`).
162+
163+
Each input is replayed on a freshly spawned thread that is joined before returning, so
164+
the pthread key destructor for whatever is left in a slot fires synchronously inside
165+
that `join()` - the one part of the lifecycle a persistent driver thread would never
166+
exercise. A shadow model tracks the expected create/get/set/clear state and traps on
167+
any mismatch.
168+
169+
**Expected bugs**: stale/mismatched values from `get()`, `create_tracked()`/`free_tracked()`
170+
running the wrong number of times (double free or leak across `clear()`, an overwriting
171+
`set()`, or thread-exit teardown), `set(nullptr)` failing to trigger a lazy recreate on
172+
the next `get()`, and non-bit-exact round-trips for the `double` specialization (NaN,
173+
infinities, subnormals, -0.0).
174+
157175
## Corpus
158176

159177
Seed corpus files are in `corpus/<target_name>/`. These provide starting points
160178
for the fuzzer to understand the expected input format.
161179

180+
`corpus/fuzz_threadLocal/` seeds a few inputs per opcode range documented in
181+
`fuzz_threadLocal.cpp` (tracked lifecycle/overwrite, `nullptr` recreate, double
182+
and intptr bit-exact round-trips including NaN/inf/-0.0/subnormal, and a
183+
truncated-payload case) to speed up initial coverage discovery. Not required —
184+
the fuzzer runs fine from an empty corpus — but these give it a head start.
185+
162186
During fuzzing, libFuzzer will add new interesting inputs to the corpus
163187
directory. These additions are machine-generated and should not be committed.
164188

Binary file not shown.
Binary file not shown.
48 Bytes
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
/*
2+
* Copyright 2026, Datadog, Inc.
3+
* SPDX-License-Identifier: Apache-2.0
4+
*
5+
* libFuzzer target for ThreadLocal<T, CREATE_FUNC, CLEAN_FUNC> (threadLocal.h).
6+
*
7+
* ThreadLocal wraps pthread TSD to work around pthread_(get/set)specific() not
8+
* being safe to introduce mid-signal-handling (see the big comment at the top
9+
* of threadLocal.h). The interesting bug surface is the lazy create/cleanup
10+
* state machine across a *fresh thread's* full lifecycle — a slot that starts
11+
* unset, is get()/set()/clear()'d in some order, and is then torn down by the
12+
* pthread key destructor when the thread exits. Running everything on
13+
* libFuzzer's persistent driver thread would never exercise the exit-time
14+
* destructor path, so each input is replayed on a freshly spawned thread that
15+
* is joined before returning — the destructor for whatever is left in the
16+
* slot fires synchronously inside that join().
17+
*
18+
* Input bytes are consumed as a stream of operations against three static
19+
* ThreadLocal specializations that live side by side in the same input:
20+
* 0x00-0x0F -> tracked.get() (lazy-creates via create_tracked() if empty)
21+
* 0x10-0x1F -> tracked.clear() (invokes free_tracked() if non-empty)
22+
* 0x20-0x2F -> tracked.set(fresh pointer) (caller frees any old value first,
23+
* mirroring the documented contract)
24+
* 0x30-0x3F -> tracked.set(nullptr) (frees old value; next get() must
25+
* lazily recreate, per threadLocal.h)
26+
* 0x40-0x4F -> double.set(next 8 bytes as raw bit pattern)
27+
* 0x50-0x5F -> double.get() (verify exact bit round-trip)
28+
* 0x60-0x6F -> double.clear()
29+
* 0x70-0x7F -> intptr.set(next 8 bytes as raw bit pattern)
30+
* 0x80-0x8F -> intptr.get() (verify exact round-trip)
31+
* 0x90-0x9F -> intptr.clear()
32+
* 0xA0-0xFF -> no-op (padding / density filler for the mutator)
33+
*
34+
* Invariants verified (violation -> __builtin_trap() -> ASan/fuzzer crash):
35+
* I1. get() never returns a stale/mismatched value: the payload behind the
36+
* pointer/bits returned always matches what the model last stored.
37+
* I2. create_tracked() runs at most once per "empty -> get()" transition and
38+
* free_tracked() runs exactly once per value that ever occupied the slot
39+
* (via clear(), an overwriting set(), or the pthread key destructor at
40+
* thread exit) - checked via the create/free counters delta across the
41+
* whole spawned-thread run.
42+
* I3. set(nullptr) followed by get() lazily recreates (documented contract).
43+
* I4. double/intptr specializations round-trip bit-for-bit, including NaN,
44+
* infinities, subnormals and -0.0, which plain == comparison would hide.
45+
*/
46+
47+
#include <stddef.h>
48+
#include <stdint.h>
49+
50+
#include <atomic>
51+
#include <cstring>
52+
#include <system_error>
53+
#include <thread>
54+
55+
#include "threadLocal.h"
56+
57+
namespace {
58+
59+
std::atomic<uint64_t> g_create_count{0};
60+
std::atomic<uint64_t> g_free_count{0};
61+
62+
void *create_tracked() {
63+
g_create_count.fetch_add(1, std::memory_order_relaxed);
64+
return new int(1234);
65+
}
66+
67+
void free_tracked(void *p) {
68+
g_free_count.fetch_add(1, std::memory_order_relaxed);
69+
delete static_cast<int *>(p);
70+
}
71+
72+
ThreadLocal<int *, create_tracked, free_tracked> g_tracked_tl;
73+
ThreadLocal<double> g_double_tl;
74+
ThreadLocal<intptr_t> g_intptr_tl;
75+
76+
u64 take8(const uint8_t *data, size_t pos) {
77+
u64 v = 0;
78+
for (int i = 0; i < 8; i++) {
79+
v = (v << 8) | data[pos + i];
80+
}
81+
return v;
82+
}
83+
84+
// Runs the whole decoded op sequence on the calling thread. Executed inside a
85+
// freshly spawned std::thread so the tracked slot starts empty and whatever
86+
// is left occupying it is torn down by the pthread key destructor at thread
87+
// exit (join() below), exercising the path a persistent fuzzer-driver thread
88+
// never would.
89+
void runOnWorkerThread(const uint8_t *data, size_t size, uint64_t *expected_creates,
90+
uint64_t *expected_frees) {
91+
bool tracked_present = false;
92+
int *tracked_ptr = nullptr;
93+
int tracked_expected = 0;
94+
int manual_marker_seq = 1;
95+
96+
bool double_present = false;
97+
u64 double_expected_bits = 0;
98+
99+
bool intptr_present = false;
100+
intptr_t intptr_expected = 0;
101+
102+
size_t pos = 0;
103+
while (pos < size) {
104+
uint8_t op = data[pos++];
105+
106+
if (op < 0x10) {
107+
// tracked.get()
108+
int *p = g_tracked_tl.get();
109+
if (!tracked_present) {
110+
// Empty slot with a non-null CREATE_FUNC must lazily create.
111+
if (p == nullptr) __builtin_trap();
112+
tracked_present = true;
113+
tracked_ptr = p;
114+
tracked_expected = 1234;
115+
(*expected_creates)++;
116+
} else if (p != tracked_ptr) {
117+
__builtin_trap(); // pointer identity must be stable across get()s
118+
}
119+
if (*p != tracked_expected) __builtin_trap(); // I1: payload corruption
120+
121+
} else if (op < 0x20) {
122+
// tracked.clear()
123+
g_tracked_tl.clear();
124+
if (tracked_present) {
125+
(*expected_frees)++;
126+
tracked_present = false;
127+
tracked_ptr = nullptr;
128+
}
129+
130+
} else if (op < 0x30) {
131+
// tracked.set(fresh manually-owned pointer). Per the documented
132+
// contract the caller frees any prior value itself before overwriting.
133+
if (tracked_present) {
134+
free_tracked(tracked_ptr);
135+
(*expected_frees)++;
136+
tracked_present = false;
137+
}
138+
// Use a negative, monotonically distinct marker so it can never be
139+
// confused with create_tracked()'s 1234 sentinel.
140+
int marker = -(manual_marker_seq++);
141+
int *fresh = new int(marker);
142+
g_tracked_tl.set(fresh);
143+
tracked_present = true;
144+
tracked_ptr = fresh;
145+
tracked_expected = marker;
146+
147+
} else if (op < 0x40) {
148+
// tracked.set(nullptr): caller frees any prior value, then the next
149+
// get() must lazily recreate (I3).
150+
if (tracked_present) {
151+
free_tracked(tracked_ptr);
152+
(*expected_frees)++;
153+
}
154+
g_tracked_tl.set(nullptr);
155+
tracked_present = false;
156+
tracked_ptr = nullptr;
157+
158+
} else if (op < 0x50) {
159+
// double.set(next 8 bytes as raw bits)
160+
if (pos + 7 >= size) break;
161+
u64 bits = take8(data, pos);
162+
pos += 8;
163+
double v;
164+
memcpy(&v, &bits, sizeof(v));
165+
g_double_tl.set(v);
166+
double_present = true;
167+
double_expected_bits = bits;
168+
169+
} else if (op < 0x60) {
170+
// double.get() - verify exact bit pattern, not value equality (NaN-safe)
171+
double v = g_double_tl.get();
172+
u64 bits;
173+
memcpy(&bits, &v, sizeof(bits));
174+
u64 expected = double_present ? double_expected_bits : 0;
175+
if (bits != expected) __builtin_trap(); // I4
176+
177+
} else if (op < 0x70) {
178+
// double.clear()
179+
g_double_tl.clear();
180+
double_present = false;
181+
182+
} else if (op < 0x80) {
183+
// intptr.set(next 8 bytes as raw bits)
184+
if (pos + 7 >= size) break;
185+
u64 bits = take8(data, pos);
186+
pos += 8;
187+
intptr_t v = static_cast<intptr_t>(bits);
188+
g_intptr_tl.set(v);
189+
intptr_present = true;
190+
intptr_expected = v;
191+
192+
} else if (op < 0x90) {
193+
// intptr.get()
194+
intptr_t v = g_intptr_tl.get();
195+
intptr_t expected = intptr_present ? intptr_expected : 0;
196+
if (v != expected) __builtin_trap(); // I4
197+
198+
} else if (op < 0xA0) {
199+
// intptr.clear()
200+
g_intptr_tl.clear();
201+
intptr_present = false;
202+
203+
}
204+
// 0xA0-0xFF: no-op padding, left for the mutator to grow/shrink sequences.
205+
}
206+
207+
// Whatever is left in the tracked slot when this thread exits is torn down
208+
// by the pthread key destructor, synchronized-with by the caller's join().
209+
if (tracked_present) {
210+
(*expected_frees)++;
211+
}
212+
}
213+
214+
} // namespace
215+
216+
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
217+
if (size == 0) return 0;
218+
219+
uint64_t expected_creates = 0;
220+
uint64_t expected_frees = 0;
221+
uint64_t create_before = g_create_count.load(std::memory_order_relaxed);
222+
uint64_t free_before = g_free_count.load(std::memory_order_relaxed);
223+
224+
try {
225+
std::thread worker(runOnWorkerThread, data, size, &expected_creates,
226+
&expected_frees);
227+
worker.join();
228+
} catch (const std::system_error &e) {
229+
// Transient thread-creation failure (e.g. resource exhaustion under a
230+
// heavily parallel fuzzer run) - not a bug in ThreadLocal itself.
231+
// Anything else (including a failure from join() itself) is unexpected
232+
// and should surface as a crash rather than be silently swallowed.
233+
if (e.code() == std::errc::resource_unavailable_try_again) {
234+
return 0;
235+
}
236+
throw;
237+
}
238+
239+
uint64_t create_after = g_create_count.load(std::memory_order_relaxed);
240+
uint64_t free_after = g_free_count.load(std::memory_order_relaxed);
241+
242+
if (create_after - create_before != expected_creates) {
243+
__builtin_trap(); // I2: create_tracked() ran the wrong number of times
244+
}
245+
if (free_after - free_before != expected_frees) {
246+
__builtin_trap(); // I2: free_tracked() ran the wrong number of times
247+
// (double free / leak from the TSD destructor path)
248+
}
249+
250+
return 0;
251+
}

utils/README.md

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -84,17 +84,21 @@ Runs tests in containers across various OS/libc/JDK combinations, mirroring the
8484
**Usage:**
8585
```bash
8686
./utils/run-containers-tests.sh [options]
87-
--libc=glibc|musl (default: glibc)
88-
--jdk=8|11|17|21|25|8-j9|... (default: 21)
89-
--arch=x64|aarch64 (default: auto-detect)
90-
--config=debug|release|asan|tsan (default: debug)
87+
--libc=glibc|musl|all[,..] (default: glibc)
88+
--jdk=8|11|17|21|25|8-j9|...|regular|j9|graal|all[,..]
89+
(default: 21)
90+
--arch=x64|aarch64|all[,..] (default: auto-detect)
91+
--config=debug|release|asan|tsan|all[,..] (default: debug)
9192
--container=podman|docker (default: podman)
9293
--tests="TestPattern" (optional)
9394
--gtest (enable C++ gtests)
9495
--gtest-task=Task (run one C++ gtest task)
9596
--shell (drop to shell instead of running tests)
9697
--mount (mount local repo instead of cloning)
9798
--rebuild (force rebuild of container images)
99+
--matrix (preview a full matrix)
100+
--run (execute an inferred matrix without prompting)
101+
--fail-fast (stop matrix execution on first failure)
98102
```
99103

100104
Examples:
@@ -104,8 +108,19 @@ Examples:
104108

105109
# Use Docker instead of the default Podman runtime
106110
./utils/run-containers-tests.sh --container=docker --libc=glibc --jdk=21
111+
112+
# Preview selected JDKs across every supported libc/architecture pair
113+
./utils/run-containers-tests.sh --libc=all --jdk=8,17,21 --arch=all
114+
115+
# Run all supported musl cells without an interactive prompt
116+
./utils/run-containers-tests.sh --matrix --libc=musl --run
117+
118+
# Run all OpenJ9 cells
119+
./utils/run-containers-tests.sh --matrix --jdk=j9 --run
107120
```
108121

122+
Single-value commands run one container test configuration immediately. When any dimension expands to multiple cells, the script prints a compact status table first; interactive terminals ask for confirmation, while non-interactive runs require `--run` to execute. Matrix execution prints the status table again after all cells finish and writes summaries to `build/reports/container-matrix/summary.md` and `build/reports/container-matrix/summary.json`. Matrix gtest runs require a short `--gtest-task` name so the task follows each cell's configuration. Cells not run because of `--fail-fast` are reported as cancelled separately from unsupported cells that are skipped.
123+
109124
### `patch-dd-java-agent.sh`
110125

111126
Patches a `dd-java-agent.jar` with a locally-built ddprof library for quick local testing without a full dd-trace-java rebuild.

0 commit comments

Comments
 (0)