Skip to content

Commit 2c58b48

Browse files
authored
fix(profiler): guard line-number-table copy against stale jmethodID (#647)
1 parent 79db79a commit 2c58b48

5 files changed

Lines changed: 582 additions & 5 deletions

File tree

AGENTS.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -437,6 +437,19 @@ table: see [doc/build/JdkUpgrades.md](doc/build/JdkUpgrades.md).
437437
- All code needs to strive to be lean in terms of resources consumption and easy to follow -
438438
do not shy away from factoring out self containing code to shorter functions with explicit name
439439

440+
## CI / Automation Script Changes
441+
- Before rewriting or replacing a script invoked from `.gitlab-ci.yml`, `.gitlab/**/*.yml`, or any CI job,
442+
run `git log -p -- <script>` (or diff against the version being replaced) and enumerate every behavior
443+
branch it has — fallbacks, env-var-driven paths, error exits. Preserve all of them unless explicitly
444+
asked to drop one; do not silently narrow behavior while refactoring.
445+
- Grep the calling `.gitlab-ci.yml` / `.gitlab/**/*.yml` jobs for how the script is invoked (env vars set,
446+
exit codes expected) and confirm the rewrite still satisfies every caller, not just the common/local case.
447+
- When removing an item from a default list or config (an antagonist, a test tag, a feature flag), cite
448+
the exact file/line proving it is inert or unused in that context before removing it — never remove
449+
based on assumption alone.
450+
- After changing a CI-invoked script, run a syntax check (e.g. `bash -n <script>`) and trace through each
451+
CI job that calls it before considering the change complete.
452+
440453
### C/C++ Code Style
441454
- **Indentation**: Match the exact indentation style of the surrounding code in each file. Do not introduce inconsistent indentation — reviewers will flag it.
442455
- **Minimal complexity**: Do not split inline logic into separate helper functions unless the helpers are reused or the original is genuinely hard to follow. Unnecessary splits add indirection without value.

ddprof-lib/src/main/cpp/counters.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@
7474
X(UNWINDING_TIME_JVMTI, "unwinding_ticks_jvmti") \
7575
X(CALLTRACE_STORAGE_DROPPED, "calltrace_storage_dropped_traces") \
7676
X(LINE_NUMBER_TABLES, "line_number_tables") \
77+
X(LINE_NUMBER_TABLE_UNREADABLE, "line_number_table_unreadable") \
7778
X(REMOTE_SYMBOLICATION_FRAMES, "remote_symbolication_frames") \
7879
X(REMOTE_SYMBOLICATION_LIBS_WITH_BUILD_ID, "remote_symbolication_libs_with_build_id") \
7980
X(REMOTE_SYMBOLICATION_BUILD_ID_CACHE_HITS, "remote_symbolication_build_id_cache_hits") \

ddprof-lib/src/main/cpp/flightRecorder.cpp

Lines changed: 47 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,16 @@
4747
static const char *const SETTING_RING[] = {NULL, "kernel", "user", "any"};
4848
static const char *const SETTING_CSTACK[] = {NULL, "no", "fp", "dwarf", "lbr"};
4949

50+
// JVM spec SS4.7.3 caps a method's bytecode (code_length) at 65535 bytes (u2),
51+
// so a well-formed LineNumberTable can never have more entries than that.
52+
// Used to sanity-bound line_number_table_size before it drives the byte-count
53+
// passed to SafeAccess::safeCopy(): if GetLineNumberTable()
54+
// returns a corrupted pointer for a stale jmethodID (see the TOCTOU race
55+
// documented in fillJavaMethodInfo below), the paired out-param size is just
56+
// as likely to be corrupted, and an implausible size should be rejected
57+
// before it is trusted to compute a byte range.
58+
static const jint MAX_LINE_NUMBER_TABLE_ENTRIES = 65535;
59+
5060
// Compute a non-negative event duration from TSC timestamps. Unsigned u64
5161
// subtraction wraps to a near-2^64 value when end < start, which can happen if
5262
// the thread migrates cores between the two TSC reads and the per-core counters
@@ -360,18 +370,50 @@ void Lookup::fillJavaMethodInfo(MethodInfo *mi, jmethodID method,
360370
// the JVMTI-allocated memory immediately. This keeps _ptr valid even
361371
// after the underlying class is unloaded.
362372
void *owned_table = nullptr;
363-
if (line_number_table_size > 0) {
373+
if (line_number_table_size > 0 &&
374+
line_number_table_size <= MAX_LINE_NUMBER_TABLE_ENTRIES) {
364375
size_t bytes = (size_t)line_number_table_size * sizeof(jvmtiLineNumberEntry);
376+
// GetLineNumberTable() is called on the same possibly-stale jmethodID
377+
// that GetMethodDeclaringClass/GetClassSignature/GetMethodName above
378+
// were probed for -- the TOCTOU race documented above (class
379+
// unloaded between sample capture and dump) applies here just as
380+
// much as to those calls, and crash telemetry already showed those
381+
// sibling calls returning JVMTI_ERROR_NONE with unmapped string
382+
// pointers despite the spec saying the returned array should be a
383+
// fresh, caller-owned allocation. A genuinely-valid returned table is
384+
// fully decoupled from jmethodID lifetime per the JVMTI spec and
385+
// can't be invalidated later by class unload; the actual risk is a
386+
// corrupted pointer that happens to alias other live memory at
387+
// check-time and stops being mapped moments later. A separate
388+
// isReadableRange() probe followed by a plain memcpy() would still
389+
// race that window, so copy via safeCopy() instead: it fault-protects
390+
// each read as it happens rather than trusting a point-in-time check
391+
// before an unprotected copy.
365392
owned_table = malloc(bytes);
366393
if (owned_table != nullptr) {
367-
memcpy(owned_table, line_number_table, bytes);
394+
if (!SafeAccess::safeCopy(owned_table, line_number_table, bytes)) {
395+
free(owned_table);
396+
owned_table = nullptr;
397+
line_number_table = nullptr; // make sure the invalid address is not used for jvmti->Deallocate
398+
Counters::increment(LINE_NUMBER_TABLE_UNREADABLE);
399+
}
368400
} else {
369401
TEST_LOG("Failed to allocate %zu bytes for line number table copy", bytes);
370402
}
403+
} else if (line_number_table_size != 0) {
404+
// A corrupted size out-param alongside a corrupted pointer is exactly
405+
// as plausible as the corrupted-pointer case above (both come from
406+
// the same GetLineNumberTable() call on the same stale jmethodID);
407+
// an implausible entry count -- including a negative one, since this
408+
// is a signed jint and a corrupted value can fall on either side of
409+
// zero -- means the pointer can't be trusted for Deallocate() either,
410+
// so treat it the same as the unreadable case.
411+
line_number_table = nullptr;
412+
Counters::increment(LINE_NUMBER_TABLE_UNREADABLE);
371413
}
372-
jvmtiError dealloc_err = jvmti->Deallocate((unsigned char *)line_number_table);
373-
if (dealloc_err != JVMTI_ERROR_NONE) {
374-
TEST_LOG("Unexpected error while deallocating linenumber table: %d", dealloc_err);
414+
if (line_number_table != nullptr) {
415+
jvmtiError dealloc_err = jvmti->Deallocate((unsigned char *)line_number_table);
416+
assert(dealloc_err == JVMTI_ERROR_NONE && "Unexpected error while deallocating linenumber table");
375417
}
376418
if (owned_table != nullptr) {
377419
mi->_line_number_table = std::make_shared<SharedLineNumberTable>(
Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
/*
2+
* Copyright 2026, Datadog, Inc.
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
// Reproduces a crash reported in production:
7+
//
8+
// __memcpy_evex_unaligned_erms
9+
// Lookup::resolveMethod(_asgct_callframe&)
10+
// ...
11+
// Recording::writeStackTraces(Buffer*, Lookup*)
12+
// ...
13+
// Profiler::dump / FlightRecorder::dump
14+
//
15+
// Lookup::fillJavaMethodInfo() (inlined into resolveMethod() under -O2/-O3)
16+
// calls jvmtiEnv::GetLineNumberTable(method, ...) and then copied the
17+
// result with an unguarded:
18+
//
19+
// memcpy(owned_table, line_number_table, bytes);
20+
//
21+
// (flightRecorder.cpp now guards this copy with SafeAccess::safeCopy() -- see
22+
// below -- so the line numbers of the unguarded version are no longer present
23+
// in the file; this file's tests reproduce that removed pattern. An earlier
24+
// version of the fix used a separate SafeAccess::isReadableRange() probe
25+
// followed by a plain memcpy(), but that still left a check-then-use race:
26+
// nothing stops the probed memory from becoming invalid in the gap between
27+
// the probe and the copy. safeCopy() closes that gap by fault-protecting
28+
// each read as it happens instead of trusting a point-in-time check.)
29+
//
30+
// Unlike the class_name/method_name/method_sig strings returned by the
31+
// preceding JVMTI calls (which ARE probed with SafeAccess::isReadableRange
32+
// before use, see PR #537 / commit ef13aa29f), the line_number_table pointer
33+
// is used directly.
34+
//
35+
// Per the JVMTI spec, GetLineNumberTable() hands back a freshly-allocated,
36+
// caller-owned array (hence the Deallocate() call after the copy) that is
37+
// decoupled from the Method's lifetime -- a spec-compliant implementation
38+
// cannot invalidate it out from under the caller. The actual risk, per the
39+
// comments already in fillJavaMethodInfo, is the TOCTOU race documented
40+
// right there: "GetMethodDeclaringClass may return a jclass wrapping a
41+
// stale/garbage oop when the class was unloaded between sample capture and
42+
// dump" -- a race against class unloading, not tied to any one JVM vendor.
43+
// That same comment block notes crash telemetry showed GetClassSignature/
44+
// GetMethodName returning JVMTI_ERROR_NONE with unmapped string pointers
45+
// despite the spec saying they should be valid (a separate OpenJ9-specific
46+
// jmethodID-corruption mode is also handled a few lines above, but is not
47+
// the only source of this). The production crash motivating this file was
48+
// on a stock HotSpot JDK 11, confirming the race is not OpenJ9-specific.
49+
// GetLineNumberTable() is called on the exact same jmethodID as
50+
// GetMethodDeclaringClass/GetClassSignature/GetMethodName, with nothing
51+
// about it that would exempt it from that same observed failure mode.
52+
//
53+
// These tests isolate the copy step from JVMTI/JNI (which fillJavaMethodInfo
54+
// requires and which is impractical to fake in a plain gtest) by exercising
55+
// the exact copy pattern against a pointer whose backing memory is gone --
56+
// standing in for "GetLineNumberTable() returned a bad pointer" regardless
57+
// of the precise reason. The result at the memcpy call site is identical
58+
// either way, so this is sufficient to prove both the crash and the fix.
59+
60+
#include <gtest/gtest.h>
61+
#include <cstring>
62+
#include <signal.h>
63+
#include <sys/mman.h>
64+
#include <unistd.h>
65+
66+
#include "flightRecorder.h"
67+
#include "os.h"
68+
#include "safeAccess.h"
69+
70+
namespace {
71+
72+
// SafeAccess::isReadableRange()/isReadable() rely on SIGSEGV/SIGBUS handlers
73+
// registered via OS::replaceSigsegvHandler()/replaceSigbusHandler() to catch
74+
// the fault at a known trampoline address and turn it into a safe return
75+
// value (see safefetch_ut.cpp for the same pattern). The safefetch trampoline
76+
// can fault with either signal depending on platform, so both must be
77+
// installed; otherwise a fault delivered as the other signal is an ordinary
78+
// unhandled fault.
79+
void (*orig_segvHandler)(int signo, siginfo_t *siginfo, void *ucontext);
80+
void (*orig_busHandler)(int signo, siginfo_t *siginfo, void *ucontext);
81+
82+
void lineNumberTableSegvHandler(int signo, siginfo_t *siginfo, void *context) {
83+
if (!SafeAccess::handle_safefetch(signo, context)) {
84+
if (signo == SIGBUS) {
85+
if (orig_busHandler != nullptr) {
86+
orig_busHandler(signo, siginfo, context);
87+
}
88+
} else if (orig_segvHandler != nullptr) {
89+
orig_segvHandler(signo, siginfo, context);
90+
}
91+
}
92+
}
93+
94+
class LineNumberTableCopyTest : public ::testing::Test {
95+
protected:
96+
void SetUp() override {
97+
orig_segvHandler = OS::replaceSigsegvHandler(lineNumberTableSegvHandler);
98+
orig_busHandler = OS::replaceSigbusHandler(lineNumberTableSegvHandler);
99+
}
100+
101+
void TearDown() override {
102+
OS::replaceSigsegvHandler(orig_segvHandler);
103+
OS::replaceSigbusHandler(orig_busHandler);
104+
}
105+
};
106+
107+
// Allocates a page-sized region seeded with `count` jvmtiLineNumberEntry
108+
// records, standing in for a buffer jvmtiEnv::GetLineNumberTable() would
109+
// have returned.
110+
jvmtiLineNumberEntry *makeFakeLineNumberTable(void *page, int count) {
111+
jvmtiLineNumberEntry *table = (jvmtiLineNumberEntry *)page;
112+
for (int i = 0; i < count; i++) {
113+
table[i].start_location = i * 4;
114+
table[i].line_number = i + 1;
115+
}
116+
return table;
117+
}
118+
119+
} // namespace
120+
121+
// Reproducer: the code shape that used to be in flightRecorder.cpp's
122+
// fillJavaMethodInfo() before the fix, with no readability guard before the
123+
// memcpy. Demonstrates that once the source page is gone, the copy step used
124+
// by resolveMethod() crashes the process rather than failing gracefully.
125+
TEST(LineNumberTableCopyRawTest, UnguardedCopyCrashesWhenSourceUnmapped) {
126+
EXPECT_DEATH(
127+
{
128+
long page_size = sysconf(_SC_PAGESIZE);
129+
void *page = mmap(NULL, page_size, PROT_READ | PROT_WRITE,
130+
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
131+
if (page == MAP_FAILED) {
132+
_exit(1); // treat as death via non-zero exit if mmap itself fails
133+
}
134+
jint line_number_table_size = 4;
135+
jvmtiLineNumberEntry *line_number_table =
136+
makeFakeLineNumberTable(page, line_number_table_size);
137+
138+
// Stand-in for GetLineNumberTable() handing back a bad pointer for a
139+
// corrupted/stale jmethodID: the backing memory is simply gone by
140+
// the time the copy runs.
141+
munmap(page, page_size);
142+
143+
// This mirrors the pre-fix fillJavaMethodInfo() shape verbatim: no
144+
// readability check on line_number_table before the copy.
145+
size_t bytes =
146+
(size_t)line_number_table_size * sizeof(jvmtiLineNumberEntry);
147+
void *owned_table = malloc(bytes);
148+
memcpy(owned_table, line_number_table, bytes); // <-- crashes here
149+
// Force the copied bytes to be observed before free(); otherwise an
150+
// optimizing (Release) build can prove owned_table's contents are
151+
// never read and eliminate the memcpy as dead code, silently
152+
// skipping the very fault this test exists to demonstrate.
153+
volatile unsigned char sink = *(volatile unsigned char *)owned_table;
154+
(void)sink;
155+
free(owned_table);
156+
},
157+
"");
158+
}
159+
160+
// Documents the fix: copying via SafeAccess::safeCopy() (which fault-protects
161+
// each read as it happens, rather than a point-in-time isReadableRange()
162+
// probe followed by a separate, unprotected memcpy()) turns the crash into a
163+
// clean, detectable failure with no memory touched past the fault.
164+
TEST_F(LineNumberTableCopyTest, GuardedCopySkipsSafelyWhenSourceUnmapped) {
165+
long page_size = sysconf(_SC_PAGESIZE);
166+
void *page = mmap(NULL, page_size, PROT_READ | PROT_WRITE,
167+
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
168+
ASSERT_NE(page, MAP_FAILED);
169+
170+
jint line_number_table_size = 4;
171+
jvmtiLineNumberEntry *line_number_table =
172+
makeFakeLineNumberTable(page, line_number_table_size);
173+
174+
ASSERT_EQ(0, munmap(page, page_size));
175+
176+
size_t bytes = (size_t)line_number_table_size * sizeof(jvmtiLineNumberEntry);
177+
void *owned_table = malloc(bytes);
178+
ASSERT_NE(owned_table, nullptr);
179+
if (!SafeAccess::safeCopy(owned_table, line_number_table, bytes)) {
180+
free(owned_table);
181+
owned_table = nullptr;
182+
}
183+
184+
EXPECT_EQ(nullptr, owned_table);
185+
}
186+
187+
// Sanity check: the guard must not reject a genuinely valid table, or every
188+
// real dump would silently lose line-number info.
189+
TEST_F(LineNumberTableCopyTest, GuardedCopyStillWorksForValidSource) {
190+
jint line_number_table_size = 8;
191+
jvmtiLineNumberEntry stack_table[8];
192+
jvmtiLineNumberEntry *line_number_table =
193+
makeFakeLineNumberTable(stack_table, line_number_table_size);
194+
195+
size_t bytes = (size_t)line_number_table_size * sizeof(jvmtiLineNumberEntry);
196+
void *owned_table = malloc(bytes);
197+
ASSERT_NE(owned_table, nullptr);
198+
if (!SafeAccess::safeCopy(owned_table, line_number_table, bytes)) {
199+
free(owned_table);
200+
owned_table = nullptr;
201+
}
202+
203+
ASSERT_NE(nullptr, owned_table);
204+
EXPECT_EQ(0, memcmp(owned_table, line_number_table, bytes));
205+
free(owned_table);
206+
}

0 commit comments

Comments
 (0)