Unset inherited JVM env vars in crashtracking scripts#10819
Unset inherited JVM env vars in crashtracking scripts#10819gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits intomasterfrom
Conversation
The OOME notifier and crash uploader scripts now unset JDK_JAVA_OPTIONS, JAVA_TOOL_OPTIONS, and _JAVA_OPTIONS before spawning a child JVM. This prevents port conflicts, memory contention, and lost diagnostics when the parent environment contains flags like JMX remote ports. Fixes #10766 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.074 s) : 0, 1073540
Total [baseline] (8.899 s) : 0, 8899003
Agent [candidate] (1.058 s) : 0, 1058446
Total [candidate] (8.826 s) : 0, 8825675
section iast
Agent [baseline] (1.24 s) : 0, 1239733
Total [baseline] (9.526 s) : 0, 9526028
Agent [candidate] (1.227 s) : 0, 1226811
Total [candidate] (9.521 s) : 0, 9521071
gantt
title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.239 ms) : 0, 1239
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (636.506 ms) : 0, 636506
BytebuddyAgent [candidate] (628.499 ms) : 0, 628499
AgentMeter [baseline] (29.558 ms) : 0, 29558
AgentMeter [candidate] (29.041 ms) : 0, 29041
GlobalTracer [baseline] (259.854 ms) : 0, 259854
GlobalTracer [candidate] (257.255 ms) : 0, 257255
AppSec [baseline] (32.126 ms) : 0, 32126
AppSec [candidate] (31.628 ms) : 0, 31628
Debugger [baseline] (60.266 ms) : 0, 60266
Debugger [candidate] (59.434 ms) : 0, 59434
Remote Config [baseline] (597.714 µs) : 0, 598
Remote Config [candidate] (590.234 µs) : 0, 590
Telemetry [baseline] (8.22 ms) : 0, 8220
Telemetry [candidate] (8.092 ms) : 0, 8092
Flare Poller [baseline] (8.761 ms) : 0, 8761
Flare Poller [candidate] (6.56 ms) : 0, 6560
section iast
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (804.684 ms) : 0, 804684
BytebuddyAgent [candidate] (795.641 ms) : 0, 795641
AgentMeter [baseline] (11.762 ms) : 0, 11762
AgentMeter [candidate] (11.371 ms) : 0, 11371
GlobalTracer [baseline] (249.38 ms) : 0, 249380
GlobalTracer [candidate] (247.369 ms) : 0, 247369
AppSec [baseline] (26.908 ms) : 0, 26908
AppSec [candidate] (26.524 ms) : 0, 26524
Debugger [baseline] (70.04 ms) : 0, 70040
Debugger [candidate] (69.432 ms) : 0, 69432
Remote Config [baseline] (541.254 µs) : 0, 541
Remote Config [candidate] (529.009 µs) : 0, 529
Telemetry [baseline] (9.672 ms) : 0, 9672
Telemetry [candidate] (9.705 ms) : 0, 9705
Flare Poller [baseline] (3.493 ms) : 0, 3493
Flare Poller [candidate] (3.461 ms) : 0, 3461
IAST [baseline] (25.647 ms) : 0, 25647
IAST [candidate] (25.348 ms) : 0, 25348
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067367
Total [baseline] (11.154 s) : 0, 11154038
Agent [candidate] (1.08 s) : 0, 1079654
Total [candidate] (11.066 s) : 0, 11066470
section appsec
Agent [baseline] (1.245 s) : 0, 1245443
Total [baseline] (11.184 s) : 0, 11184154
Agent [candidate] (1.246 s) : 0, 1245620
Total [candidate] (11.238 s) : 0, 11238104
section iast
Agent [baseline] (1.226 s) : 0, 1226168
Total [baseline] (11.273 s) : 0, 11273239
Agent [candidate] (1.227 s) : 0, 1226610
Total [candidate] (11.28 s) : 0, 11279800
section profiling
Agent [baseline] (1.181 s) : 0, 1181389
Total [baseline] (10.922 s) : 0, 10922123
Agent [candidate] (1.188 s) : 0, 1188324
Total [candidate] (11.013 s) : 0, 11013199
gantt
title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.209 ms) : 0, 1209
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (631.85 ms) : 0, 631850
BytebuddyAgent [candidate] (641.992 ms) : 0, 641992
AgentMeter [baseline] (29.523 ms) : 0, 29523
AgentMeter [candidate] (29.864 ms) : 0, 29864
GlobalTracer [baseline] (259.279 ms) : 0, 259279
GlobalTracer [candidate] (261.412 ms) : 0, 261412
AppSec [baseline] (32.251 ms) : 0, 32251
AppSec [candidate] (32.397 ms) : 0, 32397
Debugger [baseline] (60.952 ms) : 0, 60952
Debugger [candidate] (61.569 ms) : 0, 61569
Remote Config [baseline] (613.985 µs) : 0, 614
Remote Config [candidate] (613.275 µs) : 0, 613
Telemetry [baseline] (8.125 ms) : 0, 8125
Telemetry [candidate] (8.227 ms) : 0, 8227
Flare Poller [baseline] (7.337 ms) : 0, 7337
Flare Poller [candidate] (5.885 ms) : 0, 5885
section appsec
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (657.237 ms) : 0, 657237
BytebuddyAgent [candidate] (656.828 ms) : 0, 656828
AgentMeter [baseline] (12.092 ms) : 0, 12092
AgentMeter [candidate] (12.061 ms) : 0, 12061
GlobalTracer [baseline] (257.978 ms) : 0, 257978
GlobalTracer [candidate] (258.219 ms) : 0, 258219
IAST [baseline] (24.137 ms) : 0, 24137
IAST [candidate] (24.178 ms) : 0, 24178
AppSec [baseline] (177.689 ms) : 0, 177689
AppSec [candidate] (177.678 ms) : 0, 177678
Debugger [baseline] (66.252 ms) : 0, 66252
Debugger [candidate] (66.546 ms) : 0, 66546
Remote Config [baseline] (624.753 µs) : 0, 625
Remote Config [candidate] (627.781 µs) : 0, 628
Telemetry [baseline] (8.268 ms) : 0, 8268
Telemetry [candidate] (8.313 ms) : 0, 8313
Flare Poller [baseline] (3.588 ms) : 0, 3588
Flare Poller [candidate] (3.603 ms) : 0, 3603
section iast
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (794.541 ms) : 0, 794541
BytebuddyAgent [candidate] (795.323 ms) : 0, 795323
AgentMeter [baseline] (11.335 ms) : 0, 11335
AgentMeter [candidate] (11.328 ms) : 0, 11328
GlobalTracer [baseline] (247.294 ms) : 0, 247294
GlobalTracer [candidate] (247.103 ms) : 0, 247103
IAST [baseline] (25.319 ms) : 0, 25319
IAST [candidate] (25.345 ms) : 0, 25345
AppSec [baseline] (26.475 ms) : 0, 26475
AppSec [candidate] (26.465 ms) : 0, 26465
Debugger [baseline] (70.747 ms) : 0, 70747
Debugger [candidate] (70.654 ms) : 0, 70654
Remote Config [baseline] (531.11 µs) : 0, 531
Remote Config [candidate] (534.729 µs) : 0, 535
Telemetry [baseline] (9.247 ms) : 0, 9247
Telemetry [candidate] (9.215 ms) : 0, 9215
Flare Poller [baseline] (3.333 ms) : 0, 3333
Flare Poller [candidate] (3.355 ms) : 0, 3355
section profiling
crashtracking [baseline] (1.181 ms) : 0, 1181
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (681.657 ms) : 0, 681657
BytebuddyAgent [candidate] (686.524 ms) : 0, 686524
AgentMeter [baseline] (8.625 ms) : 0, 8625
AgentMeter [candidate] (8.73 ms) : 0, 8730
GlobalTracer [baseline] (215.272 ms) : 0, 215272
GlobalTracer [candidate] (216.73 ms) : 0, 216730
AppSec [baseline] (32.258 ms) : 0, 32258
AppSec [candidate] (32.483 ms) : 0, 32483
Debugger [baseline] (64.501 ms) : 0, 64501
Debugger [candidate] (66.288 ms) : 0, 66288
Remote Config [baseline] (590.626 µs) : 0, 591
Remote Config [candidate] (580.842 µs) : 0, 581
Telemetry [baseline] (9.257 ms) : 0, 9257
Telemetry [candidate] (7.692 ms) : 0, 7692
Flare Poller [baseline] (3.483 ms) : 0, 3483
Flare Poller [candidate] (3.427 ms) : 0, 3427
ProfilingAgent [baseline] (93.764 ms) : 0, 93764
ProfilingAgent [candidate] (93.571 ms) : 0, 93571
Profiling [baseline] (94.334 ms) : 0, 94334
Profiling [candidate] (94.13 ms) : 0, 94130
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 19 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section baseline
no_agent (18.098 ms) : 17913, 18283
. : milestone, 18098,
appsec (19.219 ms) : 19020, 19418
. : milestone, 19219,
code_origins (17.425 ms) : 17255, 17594
. : milestone, 17425,
iast (17.674 ms) : 17494, 17855
. : milestone, 17674,
profiling (18.393 ms) : 18211, 18575
. : milestone, 18393,
tracing (17.935 ms) : 17761, 18108
. : milestone, 17935,
section candidate
no_agent (17.747 ms) : 17564, 17930
. : milestone, 17747,
appsec (18.714 ms) : 18527, 18901
. : milestone, 18714,
code_origins (17.551 ms) : 17375, 17728
. : milestone, 17551,
iast (17.446 ms) : 17272, 17620
. : milestone, 17446,
profiling (18.541 ms) : 18351, 18731
. : milestone, 18541,
tracing (17.663 ms) : 17485, 17841
. : milestone, 17663,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section baseline
no_agent (1.173 ms) : 1162, 1184
. : milestone, 1173,
iast (3.174 ms) : 3132, 3216
. : milestone, 3174,
iast_FULL (5.947 ms) : 5886, 6007
. : milestone, 5947,
iast_GLOBAL (3.524 ms) : 3465, 3583
. : milestone, 3524,
profiling (1.93 ms) : 1914, 1945
. : milestone, 1930,
tracing (1.78 ms) : 1765, 1795
. : milestone, 1780,
section candidate
no_agent (1.176 ms) : 1164, 1187
. : milestone, 1176,
iast (3.183 ms) : 3141, 3225
. : milestone, 3183,
iast_FULL (5.947 ms) : 5887, 6007
. : milestone, 5947,
iast_GLOBAL (3.634 ms) : 3576, 3693
. : milestone, 3634,
profiling (2.192 ms) : 2171, 2213
. : milestone, 2192,
tracing (1.8 ms) : 1784, 1815
. : milestone, 1800,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section baseline
no_agent (1.467 ms) : 1456, 1478
. : milestone, 1467,
appsec (2.517 ms) : 2463, 2572
. : milestone, 2517,
iast (2.253 ms) : 2184, 2322
. : milestone, 2253,
iast_GLOBAL (2.291 ms) : 2222, 2361
. : milestone, 2291,
profiling (2.076 ms) : 2021, 2130
. : milestone, 2076,
tracing (2.056 ms) : 2003, 2109
. : milestone, 2056,
section candidate
no_agent (1.474 ms) : 1462, 1485
. : milestone, 1474,
appsec (3.811 ms) : 3589, 4032
. : milestone, 3811,
iast (2.257 ms) : 2188, 2326
. : milestone, 2257,
iast_GLOBAL (2.301 ms) : 2231, 2370
. : milestone, 2301,
profiling (2.101 ms) : 2045, 2158
. : milestone, 2101,
tracing (2.075 ms) : 2022, 2129
. : milestone, 2075,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~737420a170, baseline=1.61.0-SNAPSHOT~99d47ca835
dateFormat X
axisFormat %s
section baseline
no_agent (15.51 s) : 15510000, 15510000
. : milestone, 15510000,
appsec (14.818 s) : 14818000, 14818000
. : milestone, 14818000,
iast (18.288 s) : 18288000, 18288000
. : milestone, 18288000,
iast_GLOBAL (18.095 s) : 18095000, 18095000
. : milestone, 18095000,
profiling (15.091 s) : 15091000, 15091000
. : milestone, 15091000,
tracing (15.029 s) : 15029000, 15029000
. : milestone, 15029000,
section candidate
no_agent (15.625 s) : 15625000, 15625000
. : milestone, 15625000,
appsec (14.638 s) : 14638000, 14638000
. : milestone, 14638000,
iast (18.339 s) : 18339000, 18339000
. : milestone, 18339000,
iast_GLOBAL (17.79 s) : 17790000, 17790000
. : milestone, 17790000,
profiling (14.98 s) : 14980000, 14980000
. : milestone, 14980000,
tracing (14.951 s) : 14951000, 14951000
. : milestone, 14951000,
|
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
|
Hey @jbachorik, |
What Does This Do
Clears
JDK_JAVA_OPTIONS,JAVA_TOOL_OPTIONS, and_JAVA_OPTIONSenvironment variables before spawning a child JVM in the OOME notifier and crash uploader scripts (both.shand.batvariants). This prevents the child JVM from inheriting the parent's environment, which can cause port conflicts, memory contention, and lost diagnostics.Motivation
When the Datadog admission controller injects the agent via
JAVA_TOOL_OPTIONS, and the application sets JMX or memory flags viaJDK_JAVA_OPTIONS, the child JVM spawned bydd_oome_notifier.sh/dd_crash_uploader.shinherits these variables. This causes failures such asBindExceptionon JMX ports still held by the parent, or cgroup OOM kills from competing memory reservations — resulting in lost OOME/crash events.GH Ticket #10766
Additional Notes
notify_oome.sh,notify_oome.bat,upload_crash.sh,upload_crash.batupload_crash.*are coveredJDK_JAVA_OPTIONSwith JMX port-binding flags and asserting events are still receivedContributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueNote: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.