Skip to content

Add unified profiling feature gate config keys#11090

Draft
jbachorik wants to merge 14 commits into
masterfrom
jb/prof-10633-unified-config
Draft

Add unified profiling feature gate config keys#11090
jbachorik wants to merge 14 commits into
masterfrom
jb/prof-10633-unified-config

Conversation

@jbachorik

@jbachorik jbachorik commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Adds unified profiling.<feature>.enabled configuration keys for all GA profiling features, following the pattern established by profiling.heap.enabled:

  • profiling.cpu.enabled (default: true) — gates CPU profiling for both JFR (jdk.ExecutionSample, jdk.NativeMethodSample, jdk.CPUTimeSample) and ddprof
  • profiling.walltime.enabled (default: true) — gates wall-clock profiling for ddprof (JFR timeline events are intentionally excluded — they serve queueing-time and blocking-event purposes beyond wall-clock profiling)
  • profiling.exception.enabled (default: true) — gates JFR exception profiling (datadog.ExceptionSample, datadog.ExceptionCount)
  • profiling.io.enabled (default: true) — gates JFR I/O profiling (jdk.FileRead/Write/Force, jdk.SocketRead/Write)
  • profiling.lock.enabled (default: true) — gates JFR lock profiling (jdk.JavaMonitorEnter, jdk.BiasedLock*)
  • profiling.thread.enabled (default: true) — gates JFR thread lifecycle profiling (jdk.ThreadStart/End)
  • profiling.direct.memory.enabled (default: false) — unified replacement for deprecated profiling.directallocation.enabled; old key still works as fallback

Also:

  • Adds PROFILING_ALLOCATION_ENABLED_DEFAULT = true constant (was dynamic-only before)
  • Updates DD_PROFILING_HEAP_ENABLED and DD_PROFILING_ALLOCATION_ENABLED metadata defaults to "true"
  • Deprecates PROFILING_HEAP_ENABLED_DEFAULT (dynamic default) and PROFILING_DIRECT_ALLOCATION_ENABLED constants
  • Adds === Feature Gates === section to profiler flare report

Motivation

Users had no consistent, feature-level way to disable individual profiling modes without reaching into backend-specific knobs (profiling.ddprof.cpu.enabled, profiling.ddprof.wall.enabled, etc.). The new unified keys provide a single toggle per feature that applies to both JFR and ddprof backends.

Configuration Design

Two-tier gate hierarchy

Each profiling feature is gated at two levels. The unified key is checked first; if it passes, backend-specific keys apply their own inner overrides.

flowchart TD
    PROF["dd.profiling.enabled = true"]

    PROF --> CPU
    PROF --> WALL
    PROF --> EXC
    PROF --> IO
    PROF --> LOCK
    PROF --> THR
    PROF --> DM

    subgraph CPU["profiling.cpu.enabled - default: true"]
        CPU_JFR["JFR: ExecutionSample, NativeMethodSample,<br>CPUTimeSample, CPUTimeSamplesLost"]
        CPU_DDPROF["ddprof: profiling.ddprof.cpu.enabled"]
    end

    subgraph WALL["profiling.walltime.enabled - default: true"]
        WALL_DDPROF["ddprof: profiling.ddprof.wall.enabled"]
        WALL_NOTE["Note: JFR timeline events are NOT gated here -<br>they serve queueing-time purposes independently"]
    end

    subgraph EXC["profiling.exception.enabled - default: true"]
        EXC_JFR["JFR: ExceptionSample, ExceptionCount"]
    end

    subgraph IO["profiling.io.enabled - default: true"]
        IO_JFR["JFR: FileRead, FileWrite, FileForce,<br>SocketRead, SocketWrite"]
    end

    subgraph LOCK["profiling.lock.enabled - default: true"]
        LOCK_JFR["JFR: JavaMonitorEnter, BiasedLock"]
    end

    subgraph THR["profiling.thread.enabled - default: true"]
        THR_JFR["JFR: ThreadStart, ThreadEnd"]
    end

    subgraph DM["profiling.direct.memory.enabled - default: false"]
        DM_INST["ByteBuffer.allocateDirect,<br>DirectByteBuffer ctor,<br>FileChannelImpl.map"]
    end
Loading

Config resolution for dd.profiling.direct.memory.enabled

This key replaces the deprecated dd.profiling.directallocation.enabled. The old key is honoured as an alias so existing deployments are unaffected.

Additional Notes

  • All new keys default to true so existing deployments are unaffected
  • profiling.direct.memory.enabled defaults to false to preserve backward compatibility with profiling.directallocation.enabled
  • The backend-specific keys (profiling.ddprof.cpu.enabled, etc.) remain as inner overrides for advanced use
  • Gates are evaluated once at recording start; profiling does not use Remote Config, so a disabled feature stays disabled for the recording lifetime

Contributor Checklist

Jira ticket: PROF-10633

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@jbachorik jbachorik added tag: ai generated Largely based on code generated by an AI or LLM comp: profiling Profiling type: feature Enhancements and improvements labels Apr 13, 2026
@pr-commenter

pr-commenter Bot commented Apr 13, 2026

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/prof-10633-unified-config
git_commit_date 1777548049 1777547455
git_commit_sha f89a0b26cc accce8d
release_version 1.62.0-SNAPSHOT~9f89a0b26cc 1.62.0-SNAPSHOT~accce8db9c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1777549343 1777549343
ci_job_id 1645748652 1645748652
ci_pipeline_id 110668361 110668361
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-c0as65li 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-c0as65li 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics.

Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1065445
Total [baseline] (11.086 s) : 0, 11085691
Agent [candidate] (1.067 s) : 0, 1067122
Total [candidate] (11.115 s) : 0, 11114688
section appsec
Agent [baseline] (1.258 s) : 0, 1258380
Total [baseline] (11.198 s) : 0, 11198330
Agent [candidate] (1.248 s) : 0, 1247732
Total [candidate] (11.051 s) : 0, 11050713
section iast
Agent [baseline] (1.244 s) : 0, 1244055
Total [baseline] (11.242 s) : 0, 11242384
Agent [candidate] (1.234 s) : 0, 1233506
Total [candidate] (11.23 s) : 0, 11229890
section profiling
Agent [baseline] (1.201 s) : 0, 1201111
Total [baseline] (11.15 s) : 0, 11149600
Agent [candidate] (1.186 s) : 0, 1186076
Total [candidate] (11.099 s) : 0, 11098940
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent appsec 1.258 s 192.935 ms (18.1%)
Agent iast 1.244 s 178.611 ms (16.8%)
Agent profiling 1.201 s 135.666 ms (12.7%)
Total tracing 11.086 s -
Total appsec 11.198 s 112.64 ms (1.0%)
Total iast 11.242 s 156.693 ms (1.4%)
Total profiling 11.15 s 63.909 ms (0.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent appsec 1.248 s 180.61 ms (16.9%)
Agent iast 1.234 s 166.384 ms (15.6%)
Agent profiling 1.186 s 118.953 ms (11.1%)
Total tracing 11.115 s -
Total appsec 11.051 s -63.975 ms (-0.6%)
Total iast 11.23 s 115.202 ms (1.0%)
Total profiling 11.099 s -15.748 ms (-0.1%)
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.25 ms) : 0, 1250
BytebuddyAgent [baseline] (638.409 ms) : 0, 638409
BytebuddyAgent [candidate] (637.124 ms) : 0, 637124
AgentMeter [baseline] (29.726 ms) : 0, 29726
AgentMeter [candidate] (29.63 ms) : 0, 29630
GlobalTracer [baseline] (250.989 ms) : 0, 250989
GlobalTracer [candidate] (251.136 ms) : 0, 251136
AppSec [baseline] (32.401 ms) : 0, 32401
AppSec [candidate] (32.641 ms) : 0, 32641
Debugger [baseline] (60.619 ms) : 0, 60619
Debugger [candidate] (60.843 ms) : 0, 60843
Remote Config [baseline] (609.504 µs) : 0, 610
Remote Config [candidate] (605.473 µs) : 0, 605
Telemetry [baseline] (8.255 ms) : 0, 8255
Telemetry [candidate] (8.325 ms) : 0, 8325
Flare Poller [baseline] (6.853 ms) : 0, 6853
Flare Poller [candidate] (9.1 ms) : 0, 9100
section appsec
crashtracking [baseline] (1.255 ms) : 0, 1255
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (667.57 ms) : 0, 667570
BytebuddyAgent [candidate] (661.085 ms) : 0, 661085
AgentMeter [baseline] (12.054 ms) : 0, 12054
AgentMeter [candidate] (12.108 ms) : 0, 12108
GlobalTracer [baseline] (250.804 ms) : 0, 250804
GlobalTracer [candidate] (249.018 ms) : 0, 249018
IAST [baseline] (25.005 ms) : 0, 25005
IAST [candidate] (24.539 ms) : 0, 24539
AppSec [baseline] (185.848 ms) : 0, 185848
AppSec [candidate] (184.528 ms) : 0, 184528
Debugger [baseline] (66.42 ms) : 0, 66420
Debugger [candidate] (66.052 ms) : 0, 66052
Remote Config [baseline] (606.827 µs) : 0, 607
Remote Config [candidate] (604.404 µs) : 0, 604
Telemetry [baseline] (8.645 ms) : 0, 8645
Telemetry [candidate] (8.583 ms) : 0, 8583
Flare Poller [baseline] (3.567 ms) : 0, 3567
Flare Poller [candidate] (3.589 ms) : 0, 3589
section iast
crashtracking [baseline] (1.237 ms) : 0, 1237
crashtracking [candidate] (1.24 ms) : 0, 1240
BytebuddyAgent [baseline] (815.222 ms) : 0, 815222
BytebuddyAgent [candidate] (808.387 ms) : 0, 808387
AgentMeter [baseline] (11.611 ms) : 0, 11611
AgentMeter [candidate] (11.495 ms) : 0, 11495
GlobalTracer [baseline] (243.101 ms) : 0, 243101
GlobalTracer [candidate] (240.671 ms) : 0, 240671
IAST [baseline] (27.065 ms) : 0, 27065
IAST [candidate] (25.715 ms) : 0, 25715
AppSec [baseline] (30.486 ms) : 0, 30486
AppSec [candidate] (31.124 ms) : 0, 31124
Debugger [baseline] (61.804 ms) : 0, 61804
Debugger [candidate] (60.825 ms) : 0, 60825
Remote Config [baseline] (541.36 µs) : 0, 541
Remote Config [candidate] (519.938 µs) : 0, 520
Telemetry [baseline] (13.034 ms) : 0, 13034
Telemetry [candidate] (13.58 ms) : 0, 13580
Flare Poller [baseline] (3.515 ms) : 0, 3515
Flare Poller [candidate] (3.444 ms) : 0, 3444
section profiling
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (702.017 ms) : 0, 702017
BytebuddyAgent [candidate] (692.206 ms) : 0, 692206
AgentMeter [baseline] (9.225 ms) : 0, 9225
AgentMeter [candidate] (9.123 ms) : 0, 9123
GlobalTracer [baseline] (210.549 ms) : 0, 210549
GlobalTracer [candidate] (207.685 ms) : 0, 207685
AppSec [baseline] (32.937 ms) : 0, 32937
AppSec [candidate] (32.481 ms) : 0, 32481
Debugger [baseline] (66.249 ms) : 0, 66249
Debugger [candidate] (65.622 ms) : 0, 65622
Remote Config [baseline] (584.674 µs) : 0, 585
Remote Config [candidate] (568.534 µs) : 0, 569
Telemetry [baseline] (7.878 ms) : 0, 7878
Telemetry [candidate] (7.823 ms) : 0, 7823
Flare Poller [baseline] (3.575 ms) : 0, 3575
Flare Poller [candidate] (3.549 ms) : 0, 3549
ProfilingAgent [baseline] (94.639 ms) : 0, 94639
ProfilingAgent [candidate] (94.522 ms) : 0, 94522
Profiling [baseline] (95.203 ms) : 0, 95203
Profiling [candidate] (95.083 ms) : 0, 95083
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062907
Total [baseline] (8.851 s) : 0, 8851164
Agent [candidate] (1.061 s) : 0, 1061260
Total [candidate] (8.885 s) : 0, 8884676
section iast
Agent [baseline] (1.226 s) : 0, 1225716
Total [baseline] (9.565 s) : 0, 9565010
Agent [candidate] (1.225 s) : 0, 1225312
Total [candidate] (9.563 s) : 0, 9562915
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.226 s 162.809 ms (15.3%)
Total tracing 8.851 s -
Total iast 9.565 s 713.846 ms (8.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.225 s 164.052 ms (15.5%)
Total tracing 8.885 s -
Total iast 9.563 s 678.239 ms (7.6%)
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.262 ms) : 0, 1262
crashtracking [candidate] (1.236 ms) : 0, 1236
BytebuddyAgent [baseline] (635.604 ms) : 0, 635604
BytebuddyAgent [candidate] (637.811 ms) : 0, 637811
AgentMeter [baseline] (29.656 ms) : 0, 29656
AgentMeter [candidate] (29.512 ms) : 0, 29512
GlobalTracer [baseline] (249.642 ms) : 0, 249642
GlobalTracer [candidate] (249.001 ms) : 0, 249001
AppSec [baseline] (32.045 ms) : 0, 32045
AppSec [candidate] (32.003 ms) : 0, 32003
Debugger [baseline] (59.137 ms) : 0, 59137
Debugger [candidate] (59.208 ms) : 0, 59208
Remote Config [baseline] (595.751 µs) : 0, 596
Remote Config [candidate] (614.353 µs) : 0, 614
Telemetry [baseline] (8.073 ms) : 0, 8073
Telemetry [candidate] (8.045 ms) : 0, 8045
Flare Poller [baseline] (10.643 ms) : 0, 10643
Flare Poller [candidate] (7.479 ms) : 0, 7479
section iast
crashtracking [baseline] (1.24 ms) : 0, 1240
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (802.104 ms) : 0, 802104
BytebuddyAgent [candidate] (802.284 ms) : 0, 802284
AgentMeter [baseline] (11.364 ms) : 0, 11364
AgentMeter [candidate] (11.37 ms) : 0, 11370
GlobalTracer [baseline] (239.682 ms) : 0, 239682
GlobalTracer [candidate] (239.977 ms) : 0, 239977
IAST [baseline] (25.903 ms) : 0, 25903
IAST [candidate] (25.916 ms) : 0, 25916
AppSec [baseline] (27.993 ms) : 0, 27993
AppSec [candidate] (32.59 ms) : 0, 32590
Debugger [baseline] (64.174 ms) : 0, 64174
Debugger [candidate] (58.051 ms) : 0, 58051
Remote Config [baseline] (567.9 µs) : 0, 568
Remote Config [candidate] (1.129 ms) : 0, 1129
Telemetry [baseline] (12.573 ms) : 0, 12573
Telemetry [candidate] (12.901 ms) : 0, 12901
Flare Poller [baseline] (3.574 ms) : 0, 3574
Flare Poller [candidate] (3.429 ms) : 0, 3429
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/prof-10633-unified-config
git_commit_date 1777548049 1777547455
git_commit_sha f89a0b26cc accce8d
release_version 1.62.0-SNAPSHOT~9f89a0b26cc 1.62.0-SNAPSHOT~accce8db9c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1777549824 1777549824
ci_job_id 1645748653 1645748653
ci_pipeline_id 110668361 110668361
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-yrhzq2au 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-yrhzq2au 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 19 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:profiling:high_load worse
[+35.528µs; +178.183µs] or [+2.099%; +10.527%]
unstable
[+21.313µs; +995.286µs] or [+0.435%; +20.334%]
unstable
[-425.497op/s; +110.122op/s] or [-19.712%; +5.102%]
1.800ms 5.403ms 2000.906op/s 1.693ms 4.895ms 2158.594op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.771 ms) : 18577, 18966
.   : milestone, 18771,
appsec (18.619 ms) : 18434, 18804
.   : milestone, 18619,
code_origins (18.131 ms) : 17950, 18313
.   : milestone, 18131,
iast (17.885 ms) : 17709, 18060
.   : milestone, 17885,
profiling (18.637 ms) : 18453, 18821
.   : milestone, 18637,
tracing (17.822 ms) : 17649, 17995
.   : milestone, 17822,
section candidate
no_agent (19.026 ms) : 18832, 19220
.   : milestone, 19026,
appsec (18.868 ms) : 18677, 19058
.   : milestone, 18868,
code_origins (17.706 ms) : 17530, 17882
.   : milestone, 17706,
iast (18.425 ms) : 18243, 18607
.   : milestone, 18425,
profiling (19.187 ms) : 18994, 19379
.   : milestone, 19187,
tracing (17.975 ms) : 17797, 18152
.   : milestone, 17975,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.771 ms [18.577 ms, 18.966 ms] -
appsec 18.619 ms [18.434 ms, 18.804 ms] -151.951 µs (-0.8%)
code_origins 18.131 ms [17.95 ms, 18.313 ms] -639.864 µs (-3.4%)
iast 17.885 ms [17.709 ms, 18.06 ms] -886.735 µs (-4.7%)
profiling 18.637 ms [18.453 ms, 18.821 ms] -134.604 µs (-0.7%)
tracing 17.822 ms [17.649 ms, 17.995 ms] -949.423 µs (-5.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.026 ms [18.832 ms, 19.22 ms] -
appsec 18.868 ms [18.677 ms, 19.058 ms] -158.282 µs (-0.8%)
code_origins 17.706 ms [17.53 ms, 17.882 ms] -1.32 ms (-6.9%)
iast 18.425 ms [18.243 ms, 18.607 ms] -601.2 µs (-3.2%)
profiling 19.187 ms [18.994 ms, 19.379 ms] 160.611 µs (0.8%)
tracing 17.975 ms [17.797 ms, 18.152 ms] -1.051 ms (-5.5%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.235 ms) : 1225, 1246
.   : milestone, 1235,
iast (3.386 ms) : 3339, 3433
.   : milestone, 3386,
iast_FULL (6.102 ms) : 6039, 6165
.   : milestone, 6102,
iast_GLOBAL (3.601 ms) : 3538, 3664
.   : milestone, 3601,
profiling (2.094 ms) : 2073, 2115
.   : milestone, 2094,
tracing (1.892 ms) : 1875, 1908
.   : milestone, 1892,
section candidate
no_agent (1.274 ms) : 1261, 1286
.   : milestone, 1274,
iast (3.423 ms) : 3376, 3471
.   : milestone, 3423,
iast_FULL (6.182 ms) : 6118, 6245
.   : milestone, 6182,
iast_GLOBAL (3.687 ms) : 3624, 3750
.   : milestone, 3687,
profiling (2.264 ms) : 2241, 2286
.   : milestone, 2264,
tracing (1.864 ms) : 1848, 1880
.   : milestone, 1864,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.235 ms [1.225 ms, 1.246 ms] -
iast 3.386 ms [3.339 ms, 3.433 ms] 2.15 ms (174.1%)
iast_FULL 6.102 ms [6.039 ms, 6.165 ms] 4.867 ms (393.9%)
iast_GLOBAL 3.601 ms [3.538 ms, 3.664 ms] 2.365 ms (191.5%)
profiling 2.094 ms [2.073 ms, 2.115 ms] 858.314 µs (69.5%)
tracing 1.892 ms [1.875 ms, 1.908 ms] 656.347 µs (53.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.274 ms [1.261 ms, 1.286 ms] -
iast 3.423 ms [3.376 ms, 3.471 ms] 2.15 ms (168.8%)
iast_FULL 6.182 ms [6.118 ms, 6.245 ms] 4.908 ms (385.4%)
iast_GLOBAL 3.687 ms [3.624 ms, 3.75 ms] 2.413 ms (189.5%)
profiling 2.264 ms [2.241 ms, 2.286 ms] 989.979 µs (77.7%)
tracing 1.864 ms [1.848 ms, 1.88 ms] 590.59 µs (46.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/prof-10633-unified-config
git_commit_date 1777548049 1777547455
git_commit_sha f89a0b26cc accce8d
release_version 1.62.0-SNAPSHOT~9f89a0b26cc 1.62.0-SNAPSHOT~accce8db9c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1777549561 1777549561
ci_job_id 1645748654 1645748654
ci_pipeline_id 110668361 110668361
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-m272bpwg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-m272bpwg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (3.812 ms) : 3591, 4034
.   : milestone, 3812,
iast (2.272 ms) : 2202, 2341
.   : milestone, 2272,
iast_GLOBAL (2.307 ms) : 2237, 2377
.   : milestone, 2307,
profiling (2.09 ms) : 2035, 2144
.   : milestone, 2090,
tracing (2.072 ms) : 2019, 2125
.   : milestone, 2072,
section candidate
no_agent (1.484 ms) : 1473, 1496
.   : milestone, 1484,
appsec (3.811 ms) : 3590, 4032
.   : milestone, 3811,
iast (2.262 ms) : 2193, 2331
.   : milestone, 2262,
iast_GLOBAL (2.314 ms) : 2245, 2384
.   : milestone, 2314,
profiling (2.094 ms) : 2039, 2149
.   : milestone, 2094,
tracing (2.084 ms) : 2030, 2138
.   : milestone, 2084,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.469 ms, 1.492 ms] -
appsec 3.812 ms [3.591 ms, 4.034 ms] 2.331 ms (157.5%)
iast 2.272 ms [2.202 ms, 2.341 ms] 790.889 µs (53.4%)
iast_GLOBAL 2.307 ms [2.237 ms, 2.377 ms] 825.989 µs (55.8%)
profiling 2.09 ms [2.035 ms, 2.144 ms] 608.832 µs (41.1%)
tracing 2.072 ms [2.019 ms, 2.125 ms] 591.122 µs (39.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.484 ms [1.473 ms, 1.496 ms] -
appsec 3.811 ms [3.59 ms, 4.032 ms] 2.327 ms (156.8%)
iast 2.262 ms [2.193 ms, 2.331 ms] 777.779 µs (52.4%)
iast_GLOBAL 2.314 ms [2.245 ms, 2.384 ms] 829.962 µs (55.9%)
profiling 2.094 ms [2.039 ms, 2.149 ms] 609.719 µs (41.1%)
tracing 2.084 ms [2.03 ms, 2.138 ms] 600.195 µs (40.4%)
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~accce8db9c, baseline=1.62.0-SNAPSHOT~9f89a0b26cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.54 s) : 15540000, 15540000
.   : milestone, 15540000,
appsec (14.759 s) : 14759000, 14759000
.   : milestone, 14759000,
iast (18.413 s) : 18413000, 18413000
.   : milestone, 18413000,
iast_GLOBAL (17.816 s) : 17816000, 17816000
.   : milestone, 17816000,
profiling (15.029 s) : 15029000, 15029000
.   : milestone, 15029000,
tracing (14.867 s) : 14867000, 14867000
.   : milestone, 14867000,
section candidate
no_agent (15.602 s) : 15602000, 15602000
.   : milestone, 15602000,
appsec (14.967 s) : 14967000, 14967000
.   : milestone, 14967000,
iast (18.278 s) : 18278000, 18278000
.   : milestone, 18278000,
iast_GLOBAL (17.958 s) : 17958000, 17958000
.   : milestone, 17958000,
profiling (15.094 s) : 15094000, 15094000
.   : milestone, 15094000,
tracing (14.847 s) : 14847000, 14847000
.   : milestone, 14847000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.54 s [15.54 s, 15.54 s] -
appsec 14.759 s [14.759 s, 14.759 s] -781.0 ms (-5.0%)
iast 18.413 s [18.413 s, 18.413 s] 2.873 s (18.5%)
iast_GLOBAL 17.816 s [17.816 s, 17.816 s] 2.276 s (14.6%)
profiling 15.029 s [15.029 s, 15.029 s] -511.0 ms (-3.3%)
tracing 14.867 s [14.867 s, 14.867 s] -673.0 ms (-4.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.602 s [15.602 s, 15.602 s] -
appsec 14.967 s [14.967 s, 14.967 s] -635.0 ms (-4.1%)
iast 18.278 s [18.278 s, 18.278 s] 2.676 s (17.2%)
iast_GLOBAL 17.958 s [17.958 s, 17.958 s] 2.356 s (15.1%)
profiling 15.094 s [15.094 s, 15.094 s] -508.0 ms (-3.3%)
tracing 14.847 s [14.847 s, 14.847 s] -755.0 ms (-4.8%)

jbachorik and others added 3 commits April 13, 2026 19:07
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Deprecate misleading PROFILING_ALLOCATION_ENABLED_DEFAULT constant
- Update PROFILING_HEAP_ENABLED_DEFAULT javadoc to explain dynamic default
- Fix PROFILING_DIRECT_ALLOCATION_ENABLED missing @deprecated javadoc
- Drop JavaMonitorWait/ThreadPark/ThreadSleep from wall gate (timeline events)
- Add startup-only/no-RC comment to feature gate block in OpenJdkController
- Add BiasedLock noop comment for JDK 18+ awareness
- Add lock sampler note to DatadogProfilerConfig
- Extract DirectMemoryProfilingHelper shared utility
- Add gate tests: CPU, exception, I/O, lock, thread, wall-timeline
- Set metadata defaults to false (safe fallback for dynamic defaults)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbachorik jbachorik force-pushed the jb/prof-10633-unified-config branch from a020726 to d19b09b Compare April 13, 2026 17:08
@jbachorik jbachorik marked this pull request as ready for review April 14, 2026 13:46
@jbachorik jbachorik requested review from a team as code owners April 14, 2026 13:46
@jbachorik jbachorik requested review from dougqh and removed request for a team April 14, 2026 13:46
@jbachorik jbachorik requested a review from anandreshmi46 April 21, 2026 17:38
"aliases": []
}
],
"DD_PROFILING_DIRECTALLOCATION_ENABLED": [

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this is a duplicate entry? A new entry with alias is already present

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed — removed the duplicate entry. The alias was added to the existing DD_PROFILING_DIRECTALLOCATION_ENABLED rather than promoting DD_PROFILING_DIRECT_MEMORY_ENABLED to primary, because the feature-parity dashboard does not permit renaming an existing key.


public static final String PROFILING_CPU_ENABLED = "profiling.cpu.enabled";
public static final boolean PROFILING_CPU_ENABLED_DEFAULT = true;
public static final String PROFILING_WALL_ENABLED = "profiling.walltime.enabled";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All other constant mirrors config keys, should the constant here also should be renamed to PROFILING_WALLTIME_ENABLED? In supported-configurations.json its PROFILING_WALLTIME_ENABLED

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Should be WALLTIME

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed in a later commit — the constant is now PROFILING_WALLTIME_ENABLED, matching the metadata key.

Comment thread metadata/supported-configurations.json Outdated
"aliases": []
}
],
"DD_PROFILING_DIRECTALLOCATION_ENABLED": [

@anandreshmi46 anandreshmi46 Apr 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the alias direction is inverted , the rest of this file always makes the new key primary and the old key the alias. Since PROFILING_DIRECT_MEMORY_ENABLED is the new replacement and PROFILING_DIRECT_ALLOCATION_ENABLED is Deprecated, I think this should be:
"DD_PROFILING_DIRECT_MEMORY_ENABLED": [{ "aliases": ["DD_PROFILING_DIRECTALLOCATION_ENABLED"] }]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I agree, this looks odd. However, I was not able to pass the validation of the config flags, IIRC.
I can try again, but I thing the config telemetry was not happy about removing the top level DD_PROFILING_DIRECTALLOCATION_ENABLED and keeping it as an alias or something.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying something to make this less confusing; let's see what system tests will tell us

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I know. I was not able to add a new key with alias to an old key in the feature parity dashboard.
I tried the change but the validation fails - https://gitlab.ddbuild.io/datadog/apm-reliability/dd-trace-java/builds/1619017677

I will revert the debug commit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duplicate entry is now removed. We kept DD_PROFILING_DIRECTALLOCATION_ENABLED as the primary key (with DD_PROFILING_DIRECT_MEMORY_ENABLED as its alias) because the feature-parity dashboard does not allow promoting a new key over an existing one.

@jbachorik jbachorik force-pushed the jb/prof-10633-unified-config branch from 13a2570 to 3ead772 Compare April 22, 2026 12:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces unified, feature-level profiling gates (profiling.<feature>.enabled) to consistently enable/disable GA profiling features across JFR and ddprof backends, plus exposes these gates in profiler diagnostics.

Changes:

  • Add new unified config keys/constants for CPU, walltime, exception, I/O, lock, thread, and direct-memory profiling gates.
  • Wire the new gates into ddprof config evaluation, OpenJDK JFR event settings, and direct-memory instrumentation enablement (with fallback to deprecated direct-allocation flag).
  • Extend profiler flare reporting and OpenJDK controller tests to cover the new gating behavior.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
metadata/supported-configurations.json Adds metadata entries for the new unified gate env vars (but contains a duplicate key + alias issues).
dd-trace-api/src/main/java/datadog/trace/api/config/ProfilingConfig.java Adds unified gate constants and deprecations for old/dynamic defaults (but has broken Javadoc links).
dd-java-agent/instrumentation/java/java-nio-1.8/.../FileChannelImplInstrumentation.java Switches enablement check to unified direct-memory gate with deprecated-key fallback.
dd-java-agent/instrumentation/java/java-nio-1.8/.../DirectByteBufferInstrumentation.java Switches enablement check to unified direct-memory gate with deprecated-key fallback.
dd-java-agent/instrumentation/java/java-nio-1.8/.../ByteBufferInstrumentation.java Switches enablement check to unified direct-memory gate with deprecated-key fallback.
dd-java-agent/agent-profiling/profiling-ddprof/.../DatadogProfilerConfig.java Applies unified CPU + walltime gates before backend-specific ddprof flags.
dd-java-agent/agent-profiling/profiling-controller/.../ProfilerSettingsSupport.java Captures unified gate values into controller settings for diagnostics/toString.
dd-java-agent/agent-profiling/profiling-controller/.../ProfilerFlareReporter.java Adds a “Feature Gates” section reporting unified gate state (with direct-memory fallback logic).
dd-java-agent/agent-profiling/profiling-controller-openjdk/src/test/.../OpenJdkControllerTest.java Adds tests asserting unified gates disable the corresponding JFR events (and walltime does not disable timeline events).
dd-java-agent/agent-profiling/profiling-controller-openjdk/src/main/.../OpenJdkController.java Disables relevant JFR events when unified gates are turned off.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread metadata/supported-configurations.json Outdated
Comment thread metadata/supported-configurations.json
jbachorik and others added 3 commits April 30, 2026 13:10
- metadata: dedup DD_PROFILING_DIRECTALLOCATION_ENABLED, add
  DD_PROFILING_DIRECT_MEMORY_ENABLED as its alias, flip
  DD_PROFILING_HEAP_ENABLED and DD_PROFILING_ALLOCATION_ENABLED
  defaults to "true"
- ProfilingConfig: replace broken {@link ProfilingSupport...}
  with {@code ...} (helper lives in another module)
- OpenJdkController: fix PROFILING_WALL_ENABLED comment to
  PROFILING_WALLTIME_ENABLED; reword disableEvent reasons to
  read grammatically in the "because it is ..." log
- DatadogProfilerConfigTest: add 5 tests for unified-gate
  behavior of isCpuProfilerEnabled / isWallClockProfilerEnabled
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jbachorik jbachorik marked this pull request as draft May 4, 2026 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: profiling Profiling tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants