Implement Bazel support for CI Visibility#11150
Draft
daniel-mohedano wants to merge 13 commits intomasterfrom
Draft
Implement Bazel support for CI Visibility#11150daniel-mohedano wants to merge 13 commits intomasterfrom
daniel-mohedano wants to merge 13 commits intomasterfrom
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 60 metrics, 10 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066211
Total [baseline] (11.093 s) : 0, 11093179
Agent [candidate] (1.061 s) : 0, 1060658
Total [candidate] (11.092 s) : 0, 11092024
section appsec
Agent [baseline] (1.251 s) : 0, 1250578
Total [baseline] (11.011 s) : 0, 11011156
Agent [candidate] (1.249 s) : 0, 1249394
Total [candidate] (11.059 s) : 0, 11058975
section iast
Agent [baseline] (1.227 s) : 0, 1226911
Total [baseline] (11.351 s) : 0, 11351352
Agent [candidate] (1.231 s) : 0, 1230876
Total [candidate] (11.255 s) : 0, 11254715
section profiling
Agent [baseline] (1.186 s) : 0, 1185935
Total [baseline] (11.038 s) : 0, 11038330
Agent [candidate] (1.193 s) : 0, 1192509
Total [candidate] (11.044 s) : 0, 11044036
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.234 ms) : 0, 1234
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (637.871 ms) : 0, 637871
BytebuddyAgent [candidate] (632.996 ms) : 0, 632996
AgentMeter [baseline] (29.706 ms) : 0, 29706
AgentMeter [candidate] (29.956 ms) : 0, 29956
GlobalTracer [baseline] (250.462 ms) : 0, 250462
GlobalTracer [candidate] (250.653 ms) : 0, 250653
AppSec [baseline] (32.672 ms) : 0, 32672
AppSec [candidate] (32.72 ms) : 0, 32720
Debugger [baseline] (60.257 ms) : 0, 60257
Debugger [candidate] (60.336 ms) : 0, 60336
Remote Config [baseline] (594.364 µs) : 0, 594
Remote Config [candidate] (595.148 µs) : 0, 595
Telemetry [baseline] (9.596 ms) : 0, 9596
Telemetry [candidate] (9.151 ms) : 0, 9151
Flare Poller [baseline] (7.49 ms) : 0, 7490
Flare Poller [candidate] (6.784 ms) : 0, 6784
section appsec
crashtracking [baseline] (1.238 ms) : 0, 1238
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (662.837 ms) : 0, 662837
BytebuddyAgent [candidate] (661.813 ms) : 0, 661813
AgentMeter [baseline] (12.262 ms) : 0, 12262
AgentMeter [candidate] (12.314 ms) : 0, 12314
GlobalTracer [baseline] (249.078 ms) : 0, 249078
GlobalTracer [candidate] (249.056 ms) : 0, 249056
AppSec [baseline] (185.488 ms) : 0, 185488
AppSec [candidate] (185.228 ms) : 0, 185228
Debugger [baseline] (66.142 ms) : 0, 66142
Debugger [candidate] (65.786 ms) : 0, 65786
Remote Config [baseline] (608.561 µs) : 0, 609
Remote Config [candidate] (615.861 µs) : 0, 616
Telemetry [baseline] (8.428 ms) : 0, 8428
Telemetry [candidate] (8.712 ms) : 0, 8712
Flare Poller [baseline] (3.533 ms) : 0, 3533
Flare Poller [candidate] (3.537 ms) : 0, 3537
IAST [baseline] (24.522 ms) : 0, 24522
IAST [candidate] (24.571 ms) : 0, 24571
section iast
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (802.286 ms) : 0, 802286
BytebuddyAgent [candidate] (804.141 ms) : 0, 804141
AgentMeter [baseline] (11.617 ms) : 0, 11617
AgentMeter [candidate] (11.672 ms) : 0, 11672
GlobalTracer [baseline] (239.544 ms) : 0, 239544
GlobalTracer [candidate] (241.011 ms) : 0, 241011
AppSec [baseline] (32.011 ms) : 0, 32011
AppSec [candidate] (33.686 ms) : 0, 33686
Debugger [baseline] (64.783 ms) : 0, 64783
Debugger [candidate] (63.081 ms) : 0, 63081
Remote Config [baseline] (525.243 µs) : 0, 525
Remote Config [candidate] (533.232 µs) : 0, 533
Telemetry [baseline] (9.348 ms) : 0, 9348
Telemetry [candidate] (9.611 ms) : 0, 9611
Flare Poller [baseline] (3.541 ms) : 0, 3541
Flare Poller [candidate] (3.604 ms) : 0, 3604
IAST [baseline] (25.775 ms) : 0, 25775
IAST [candidate] (26.049 ms) : 0, 26049
section profiling
ProfilingAgent [baseline] (94.245 ms) : 0, 94245
ProfilingAgent [candidate] (93.878 ms) : 0, 93878
crashtracking [baseline] (1.181 ms) : 0, 1181
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (691.898 ms) : 0, 691898
BytebuddyAgent [candidate] (696.201 ms) : 0, 696201
AgentMeter [baseline] (9.197 ms) : 0, 9197
AgentMeter [candidate] (9.274 ms) : 0, 9274
GlobalTracer [baseline] (207.249 ms) : 0, 207249
GlobalTracer [candidate] (208.403 ms) : 0, 208403
AppSec [baseline] (32.854 ms) : 0, 32854
AppSec [candidate] (32.912 ms) : 0, 32912
Debugger [baseline] (65.864 ms) : 0, 65864
Debugger [candidate] (66.128 ms) : 0, 66128
Remote Config [baseline] (588.104 µs) : 0, 588
Remote Config [candidate] (582.447 µs) : 0, 582
Telemetry [baseline] (7.764 ms) : 0, 7764
Telemetry [candidate] (8.125 ms) : 0, 8125
Flare Poller [baseline] (3.591 ms) : 0, 3591
Flare Poller [candidate] (3.583 ms) : 0, 3583
Profiling [baseline] (94.825 ms) : 0, 94825
Profiling [candidate] (94.435 ms) : 0, 94435
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1055817
Total [baseline] (8.865 s) : 0, 8864982
Agent [candidate] (1.058 s) : 0, 1057892
Total [candidate] (8.833 s) : 0, 8832906
section iast
Agent [baseline] (1.23 s) : 0, 1229682
Total [baseline] (9.589 s) : 0, 9589096
Agent [candidate] (1.223 s) : 0, 1223177
Total [candidate] (9.552 s) : 0, 9551659
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.242 ms) : 0, 1242
crashtracking [candidate] (1.239 ms) : 0, 1239
BytebuddyAgent [baseline] (632.069 ms) : 0, 632069
BytebuddyAgent [candidate] (632.044 ms) : 0, 632044
AgentMeter [baseline] (29.536 ms) : 0, 29536
AgentMeter [candidate] (29.556 ms) : 0, 29556
GlobalTracer [baseline] (248.31 ms) : 0, 248310
GlobalTracer [candidate] (249.053 ms) : 0, 249053
AppSec [baseline] (32.368 ms) : 0, 32368
AppSec [candidate] (32.351 ms) : 0, 32351
Debugger [baseline] (59.218 ms) : 0, 59218
Debugger [candidate] (59.065 ms) : 0, 59065
Remote Config [baseline] (596.349 µs) : 0, 596
Remote Config [candidate] (589.736 µs) : 0, 590
Telemetry [baseline] (8.045 ms) : 0, 8045
Telemetry [candidate] (8.261 ms) : 0, 8261
Flare Poller [baseline] (8.211 ms) : 0, 8211
Flare Poller [candidate] (9.585 ms) : 0, 9585
section iast
crashtracking [baseline] (1.226 ms) : 0, 1226
crashtracking [candidate] (1.232 ms) : 0, 1232
BytebuddyAgent [baseline] (802.873 ms) : 0, 802873
BytebuddyAgent [candidate] (799.139 ms) : 0, 799139
AgentMeter [baseline] (11.614 ms) : 0, 11614
AgentMeter [candidate] (11.581 ms) : 0, 11581
GlobalTracer [baseline] (241.338 ms) : 0, 241338
GlobalTracer [candidate] (239.414 ms) : 0, 239414
IAST [baseline] (26.959 ms) : 0, 26959
IAST [candidate] (25.906 ms) : 0, 25906
AppSec [baseline] (29.958 ms) : 0, 29958
AppSec [candidate] (30.483 ms) : 0, 30483
Debugger [baseline] (65.917 ms) : 0, 65917
Debugger [candidate] (65.481 ms) : 0, 65481
Remote Config [baseline] (540.375 µs) : 0, 540
Remote Config [candidate] (524.209 µs) : 0, 524
Telemetry [baseline] (9.41 ms) : 0, 9410
Telemetry [candidate] (9.57 ms) : 0, 9570
Flare Poller [baseline] (3.671 ms) : 0, 3671
Flare Poller [candidate] (3.624 ms) : 0, 3624
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 17 metrics, 18 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section baseline
no_agent (17.929 ms) : 17748, 18109
. : milestone, 17929,
appsec (18.547 ms) : 18363, 18731
. : milestone, 18547,
code_origins (17.938 ms) : 17757, 18119
. : milestone, 17938,
iast (17.895 ms) : 17719, 18071
. : milestone, 17895,
profiling (18.354 ms) : 18170, 18537
. : milestone, 18354,
tracing (17.898 ms) : 17716, 18079
. : milestone, 17898,
section candidate
no_agent (17.473 ms) : 17298, 17648
. : milestone, 17473,
appsec (18.732 ms) : 18543, 18920
. : milestone, 18732,
code_origins (17.753 ms) : 17579, 17928
. : milestone, 17753,
iast (18.07 ms) : 17891, 18249
. : milestone, 18070,
profiling (18.429 ms) : 18243, 18614
. : milestone, 18429,
tracing (17.873 ms) : 17700, 18047
. : milestone, 17873,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section baseline
no_agent (1.237 ms) : 1225, 1248
. : milestone, 1237,
iast (3.408 ms) : 3358, 3457
. : milestone, 3408,
iast_FULL (5.932 ms) : 5872, 5992
. : milestone, 5932,
iast_GLOBAL (3.765 ms) : 3692, 3838
. : milestone, 3765,
profiling (2.189 ms) : 2168, 2210
. : milestone, 2189,
tracing (1.973 ms) : 1956, 1990
. : milestone, 1973,
section candidate
no_agent (1.243 ms) : 1230, 1255
. : milestone, 1243,
iast (3.205 ms) : 3163, 3248
. : milestone, 3205,
iast_FULL (6.158 ms) : 6095, 6221
. : milestone, 6158,
iast_GLOBAL (3.75 ms) : 3684, 3815
. : milestone, 3750,
profiling (2.049 ms) : 2032, 2067
. : milestone, 2049,
tracing (1.924 ms) : 1907, 1940
. : milestone, 1924,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section baseline
no_agent (1.486 ms) : 1475, 1498
. : milestone, 1486,
appsec (3.777 ms) : 3555, 3998
. : milestone, 3777,
iast (2.262 ms) : 2193, 2330
. : milestone, 2262,
iast_GLOBAL (2.302 ms) : 2233, 2371
. : milestone, 2302,
profiling (2.095 ms) : 2040, 2150
. : milestone, 2095,
tracing (2.076 ms) : 2023, 2130
. : milestone, 2076,
section candidate
no_agent (1.483 ms) : 1471, 1495
. : milestone, 1483,
appsec (2.538 ms) : 2483, 2593
. : milestone, 2538,
iast (2.267 ms) : 2198, 2336
. : milestone, 2267,
iast_GLOBAL (2.312 ms) : 2242, 2381
. : milestone, 2312,
profiling (2.085 ms) : 2031, 2140
. : milestone, 2085,
tracing (2.076 ms) : 2023, 2130
. : milestone, 2076,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~e7e08284de, baseline=1.62.0-SNAPSHOT~081af53226
dateFormat X
axisFormat %s
section baseline
no_agent (14.891 s) : 14891000, 14891000
. : milestone, 14891000,
appsec (14.691 s) : 14691000, 14691000
. : milestone, 14691000,
iast (18.105 s) : 18105000, 18105000
. : milestone, 18105000,
iast_GLOBAL (17.893 s) : 17893000, 17893000
. : milestone, 17893000,
profiling (14.853 s) : 14853000, 14853000
. : milestone, 14853000,
tracing (15.029 s) : 15029000, 15029000
. : milestone, 15029000,
section candidate
no_agent (15.552 s) : 15552000, 15552000
. : milestone, 15552000,
appsec (14.908 s) : 14908000, 14908000
. : milestone, 14908000,
iast (18.515 s) : 18515000, 18515000
. : milestone, 18515000,
iast_GLOBAL (18.424 s) : 18424000, 18424000
. : milestone, 18424000,
profiling (15.063 s) : 15063000, 15063000
. : milestone, 15063000,
tracing (14.915 s) : 14915000, 14915000
. : milestone, 14915000,
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Adds Bazel-focused CI Visibility support with two offline execution modes, mirroring dd-trace-go#4503 and dd-trace-py#17197:
DD_TEST_OPTIMIZATION_MANIFEST_FILE): reads settings, known tests, flaky tests, and test management data from pre-fetched JSON cache files instead of hitting the backend.DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES): writes CI test cycle, coverage, and tracer telemetry to$TEST_UNDECLARED_OUTPUTS_DIR/payloads/{tests,coverage,telemetry}/*.jsoninstead of POSTing them.Key Changes
BazelMode(internal-api): detects both modes, resolves the manifest path via Bazel's rlocation algorithm, parses theversion=<int>header, and exposes thetests/,coverage/, andtelemetry/output directories.FileBasedConfigurationApi(agent-ci-visibility): reads the same JSON envelopes as the HTTP API from disk; null paths return safe defaults.FileBasedPayloadDispatcher(dd-trace-core): serializes CI test cycle and coverage spans as JSON files; stripsci.*/git.*/runtime.*/os.*tags to avoid cache invalidation; atomic temp-file + rename. Writestrace_id/span_id/parent_idas unsigned 64-bit JSON numbers (not strings) so backend schema validation passes.FileBasedTelemetryClient(telemetry): subclass ofTelemetryClientthat writes the existing Moshi-encoded telemetry request body to a file;TelemetryRoutergets a single-client path that skips feature discovery;TelemetrySystemswaps in the file-based client when Bazel mode is active.WriterFactory/CiVisibilityServices/CiVisibilityRepoServices: wire the file-based dispatcher/config API, disable the git client, and skip git-data upload when Bazel mode is active.CoreTracer: in Bazel payload-files mode, usesStreamingTraceCollector(streams each CI Visibility span individually) andDDIntakeTraceInterceptor(not the APM-protocol interceptor, which stripstest_{session,module,suite}_endspans) — same treatment as agentless, so all CITESTCYCLE events reach the file dispatcher.JUnit4TracingListener/JUnit4Utils: lazy-register the test suite intestStartedso runners that don't firetestSuiteStartedstill produce a proper suite span; unwrapcom.google.testing.junit.junit4.runner.RunNotifierWrapperinrunListenersFromRunNotifierso the idempotency check sees listeners installed on the inner notifier (fixes duplicate-listener installation underBazelTestRunner).Config: addsDD_TEST_OPTIMIZATION_MANIFEST_FILEandDD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES; skips API-key validation in these modes.TEST_UNDECLARED_OUTPUTS_DIRis read directly inBazelMode(it's a Bazel-provided env var, not a DD config).Motivation
Bazel runs tests in hermetic sandboxes with no network access. The existing CI Visibility pipeline requires HTTP calls to fetch configuration and submit payloads, which is incompatible with Bazel's execution model. This PR enables CI Visibility under Bazel by reading configuration from pre-fetched cache files and writing payloads/telemetry to files.
Additional Notes
BazelModeTest,FileBasedConfigurationApiTest(shares the existing*-response.ftlfixtures withConfigurationApiImplTestto keep the HTTP and file code paths in sync),FileBasedPayloadDispatcherTest,FileBasedTelemetryClientTest, and extendedTelemetryRouterSpecification.DataDog/rules_test_optimization_tests: 3test+ 1test_suite_end+ 1test_module_end+ 1test_session_endevents emitted to the payload file, no duplicate listener errors, no schema-validation failures.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting `/merge`. `/merge -c` cancels the queue request. `/merge -f --reason "reason"` skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.