Implement Bazel support for CI Visibility (#11150)

# What Does This Do

Adds Bazel-focused CI Visibility support with two offline execution modes, mirroring [dd-trace-go#4503](DataDog/dd-trace-go#4503) and [dd-trace-py#17197](DataDog/dd-trace-py#17197):

- **Manifest mode** (`DD_TEST_OPTIMIZATION_MANIFEST_FILE`): reads settings, known tests, flaky tests, and test management data from pre-fetched JSON cache files instead of hitting the backend.
- **Payload-files mode** (`DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES`): writes CI test cycle, coverage, and tracer telemetry to `$TEST_UNDECLARED_OUTPUTS_DIR/payloads/{tests,coverage,telemetry}/*.json` instead of POSTing them.

## Key Changes

- `BazelMode` (internal-api): detects both modes, resolves the manifest path via Bazel's rlocation algorithm, parses the `version=<int>` header, and exposes the `tests/`, `coverage/`, and `telemetry/` output directories.
- `FileBasedConfigurationApi` (agent-ci-visibility): reads the same JSON envelopes as the HTTP API from disk; null paths return safe defaults.
- `FileBasedPayloadDispatcher` (dd-trace-core): serializes CI test cycle and coverage spans as JSON files; strips `ci.*`/`git.*`/`runtime.*`/`os.*` tags to avoid cache invalidation; atomic temp-file + rename. Writes `trace_id`/`span_id`/`parent_id` as unsigned 64-bit JSON numbers (not strings) so backend schema validation passes.
- `FileBasedTelemetryClient` (telemetry): subclass of `TelemetryClient` that writes the existing Moshi-encoded telemetry request body to a file; `TelemetryRouter` gets a single-client path that skips feature discovery; `TelemetrySystem` swaps in the file-based client when Bazel mode is active.
- `WriterFactory` / `CiVisibilityServices` / `CiVisibilityRepoServices`: wire the file-based dispatcher/config API, disable the git client, and skip git-data upload when Bazel mode is active.
- `CoreTracer`: in Bazel payload-files mode, uses `StreamingTraceCollector` (streams each CI Visibility span individually) and `DDIntakeTraceInterceptor` (not the APM-protocol interceptor, which strips `test_{session,module,suite}_end` spans) — same treatment as agentless, so all CITESTCYCLE events reach the file dispatcher.
- `JUnit4TracingListener` / `JUnit4Utils`: lazy-register the test suite in `testStarted` so runners that don't fire `testSuiteStarted` still produce a proper suite span; unwrap `com.google.testing.junit.junit4.runner.RunNotifierWrapper` in `runListenersFromRunNotifier` so the idempotency check sees listeners installed on the inner notifier (fixes duplicate-listener installation under `BazelTestRunner`).
- `Config`: adds `DD_TEST_OPTIMIZATION_MANIFEST_FILE` and `DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES`; skips API-key validation in these modes. `TEST_UNDECLARED_OUTPUTS_DIR` is read directly in `BazelMode` (it's a Bazel-provided env var, not a DD config).

# Motivation

Bazel can run tests in hermetic sandboxes with no network access. The existing CI Visibility pipeline requires HTTP calls to fetch configuration and submit payloads, which is incompatible with Bazel's execution model. Most of our operations, such as tagging tests with git metadata, also invalid Bazel's cache. This PR enables CI Visibility under Bazel by reading configuration from pre-fetched cache files and writing payloads/telemetry to files, with the orchestration of everything else being handled by our custom testing rule.

# Additional Notes

- Unit tests cover each new component: `BazelModeTest`, `FileBasedConfigurationApiTest` (shares the existing `*-response.ftl` fixtures with `ConfigurationApiImplTest` to keep the HTTP and file code paths in sync), `FileBasedPayloadDispatcherTest`, `FileBasedTelemetryClientTest`, and extended `TelemetryRouterSpecification`.
- End-to-end repro validated locally against `DataDog/rules_test_optimization_tests`: 3 `test` + 1 `test_suite_end` + 1 `test_module_end` + 1 `test_session_end` events emitted to the payload file, no duplicate listener errors, no schema-validation failures.
- Future work, not included in this PR to avoid changes too big:
	- Include instrumentation improvements to better handle Bazel's custom JUnit4 test runner
	- Refactoring of configuration API related DTOs to common utilities
	- Define a specification for mapping and serializing CI Vis spans to avoid logic mirroring between the two approaches (original vs file-based)
	- Possibly introduce a smoke test for e2e testing of bazel process, to avoid dependencies on an external repository.

# Contributor Checklist

- [x] Format the title according to [the contribution guidelines](https://github.com/DataDog/dd-trace-java/blob/master/CONTRIBUTING.md#title-format)
- [ ] Assign the `type:` and (`comp:` or `inst:`) labels in addition to [any other useful labels](https://github.com/DataDog/dd-trace-java/blob/master/CONTRIBUTING.md#labels)
- [ ] Update the [CODEOWNERS](https://github.com/DataDog/dd-trace-java/blob/master/.github/CODEOWNERS) file on source file addition, migration, or deletion
- [ ] Update [public documentation](https://docs.datadoghq.com/tracing/trace_collection/library_config/java/) with any new configuration flags or behaviors

Jira ticket: [SDTEST-3335]

***Note:*** **Once your PR is ready to merge, add it to the merge queue by commenting \`/merge\`.** \`/merge -c\` cancels the queue request. \`/merge -f --reason "reason"\` skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see [this doc](https://datadoghq.atlassian.net/wiki/spaces/DEVX/pages/3121612126/MergeQueue).

[SDTEST-3335]: https://datadoghq.atlassian.net/browse/SDTEST-3335?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Co-authored-by: daniel.mohedano <daniel.mohedano@datadoghq.com>

Author: daniel-mohedano

dd-java-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/CiVisibilityRepoServices.java

@@ -2,6 +2,7 @@
 
 import datadog.communication.BackendApi;
 import datadog.trace.api.Config;
+import datadog.trace.api.civisibility.config.BazelMode;
 import datadog.trace.api.civisibility.telemetry.CiVisibilityMetricCollector;
 import datadog.trace.api.civisibility.telemetry.tag.Provider;
 import datadog.trace.api.git.CommitInfo;
@@ -19,6 +20,7 @@
 import datadog.trace.civisibility.config.ExecutionSettings;
 import datadog.trace.civisibility.config.ExecutionSettingsFactory;
 import datadog.trace.civisibility.config.ExecutionSettingsFactoryImpl;
+import datadog.trace.civisibility.config.FileBasedConfigurationApi;
 import datadog.trace.civisibility.config.JvmInfo;
 import datadog.trace.civisibility.config.MultiModuleExecutionSettingsFactory;
 import datadog.trace.civisibility.git.tree.GitClient;
@@ -81,15 +83,22 @@ public class CiVisibilityRepoServices {
 
     ciTags = new CITagsProvider().getCiTags(ciInfo, pullRequestInfo);
 
-    gitDataUploader =
-        buildGitDataUploader(
-            services.config,
-            services.metricCollector,
-            services.gitInfoProvider,
-            gitClient,
-            gitRepoUnshallow,
-            services.backendApi,
-            repoRoot);
+    if (BazelMode.get().isEnabled()) {
+      // bazel rule takes care of the git data upload
+      LOGGER.info("[bazel mode] Skipping git data upload");
+      gitDataUploader = () -> CompletableFuture.completedFuture(null);
+    } else {
+      gitDataUploader =
+          buildGitDataUploader(
+              services.config,
+              services.metricCollector,
+              services.gitInfoProvider,
+              gitClient,
+              gitRepoUnshallow,
+              services.backendApi,
+              repoRoot);
+    }
+
     repoIndexProvider = services.repoIndexProviderFactory.create(repoRoot);
     codeowners = buildCodeowners(repoRoot);
     sourcePathResolver = buildSourcePathResolver(repoRoot, repoIndexProvider);
@@ -242,7 +251,17 @@ private static ExecutionSettingsFactory buildExecutionSettingsFactory(
       PullRequestInfo pullRequestInfo,
       @Nullable String repoRoot) {
     ConfigurationApi configurationApi;
-    if (backendApi == null) {
+    BazelMode bazelMode = BazelMode.get();
+    if (bazelMode.isManifestModeEnabled()) {
+      LOGGER.info("[bazel mode] Manifest mode detected. Using file-based configuration API");
+      configurationApi =
+          new FileBasedConfigurationApi(
+              bazelMode.getSettingsPath(),
+              null,
+              bazelMode.getFlakyTestsPath(),
+              bazelMode.getKnownTestsPath(),
+              bazelMode.getTestManagementPath());
+    } else if (backendApi == null) {
       LOGGER.warn(
           "Remote config and skippable tests requests will be skipped since backend API client could not be created");
       configurationApi = ConfigurationApi.NO_OP;

dd-java-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/CiVisibilityServices.java

@@ -5,6 +5,7 @@
 import datadog.communication.ddagent.SharedCommunicationObjects;
 import datadog.communication.util.IOUtils;
 import datadog.trace.api.Config;
+import datadog.trace.api.civisibility.config.BazelMode;
 import datadog.trace.api.civisibility.telemetry.CiVisibilityCountMetric;
 import datadog.trace.api.civisibility.telemetry.CiVisibilityMetricCollector;
 import datadog.trace.api.civisibility.telemetry.tag.Command;
@@ -83,7 +84,14 @@ public class CiVisibilityServices {
     this.backendApi = new BackendApiFactory(config, sco).createBackendApi(Intake.API);
     this.ciIntake = new BackendApiFactory(config, sco).createBackendApi(Intake.CI_INTAKE);
     this.jvmInfoFactory = new CachingJvmInfoFactory(config, new JvmInfoFactoryImpl());
-    this.gitClientFactory = buildGitClientFactory(config, metricCollector);
+
+    if (BazelMode.get().isPayloadFilesEnabled()) {
+      // git commands should not be executed in payload files mode
+      logger.info("[bazel mode] Payload-in-files mode detected. Disabling git commands");
+      this.gitClientFactory = r -> NoOpGitClient.INSTANCE;
+    } else {
+      this.gitClientFactory = buildGitClientFactory(config, metricCollector);
+    }
 
     this.environment = buildCiEnvironment();
     this.ciProviderInfoFactory = new CIProviderInfoFactory(config, environment);

dd-java-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/config/ConfigurationApiImpl.java

@@ -1,10 +1,8 @@
 package datadog.trace.civisibility.config;
 
-import com.squareup.moshi.FromJson;
 import com.squareup.moshi.Json;
 import com.squareup.moshi.JsonAdapter;
 import com.squareup.moshi.Moshi;
-import com.squareup.moshi.ToJson;
 import com.squareup.moshi.Types;
 import datadog.communication.BackendApi;
 import datadog.communication.http.OkHttpUtils;
@@ -27,10 +25,8 @@
 import datadog.trace.api.civisibility.telemetry.tag.TestManagementEnabled;
 import datadog.trace.civisibility.communication.TelemetryListener;
 import datadog.trace.util.RandomUtils;
-import java.io.File;
 import java.io.IOException;
 import java.lang.reflect.ParameterizedType;
-import java.util.Base64;
 import java.util.BitSet;
 import java.util.Collection;
 import java.util.Collections;
@@ -87,7 +83,7 @@ public ConfigurationApiImpl(BackendApi backendApi, CiVisibilityMetricCollector m
             .add(ConfigurationsJsonAdapter.INSTANCE)
             .add(CiVisibilitySettings.JsonAdapter.INSTANCE)
             .add(EarlyFlakeDetectionSettings.JsonAdapter.INSTANCE)
-            .add(MetaDtoJsonAdapter.INSTANCE)
+            .add(MetaDto.JsonAdapter.INSTANCE)
             .build();
 
     ParameterizedType requestType =
@@ -208,7 +204,7 @@ public SkippableTests getSkippableTests(TracerEnvironment tracerEnvironment) thr
     metricCollector.add(
         CiVisibilityCountMetric.ITR_SKIPPABLE_TESTS_RESPONSE_TESTS, response.data.size());
 
-    String correlationId = response.meta != null ? response.meta.correlation_id : null;
+    String correlationId = response.meta != null ? response.meta.correlationId : null;
     Map<String, BitSet> coveredLinesByRelativeSourcePath =
         response.meta != null && response.meta.coverage != null
             ? response.meta.coverage
@@ -499,6 +495,7 @@ private MultiEnvelopeDto(Collection<DataDto<T>> data, MetaDto meta) {
   }
 
   private static final class DataDto<T> {
+    // TODO: extract all DTO logic to common utilities
     private final String id;
     private final String type;
     private final T attributes;
@@ -514,52 +511,6 @@ public T getAttributes() {
     }
   }
 
-  private static final class MetaDto {
-    private final String correlation_id;
-    private final Map<String, BitSet> coverage;
-
-    private MetaDto(String correlation_id, Map<String, BitSet> coverage) {
-      this.correlation_id = correlation_id;
-      this.coverage = coverage;
-    }
-  }
-
-  private static final class MetaDtoJsonAdapter {
-
-    private static final MetaDtoJsonAdapter INSTANCE = new MetaDtoJsonAdapter();
-
-    @FromJson
-    public MetaDto fromJson(Map<String, Object> json) {
-      if (json == null) {
-        return null;
-      }
-
-      Map<String, BitSet> coverage;
-      Map<String, String> encodedCoverage = (Map<String, String>) json.get("coverage");
-      if (encodedCoverage != null) {
-        coverage = new HashMap<>();
-        for (Map.Entry<String, String> e : encodedCoverage.entrySet()) {
-          String relativeSourceFilePath = e.getKey();
-          String normalizedSourceFilePath =
-              relativeSourceFilePath.startsWith(File.separator)
-                  ? relativeSourceFilePath.substring(1)
-                  : relativeSourceFilePath;
-          byte[] decodedLines = Base64.getDecoder().decode(e.getValue());
-          coverage.put(normalizedSourceFilePath, BitSet.valueOf(decodedLines));
-        }
-      } else {
-        coverage = null;
-      }
-
-      return new MetaDto((String) json.get("correlation_id"), coverage);
-    }
-
-    @ToJson
-    public Map<String, Object> toJson(MetaDto metaDto) {
-      throw new UnsupportedOperationException();
-    }
-  }
-
   private static final class KnownTestsDto {
     private final Map<String, Map<String, List<String>>> tests;
 
@@ -648,66 +599,4 @@ private TestManagementDto(
       this.branch = branch;
     }
   }
-
-  private static final class TestManagementTestsDto {
-    private static final class Properties {
-      private final Map<String, Boolean> properties;
-
-      private Properties(Map<String, Boolean> properties) {
-        this.properties = properties;
-      }
-
-      public Boolean isQuarantined() {
-        return properties != null
-            ? properties.getOrDefault(TestSetting.QUARANTINED.asString(), false)
-            : false;
-      }
-
-      public Boolean isDisabled() {
-        return properties != null
-            ? properties.getOrDefault(TestSetting.DISABLED.asString(), false)
-            : false;
-      }
-
-      public Boolean isAttemptToFix() {
-        return properties != null
-            ? properties.getOrDefault(TestSetting.ATTEMPT_TO_FIX.asString(), false)
-            : false;
-      }
-    }
-
-    private static final class Tests {
-      private final Map<String, Properties> tests;
-
-      private Tests(Map<String, Properties> tests) {
-        this.tests = tests;
-      }
-
-      public Map<String, Properties> getTests() {
-        return tests != null ? tests : Collections.emptyMap();
-      }
-    }
-
-    private static final class Suites {
-      private final Map<String, Tests> suites;
-
-      private Suites(Map<String, Tests> suites) {
-        this.suites = suites;
-      }
-
-      public Map<String, Tests> getSuites() {
-        return suites != null ? suites : Collections.emptyMap();
-      }
-    }
-
-    private final Map<String, Suites> modules;
-
-    private TestManagementTestsDto(Map<String, Suites> modules) {
-      this.modules = modules;
-    }
-
-    public Map<String, Suites> getModules() {
-      return modules != null ? modules : Collections.emptyMap();
-    }
-  }
 }