feat: Add W3C Trace Context support for distributed tracing by jadewang-db · Pull Request #874 · databricks/databricks-jdbc

jadewang-db · 2025-07-02T04:17:51Z

Summary

This PR adds W3C Trace Context (traceparent/tracestate) header support to the Databricks JDBC
driver, enabling distributed tracing across service boundaries. The implementation follows
the W3C Trace Context specification and allows trace
context propagation for better observability.

Changes

Core Implementation

TracingUtil: Added utility class for W3C trace context operations
- Validates and parses traceparent headers
- Generates trace IDs and span IDs
- Builds W3C-compliant traceparent headers
Connection Context: Enhanced to store trace context at connection scope
- Added TraceParent and TraceState connection parameters
- Stores extracted/generated trace ID and flags per connection
- Initializes trace context when connection is created
HTTP Headers: Added traceparent/tracestate headers to all HTTP requests
- DatabricksHttpClient (volume operations, telemetry)
- Thrift transport (Thrift protocol requests)
- SDK client (SQL execution API requests)

Configuration

New connection parameters:

EnableRequestTracing=1 - Enable trace context headers (existing, now enhanced)
TraceParent=<header> - Provide external W3C traceparent header
TraceState=<header> - Provide vendor-specific trace state

Design Decisions

Connection-scoped: Trace context is scoped to connection lifetime, not thread-local
- Avoids issues with connection pooling and async operations
- Context follows the connection across thread boundaries
Stateless utility: TracingUtil is a pure utility class with no state
Backward compatible: Existing EnableRequestTracing behavior preserved

Usage Examples

// Auto-generate trace context
jdbc:databricks://host:port/db;EnableRequestTracing=1

// Propagate existing trace context
jdbc:databricks://host:port/db;EnableRequestTracing=1;TraceParent=00-4bf92f3577b34da6a3ce929d
0e0e4736-00f067aa0ba902b7-01

// With trace state
jdbc:databricks://host:port/db;EnableRequestTracing=1;TraceParent=00-4bf92f3577b34da6a3ce929d
0e0e4736-00f067aa0ba902b7-01;TraceState=vendor1=value1

Testing

- Added comprehensive unit tests for TracingUtil
- Tests cover W3C format validation, ID generation, and header building
- All existing tests pass

Impact

- Enables distributed tracing for Databricks JDBC applications
- Helps debug and monitor cross-service request flows
- No impact when tracing is disabled (default)

## Description
<!-- Provide a brief summary of the changes made and the issue they aim to address.-->

## Testing
<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer understand the changes better. This could include challenges faced, limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to focus on specifically. -->

## Summary This PR adds W3C Trace Context (traceparent/tracestate) header support to the Databricks JDBC driver, enabling distributed tracing across service boundaries. The implementation follows the [W3C Trace Context specification](https://www.w3.org/TR/trace-context/) and allows trace context propagation for better observability. ## Changes ### Core Implementation - **TracingUtil**: Added utility class for W3C trace context operations - Validates and parses traceparent headers - Generates trace IDs and span IDs - Builds W3C-compliant traceparent headers - **Connection Context**: Enhanced to store trace context at connection scope - Added `TraceParent` and `TraceState` connection parameters - Stores extracted/generated trace ID and flags per connection - Initializes trace context when connection is created - **HTTP Headers**: Added traceparent/tracestate headers to all HTTP requests - DatabricksHttpClient (volume operations, telemetry) - Thrift transport (Thrift protocol requests) - SDK client (SQL execution API requests) ### Configuration New connection parameters: - `EnableRequestTracing=1` - Enable trace context headers (existing, now enhanced) - `TraceParent=<header>` - Provide external W3C traceparent header - `TraceState=<header>` - Provide vendor-specific trace state ### Design Decisions - **Connection-scoped**: Trace context is scoped to connection lifetime, not thread-local - Avoids issues with connection pooling and async operations - Context follows the connection across thread boundaries - **Stateless utility**: TracingUtil is a pure utility class with no state - **Backward compatible**: Existing `EnableRequestTracing` behavior preserved ## Usage Examples ```java // Auto-generate trace context jdbc:databricks://host:port/db;EnableRequestTracing=1 // Propagate existing trace context jdbc:databricks://host:port/db;EnableRequestTracing=1;TraceParent=00-4bf92f3577b34da6a3ce929d 0e0e4736-00f067aa0ba902b7-01 // With trace state jdbc:databricks://host:port/db;EnableRequestTracing=1;TraceParent=00-4bf92f3577b34da6a3ce929d 0e0e4736-00f067aa0ba902b7-01;TraceState=vendor1=value1 Testing - Added comprehensive unit tests for TracingUtil - Tests cover W3C format validation, ID generation, and header building - All existing tests pass Impact - Enables distributed tracing for Databricks JDBC applications - Helps debug and monitor cross-service request flows - No impact when tracing is disabled (default)

github-actions · 2025-07-02T04:18:06Z

Please ensure that the NEXT_CHANGELOG.md file is updated with any relevant changes.
If this is not necessary for your PR, please include the following in your PR description:
NO_CHANGELOG=true
and rerun the job.

gopalldb · 2025-07-11T11:24:47Z

+        // Generate new trace context
+        traceId = TracingUtil.generateTraceId();
+        traceFlags = "01"; // sampled by default
+        if (!isNullOrEmpty(traceParent)) {


This is nullOrEmpty else-case, why are we checking again?

gopalldb · 2025-07-11T12:56:24Z

+
+      // Add tracestate if present
+      String traceState = connectionContext.getTraceState();
+      if (traceState != null && !traceState.isEmpty()) {


nit: can use Strings.isNullOrEmpty

gopalldb · 2025-07-11T13:08:28Z

-    if (connectionContext.isRequestTracingEnabled()) {
-      String traceHeader = TracingUtil.getTraceHeader();
+    if (connectionContext.isRequestTracingEnabled() && connectionContext.getTraceId() != null) {
+      String traceHeader =


shall we check for empty also?

gopalldb · 2025-07-11T13:08:52Z

+
+      // Add tracestate if present
+      String traceState = connectionContext.getTraceState();
+      if (traceState != null && !traceState.isEmpty()) {


can use !Strings.isNullOrEmpty

gopalldb · 2025-07-11T13:09:50Z

      }
+
+      // Add W3C Trace Context headers if request tracing is enabled
+      if (connectionContext.isRequestTracingEnabled() && connectionContext.getTraceId() != null) {


nit:empty check

github-actions · 2025-08-11T06:44:19Z

This PR has been marked as Stale because it has been open for 30 days with no activity. If you would like the PR to remain open, please remove the stale label or comment on the PR.

github-actions · 2025-08-19T06:41:54Z

This PR was closed because it has been inactive for 7 days since being marked as stale.

jadewang-db temporarily deployed to azure-prod July 2, 2025 04:17 — with GitHub Actions Inactive

Update DatabricksHttpTTransportTest.java

e89b83d

jayantsing-db self-requested a review July 7, 2025 11:30

gopalldb reviewed Jul 11, 2025

View reviewed changes

github-actions Bot added the Stale label Aug 11, 2025

github-actions Bot closed this Aug 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add W3C Trace Context support for distributed tracing#874

feat: Add W3C Trace Context support for distributed tracing#874
jadewang-db wants to merge 2 commits into
databricks:mainfrom
jadewang-db:trace-parent

jadewang-db commented Jul 2, 2025

Uh oh!

github-actions Bot commented Jul 2, 2025

Uh oh!

gopalldb Jul 11, 2025

Uh oh!

gopalldb Jul 11, 2025

Uh oh!

gopalldb Jul 11, 2025

Uh oh!

gopalldb Jul 11, 2025 •

edited

Loading

Uh oh!

gopalldb Jul 11, 2025

Uh oh!

github-actions Bot commented Aug 11, 2025

Uh oh!

github-actions Bot commented Aug 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jadewang-db commented Jul 2, 2025

Summary

Changes

Core Implementation

Configuration

Design Decisions

Usage Examples

Uh oh!

github-actions Bot commented Jul 2, 2025

Uh oh!

gopalldb Jul 11, 2025

Choose a reason for hiding this comment

Uh oh!

gopalldb Jul 11, 2025

Choose a reason for hiding this comment

Uh oh!

gopalldb Jul 11, 2025

Choose a reason for hiding this comment

Uh oh!

gopalldb Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gopalldb Jul 11, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 11, 2025

Uh oh!

github-actions Bot commented Aug 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gopalldb Jul 11, 2025 •

edited

Loading