|
| 1 | +# gRPC Tracing Integration |
| 2 | + |
| 3 | +This package provides OpenTelemetry tracing integration for gRPC calls in the accelerated-container-image project. |
| 4 | + |
| 5 | +## Features |
| 6 | + |
| 7 | +- **Automatic trace propagation**: Trace contexts are automatically propagated between client and server calls |
| 8 | +- **Comprehensive tracing**: Both client and server interceptors capture request/response metadata |
| 9 | +- **OpenTelemetry integration**: Uses standard OpenTelemetry libraries for compatibility |
| 10 | +- **Error handling**: Automatically records errors and sets appropriate span status |
| 11 | + |
| 12 | +## Usage |
| 13 | + |
| 14 | +### Server Setup |
| 15 | + |
| 16 | +The main snapshotter service already includes tracing interceptors: |
| 17 | + |
| 18 | +```go |
| 19 | +// Chain interceptors: tracing first, then request ID |
| 20 | +interceptors := grpc.ChainUnaryInterceptor( |
| 21 | + tracing.UnaryServerInterceptor(), |
| 22 | + requestIDInterceptor, |
| 23 | +) |
| 24 | +srv := grpc.NewServer(interceptors) |
| 25 | +``` |
| 26 | + |
| 27 | +Or use the convenience function: |
| 28 | + |
| 29 | +```go |
| 30 | +srv := grpc.NewServer(tracing.WithServerTracing()) |
| 31 | +``` |
| 32 | + |
| 33 | +### Client Setup |
| 34 | + |
| 35 | +For gRPC clients, add the tracing interceptor: |
| 36 | + |
| 37 | +```go |
| 38 | +conn, err := grpc.DialContext(ctx, target, |
| 39 | + grpc.WithUnaryInterceptor(tracing.UnaryClientInterceptor()), |
| 40 | + // other options... |
| 41 | +) |
| 42 | +``` |
| 43 | + |
| 44 | +Or use the convenience function: |
| 45 | + |
| 46 | +```go |
| 47 | +conn, err := grpc.DialContext(ctx, target, |
| 48 | + tracing.WithClientTracing(), |
| 49 | + // other options... |
| 50 | +) |
| 51 | +``` |
| 52 | + |
| 53 | +### Trace Propagation |
| 54 | + |
| 55 | +Trace IDs automatically propagate from: |
| 56 | +1. **Incoming requests** → Server-side processing |
| 57 | +2. **Server-side processing** → Outgoing client calls |
| 58 | +3. **Client calls** → Downstream services |
| 59 | + |
| 60 | +Example flow: |
| 61 | +``` |
| 62 | +Client Request [trace-id: abc123] |
| 63 | + ↓ |
| 64 | +gRPC Server Interceptor [trace-id: abc123] |
| 65 | + ↓ |
| 66 | +Snapshotter Service [trace-id: abc123] |
| 67 | + ↓ |
| 68 | +Outgoing Client Call [trace-id: abc123] |
| 69 | +``` |
| 70 | + |
| 71 | +## Trace Attributes |
| 72 | + |
| 73 | +The interceptors automatically add these attributes to spans: |
| 74 | + |
| 75 | +### Common Attributes |
| 76 | +- `rpc.system`: "grpc" |
| 77 | +- `rpc.service`: Service name (e.g., "containerd.services.snapshots.v1.Snapshots") |
| 78 | +- `rpc.method`: Method name (e.g., "Prepare") |
| 79 | +- `rpc.grpc.status_code`: gRPC status code |
| 80 | +- `rpc.grpc.duration_ms`: Request duration in milliseconds |
| 81 | + |
| 82 | +### Error Handling |
| 83 | +- Span status set to ERROR for failed requests |
| 84 | +- Error details recorded in span |
| 85 | +- gRPC status code captured |
| 86 | + |
| 87 | +## Configuration |
| 88 | + |
| 89 | +Set these environment variables for tracing: |
| 90 | + |
| 91 | +```bash |
| 92 | +# Service name for traces |
| 93 | +export OTEL_SERVICE_NAME="accelerated-container-image" |
| 94 | + |
| 95 | +# OTLP endpoint (e.g., Jaeger collector) |
| 96 | +export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" |
| 97 | + |
| 98 | +# Environment tag |
| 99 | +export ENVIRONMENT="production" |
| 100 | +``` |
| 101 | + |
| 102 | +## Baggage API Usage |
| 103 | + |
| 104 | +The implementation includes full support for OpenTelemetry Baggage, allowing you to propagate key-value pairs across service boundaries. |
| 105 | + |
| 106 | +### Setting Baggage Values |
| 107 | + |
| 108 | +```go |
| 109 | +import "github.com/containerd/accelerated-container-image/pkg/tracing" |
| 110 | + |
| 111 | +// Set individual values |
| 112 | +ctx = tracing.SetRequestID(ctx, "req-12345") |
| 113 | +ctx = tracing.SetUserID(ctx, "user-67890") |
| 114 | +ctx = tracing.SetOperation(ctx, "prepare-snapshot") |
| 115 | + |
| 116 | +// Set image information |
| 117 | +ctx = tracing.SetImageInfo(ctx, "alpine", "3.18") |
| 118 | + |
| 119 | +// Set multiple values at once |
| 120 | +baggageMap := map[string]string{ |
| 121 | + "environment": "production", |
| 122 | + "namespace": "default", |
| 123 | + "container.id": "container-123", |
| 124 | +} |
| 125 | +ctx = tracing.SetBaggageFromMap(ctx, baggageMap) |
| 126 | +``` |
| 127 | + |
| 128 | +### Getting Baggage Values |
| 129 | + |
| 130 | +```go |
| 131 | +// Get individual values |
| 132 | +requestID := tracing.GetRequestID(ctx) |
| 133 | +userID := tracing.GetUserID(ctx) |
| 134 | +operation := tracing.GetOperation(ctx) |
| 135 | + |
| 136 | +// Get image information |
| 137 | +imageName, imageTag := tracing.GetImageInfo(ctx) |
| 138 | + |
| 139 | +// Get all baggage as a map |
| 140 | +allBaggage := tracing.GetBaggageAsMap(ctx) |
| 141 | +``` |
| 142 | + |
| 143 | +### Automatic Baggage Propagation |
| 144 | + |
| 145 | +Baggage values are automatically: |
| 146 | +1. **Extracted** from incoming gRPC metadata |
| 147 | +2. **Injected** into outgoing gRPC metadata |
| 148 | +3. **Added as span attributes** with `baggage.` prefix |
| 149 | +4. **Propagated** through the entire request chain |
| 150 | + |
| 151 | +### Common Baggage Keys |
| 152 | + |
| 153 | +The package defines standard baggage keys: |
| 154 | +- `request.id` - Request identifier |
| 155 | +- `user.id` - User identifier |
| 156 | +- `session.id` - Session identifier |
| 157 | +- `operation` - Operation name |
| 158 | +- `image.name` - Container image name |
| 159 | +- `image.tag` - Container image tag |
| 160 | +- `snapshot.key` - Snapshot key |
| 161 | +- `container.id` - Container identifier |
| 162 | +- `namespace` - Kubernetes namespace |
| 163 | +- `environment` - Environment (prod, staging, etc.) |
| 164 | + |
| 165 | +## Integration with Request ID |
| 166 | + |
| 167 | +The request ID interceptor now uses baggage for propagation: |
| 168 | + |
| 169 | +```go |
| 170 | +// Check if request ID exists in baggage (from upstream) |
| 171 | +requestID := tracing.GetRequestID(ctx) |
| 172 | +if requestID == "" { |
| 173 | + // Generate new ID if not present |
| 174 | + requestID = generateRequestID() |
| 175 | + ctx = tracing.SetRequestID(ctx, requestID) |
| 176 | +} |
| 177 | +``` |
| 178 | + |
| 179 | +This ensures request IDs propagate across service boundaries while maintaining backward compatibility. |
| 180 | + |
| 181 | +## Integration with Existing Code |
| 182 | + |
| 183 | +The tracing is integrated with the existing snapshotter service: |
| 184 | + |
| 185 | +1. **Initialization**: `tracing.InitTracer()` sets up OpenTelemetry with baggage propagation |
| 186 | +2. **Server interceptor**: Added to gRPC server chain, extracts baggage from metadata |
| 187 | +3. **Service wrapper**: `tracing.WithTracing()` wraps the snapshotter service |
| 188 | +4. **Client interceptor**: Added to any outgoing gRPC clients, injects baggage into metadata |
| 189 | +5. **Baggage utilities**: Helper functions for common baggage operations |
| 190 | + |
| 191 | +This provides comprehensive tracing coverage at both the transport (gRPC) and application (snapshotter operations) levels, with full context propagation via baggage. |
0 commit comments