Skip to content

Commit 46dde27

Browse files
Feat/telemetry http exporter (#314)
* feat(telemetry): add OTLP HTTP exporter and print TraceID - Add HTTP/protobuf exporter support alongside existing gRPC exporter - Route based on OTEL_EXPORTER_OTLP_PROTOCOL config (http/protobuf vs grpc) - Print TraceID to stderr when telemetry is enabled for easier correlation - Add corresponding unit tests * feat(telemetry): add span coverage for LLM calls, tool execution, plan/filter phases - Add StartLLMSpan / RecordLLMResult helpers (span.go), symmetric with existing StartToolSpan / RecordToolResult - Wrap LLM completion calls in llmloop.RunPerFile with llm.request spans - Wrap all three tool execution paths in executeToolCall with tool.execute.* spans (dynamic tools, code_comment sync/async, other tools) - Add plan.execute span around executePlanPhase - Add main.loop span around RunPerFile call in executeSubtask - Add review_filter.execute span around executeReviewFilter, with comments.before / comments.filtered attributes - Record llm.error attribute on LLM failures for diagnosability - Record review.repo / review.from / review.to / review.model on the top-level review.run span - Metrics (RecordLLMRequest / RecordToolCall) are preserved alongside the new spans — they serve different purposes (aggregate dashboards vs per-run diagnosis) Verified end-to-end against Sunfire (OTLP HTTP gateway): full span tree observed for review.run -> subtask.execute -> plan.execute/main.loop/ review_filter.execute -> llm.request/tool.execute.* * fix(telemetry): address CR findings — span error handling, async span lifecycle, protocol robustness - Add span.RecordError(err) to RecordLLMResult and RecordToolResult for consistency with EndSpan - Use OTel standard pattern (span.SetStatus + span.RecordError) in error paths of review.run, plan.execute, main.loop, review_filter.execute - Move async code_comment span end into pool.Submit callback so span duration reflects actual execution time - Unify time.Since(startTime) in code_comment error path to a single dur - Remove http/json from supported OTLP protocols (not actually implemented) - Add stderr warning when unknown OTLP protocol falls back to gRPC * feat(telemetry): include trace_id in JSON output, restrict stderr to text format - Add trace_id as top-level field in jsonOutput struct (omitempty) - JSON format: trace_id in structured response for programmatic extraction - Text format: TraceID printed to stderr for human debugging - Telemetry disabled: trace_id field omitted entirely * fix: address PR review findings - loop.go: wrap async span lifecycle in defer to prevent leak on panic - exporter.go: update parseOTLPEndpoint comment to reflect gRPC+HTTP usage - scan_cmd.go: align traceID extraction and OTel error handling with review_cmd - output.go/shared.go: propagate traceID to outputJSONNoFiles for consistency - agent.go: move comments.filtered attribute before early return so 0 is distinguishable from not-executed * feat(telemetry): address PR review — http/json routing, LLM span coverage, trace_id tests - Route http/json to HTTP exporter (Go OTel SDK HTTP transport only supports protobuf serialization; users need HTTP transport, not JSON encoding) - Add llm.request spans to executePlanPhase, executeReviewFilter, and ReLocateComment with Usage nil-safety consistent with loop.go - Add trace_id assertions to output helper tests and emitRunResult end-to-end tests using real TracerProvider * docs: add OTLP protocol selection and endpoint format to telemetry section Sync across all 5 README language versions (en, zh-CN, ja-JP, ko-KR, ru-RU). * fix: unify time.Since in async code_comment defer to single dur variable
1 parent 14f1c22 commit 46dde27

21 files changed

Lines changed: 471 additions & 24 deletions

README.ja-JP.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -762,6 +762,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
762762

763763
エクスポートデータにLLMのプロンプトとレスポンスを含めるには、`telemetry.content_logging`を設定してください。
764764

765+
**プロトコル選択:** 環境変数 `OTEL_EXPORTER_OTLP_PROTOCOL` でエクスポートプロトコルを選択できます:
766+
767+
|| トランスポート | 説明 |
768+
|---|---|---|
769+
| `grpc`(デフォルト) | gRPC | デフォルトポート 4317 |
770+
| `http/protobuf` | HTTP | デフォルトポート 4318 |
771+
772+
**Endpoint 形式:** `telemetry.otlp_endpoint``host:port` または `http://host:port` 形式のベースURLを指定します。パスを含める必要はありません。SDKが [OTLP仕様](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request)に従いシグナルパス(例:`/v1/traces`)を自動的に付加します。
773+
765774
## コントリビューション
766775

767776
開発環境のセットアップ、コーディングガイドライン、プルリクエストの提出方法については[CONTRIBUTING.md](CONTRIBUTING.md)を参照してください。

README.ko-KR.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -719,6 +719,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
719719

720720
exported data에 LLM prompt와 response를 포함하려면 `telemetry.content_logging`을 설정합니다.
721721

722+
**프로토콜 선택:** 환경 변수 `OTEL_EXPORTER_OTLP_PROTOCOL`로 export 프로토콜을 선택할 수 있습니다:
723+
724+
|| 전송 방식 | 설명 |
725+
|---|---|---|
726+
| `grpc` (기본값) | gRPC | 기본 포트 4317 |
727+
| `http/protobuf` | HTTP | 기본 포트 4318 |
728+
729+
**Endpoint 형식:** `telemetry.otlp_endpoint``host:port` 또는 `http://host:port` 형식의 base URL을 지정합니다. 경로를 포함할 필요가 없습니다. SDK가 [OTLP 사양](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request)에 따라 signal 경로(예: `/v1/traces`)를 자동으로 추가합니다.
730+
722731
## Contributing
723732

724733
개발 환경 설정, coding guideline, pull request 제출 방법은 [CONTRIBUTING.ko-KR.md](CONTRIBUTING.ko-KR.md)를 참고하세요.

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -767,6 +767,16 @@ ocr config set telemetry.otlp_endpoint localhost:4317
767767

768768
Set `telemetry.content_logging` to include LLM prompts and responses in exported data.
769769

770+
**Protocol selection:** Set the environment variable `OTEL_EXPORTER_OTLP_PROTOCOL` to choose the export protocol:
771+
772+
| Value | Transport | Notes |
773+
|---|---|---|
774+
| `grpc` (default) | gRPC | Default port 4317 |
775+
| `http/protobuf` | HTTP | Default port 4318 |
776+
777+
**Endpoint format:** `telemetry.otlp_endpoint` expects a base URL in `host:port` or `http://host:port` format, without a path component. The SDK appends the signal path (e.g. `/v1/traces`) automatically per the [OTLP specification](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request).
778+
779+
770780
## Contributing
771781

772782
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, coding guidelines, and how to submit pull requests.

README.ru-RU.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -764,6 +764,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
764764

765765
Установите `telemetry.content_logging`, чтобы включать промпты и ответы LLM в экспортируемые данные.
766766

767+
**Выбор протокола:** Переменная окружения `OTEL_EXPORTER_OTLP_PROTOCOL` определяет протокол экспорта:
768+
769+
| Значение | Транспорт | Описание |
770+
|---|---|---|
771+
| `grpc` (по умолчанию) | gRPC | Порт по умолчанию 4317 |
772+
| `http/protobuf` | HTTP | Порт по умолчанию 4318 |
773+
774+
**Формат endpoint:** `telemetry.otlp_endpoint` принимает базовый URL в формате `host:port` или `http://host:port` без компонента пути. SDK автоматически добавляет путь сигнала (например, `/v1/traces`) в соответствии со [спецификацией OTLP](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request).
775+
767776
## Участие в разработке
768777

769778
В [CONTRIBUTING.ru-RU.md](CONTRIBUTING.ru-RU.md) описаны настройка окружения разработки, рекомендации по коду и порядок отправки pull request'ов.

README.zh-CN.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -752,6 +752,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
752752

753753
设置 `telemetry.content_logging` 可在导出数据中包含 LLM 提示词和响应。
754754

755+
**协议选择:** 通过环境变量 `OTEL_EXPORTER_OTLP_PROTOCOL` 选择导出协议:
756+
757+
|| 传输方式 | 说明 |
758+
|---|---|---|
759+
| `grpc`(默认) | gRPC | 默认端口 4317 |
760+
| `http/protobuf` | HTTP | 默认端口 4318 |
761+
762+
**Endpoint 格式:** `telemetry.otlp_endpoint` 的值为 `host:port``http://host:port`,无需包含路径。SDK 会根据 [OTLP 规范](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request)自动追加信号路径(如 `/v1/traces`)。
763+
755764
## 贡献
756765

757766
参见 [CONTRIBUTING.zh-CN.md](CONTRIBUTING.zh-CN.md) 了解开发环境搭建、编码规范以及如何提交 Pull Request。

cmd/opencodereview/emit_run_result_test.go

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ import (
77
"testing"
88
"time"
99

10+
"go.opentelemetry.io/otel"
11+
sdktrace "go.opentelemetry.io/otel/sdk/trace"
12+
1013
"github.com/open-code-review/open-code-review/internal/agent"
1114
"github.com/open-code-review/open-code-review/internal/model"
1215
)
@@ -175,3 +178,61 @@ func TestEmitRunResult_NilQuietHandle(t *testing.T) {
175178
})
176179
_ = got
177180
}
181+
182+
func TestEmitRunResult_JSONTraceIDFromContext(t *testing.T) {
183+
tp := sdktrace.NewTracerProvider()
184+
defer tp.Shutdown(context.Background())
185+
otel.SetTracerProvider(tp)
186+
187+
ctx, span := tp.Tracer("test").Start(context.Background(), "test-root")
188+
wantTraceID := span.SpanContext().TraceID().String()
189+
defer span.End()
190+
191+
ag := &mockResultProvider{
192+
filesReviewed: 2,
193+
inputTokens: 10,
194+
outputTokens: 5,
195+
totalTokens: 15,
196+
}
197+
got := captureStdout(t, func() {
198+
err := emitRunResult(ctx, ag, nil, time.Now(), "json", "developer", nil)
199+
if err != nil {
200+
t.Fatalf("unexpected error: %v", err)
201+
}
202+
})
203+
var out jsonOutput
204+
if err := json.Unmarshal([]byte(got), &out); err != nil {
205+
t.Fatalf("unmarshal: %v", err)
206+
}
207+
if out.TraceID != wantTraceID {
208+
t.Errorf("trace_id = %q, want %q", out.TraceID, wantTraceID)
209+
}
210+
}
211+
212+
func TestEmitRunResult_JSONNoFilesTraceID(t *testing.T) {
213+
tp := sdktrace.NewTracerProvider()
214+
defer tp.Shutdown(context.Background())
215+
otel.SetTracerProvider(tp)
216+
217+
ctx, span := tp.Tracer("test").Start(context.Background(), "test-root")
218+
wantTraceID := span.SpanContext().TraceID().String()
219+
defer span.End()
220+
221+
ag := &mockResultProvider{filesReviewed: 0}
222+
got := captureStdout(t, func() {
223+
err := emitRunResult(ctx, ag, nil, time.Now(), "json", "developer", nil)
224+
if err != nil {
225+
t.Fatalf("unexpected error: %v", err)
226+
}
227+
})
228+
var out jsonOutput
229+
if err := json.Unmarshal([]byte(got), &out); err != nil {
230+
t.Fatalf("unmarshal: %v", err)
231+
}
232+
if out.Status != "skipped" {
233+
t.Errorf("status = %q, want skipped", out.Status)
234+
}
235+
if out.TraceID != wantTraceID {
236+
t.Errorf("trace_id = %q, want %q", out.TraceID, wantTraceID)
237+
}
238+
}

cmd/opencodereview/output.go

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,7 @@ type jsonToolCalls struct {
240240

241241
type jsonOutput struct {
242242
Status string `json:"status"`
243+
TraceID string `json:"trace_id,omitempty"`
243244
Message string `json:"message,omitempty"`
244245
Summary *jsonSummary `json:"summary,omitempty"`
245246
ToolCalls *jsonToolCalls `json:"tool_calls"`
@@ -263,9 +264,10 @@ func outputJSON(comments []model.LlmComment) error {
263264

264265
func outputJSONWithWarnings(comments []model.LlmComment, warnings []agent.AgentWarning,
265266
filesReviewed, inputTokens, outputTokens, totalTokens, cacheReadTokens, cacheWriteTokens int64,
266-
duration time.Duration, projectSummary string, toolCalls map[string]int64) error {
267+
duration time.Duration, projectSummary string, toolCalls map[string]int64, traceID string) error {
267268
out := jsonOutput{
268269
Status: "success",
270+
TraceID: traceID,
269271
Comments: comments,
270272
Summary: &jsonSummary{
271273
FilesReviewed: filesReviewed,
@@ -311,9 +313,10 @@ func outputJSONWithWarnings(comments []model.LlmComment, warnings []agent.AgentW
311313
return enc.Encode(out)
312314
}
313315

314-
func outputJSONNoFiles() error {
316+
func outputJSONNoFiles(traceID string) error {
315317
out := jsonOutput{
316318
Status: "skipped",
319+
TraceID: traceID,
317320
Message: "No supported files changed.",
318321
Comments: []model.LlmComment{},
319322
ToolCalls: &jsonToolCalls{

cmd/opencodereview/output_helpers_test.go

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -168,8 +168,7 @@ func TestOutputJSONWithWarnings_NoCommentsSubtaskError(t *testing.T) {
168168
os.Stdout = w
169169

170170
warnings := []agent.AgentWarning{{Type: "subtask_error", File: "x.go", Message: "fail"}}
171-
err := outputJSONWithWarnings(nil, warnings, 1, 10, 5, 15, 0, 0, time.Second, "", nil)
172-
171+
err := outputJSONWithWarnings(nil, warnings, 1, 10, 5, 15, 0, 0, time.Second, "", nil, "abc123trace")
173172
w.Close()
174173
os.Stdout = old
175174

@@ -188,6 +187,9 @@ func TestOutputJSONWithWarnings_NoCommentsSubtaskError(t *testing.T) {
188187
if !strings.Contains(out.Message, "errors") {
189188
t.Errorf("message = %q, expected to mention errors", out.Message)
190189
}
190+
if out.TraceID != "abc123trace" {
191+
t.Errorf("trace_id = %q, want abc123trace", out.TraceID)
192+
}
191193
}
192194

193195
func TestStatusBadge(t *testing.T) {
@@ -275,8 +277,7 @@ func TestOutputJSONWithWarnings(t *testing.T) {
275277

276278
comments := []model.LlmComment{{Path: "b.go", Content: "test"}}
277279
warnings := []agent.AgentWarning{{Type: "subtask_error", File: "c.go", Message: "failed"}}
278-
err := outputJSONWithWarnings(comments, warnings, 5, 100, 50, 150, 10, 5, 3*time.Second, "summary", map[string]int64{"file_read": 3})
279-
280+
err := outputJSONWithWarnings(comments, warnings, 5, 100, 50, 150, 10, 5, 3*time.Second, "summary", map[string]int64{"file_read": 3}, "trace-xyz-789")
280281
w.Close()
281282
os.Stdout = old
282283

@@ -301,6 +302,9 @@ func TestOutputJSONWithWarnings(t *testing.T) {
301302
if out.ToolCalls == nil || out.ToolCalls.Total != 3 {
302303
t.Errorf("ToolCalls.Total = %v", out.ToolCalls)
303304
}
305+
if out.TraceID != "trace-xyz-789" {
306+
t.Errorf("trace_id = %q, want trace-xyz-789", out.TraceID)
307+
}
304308
}
305309

306310
func TestOutputJSONWithWarnings_NoCommentsNoErrors(t *testing.T) {
@@ -309,8 +313,7 @@ func TestOutputJSONWithWarnings_NoCommentsNoErrors(t *testing.T) {
309313
os.Stdout = w
310314

311315
warnings := []agent.AgentWarning{{Type: "warning", Message: "something"}}
312-
err := outputJSONWithWarnings(nil, warnings, 2, 50, 20, 70, 0, 0, time.Second, "", nil)
313-
316+
err := outputJSONWithWarnings(nil, warnings, 2, 50, 20, 70, 0, 0, time.Second, "", nil, "")
314317
w.Close()
315318
os.Stdout = old
316319

@@ -336,7 +339,7 @@ func TestOutputJSONNoFiles(t *testing.T) {
336339
r, w, _ := os.Pipe()
337340
os.Stdout = w
338341

339-
err := outputJSONNoFiles()
342+
err := outputJSONNoFiles("test-trace-id-456")
340343

341344
w.Close()
342345
os.Stdout = old
@@ -353,6 +356,9 @@ func TestOutputJSONNoFiles(t *testing.T) {
353356
if out.Status != "skipped" {
354357
t.Errorf("status = %q, want skipped", out.Status)
355358
}
359+
if out.TraceID != "test-trace-id-456" {
360+
t.Errorf("trace_id = %q, want test-trace-id-456", out.TraceID)
361+
}
356362
}
357363

358364
func captureStdout(t *testing.T, fn func()) string {

cmd/opencodereview/review_cmd.go

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ import (
1313
"github.com/open-code-review/open-code-review/internal/mcp"
1414
"github.com/open-code-review/open-code-review/internal/telemetry"
1515
"github.com/open-code-review/open-code-review/internal/tool"
16+
17+
"go.opentelemetry.io/otel/codes"
1618
)
1719

1820
func runReview(args []string) error {
@@ -104,11 +106,23 @@ func runReview(args []string) error {
104106

105107
ctx, span := telemetry.StartSpan(context.Background(), "review.run")
106108
defer span.End()
109+
telemetry.SetAttr(span, "review.repo", cc.RepoDir)
110+
telemetry.SetAttr(span, "review.from", opts.from)
111+
telemetry.SetAttr(span, "review.to", opts.to)
112+
telemetry.SetAttr(span, "review.model", rt.Model)
113+
var traceID string
114+
if telemetry.IsEnabled() {
115+
traceID = telemetry.TraceIDFromContext(ctx)
116+
if opts.outputFormat != "json" {
117+
fmt.Fprintf(os.Stderr, "[ocr] TraceID: %s\n", traceID)
118+
}
119+
}
107120
startTime := time.Now()
108121

109122
comments, err := ag.Run(ctx)
110123
if err != nil {
111-
telemetry.SetAttr(span, "error", err.Error())
124+
span.SetStatus(codes.Error, err.Error())
125+
span.RecordError(err)
112126
return fmt.Errorf("review failed: %w", err)
113127
}
114128

cmd/opencodereview/scan_cmd.go

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ package main
33
import (
44
"context"
55
"fmt"
6+
"os"
67
"strings"
78
"time"
89

@@ -11,6 +12,8 @@ import (
1112
"github.com/open-code-review/open-code-review/internal/scan"
1213
"github.com/open-code-review/open-code-review/internal/telemetry"
1314
"github.com/open-code-review/open-code-review/internal/tool"
15+
16+
"go.opentelemetry.io/otel/codes"
1417
)
1518

1619
// scanOptions mirrors reviewOptions for the full-scan subcommand. The two
@@ -207,11 +210,19 @@ func runScan(args []string) error {
207210

208211
ctx, span := telemetry.StartSpan(context.Background(), "scan.run")
209212
defer span.End()
213+
var traceID string
214+
if telemetry.IsEnabled() {
215+
traceID = telemetry.TraceIDFromContext(ctx)
216+
if opts.outputFormat != "json" {
217+
fmt.Fprintf(os.Stderr, "[ocr] TraceID: %s\n", traceID)
218+
}
219+
}
210220
startTime := time.Now()
211221

212222
comments, err := ag.Run(ctx)
213223
if err != nil {
214-
telemetry.SetAttr(span, "error", err.Error())
224+
span.SetStatus(codes.Error, err.Error())
225+
span.RecordError(err)
215226
return fmt.Errorf("scan failed: %w", err)
216227
}
217228

0 commit comments

Comments
 (0)