Skip to content

Commit a9279fb

Browse files
khanayan123claude
andcommitted
feat(llmobs/export): offline LLM Obs span & evaluation export
Add a public llmobs/export package: an offline client for exporting already-built LLM Observability spans and evaluations to Datadog intake (direct) or through the Agent EVP proxy. This is an export/transport API, not live instrumentation — it never starts the tracer, depends on the live LLM Obs lifecycle, or replays payloads through StartSpan/Finish. Implements the "Raw LLM Observability Export" part of the RFC. It lets offline reconstruction tools (e.g. Trajectory) depend on the SDK for the export mechanics — endpoint derivation, auth, HTTP, retry classification, size handling, and structured per-request results — while keeping reconstruction, projection, privacy, dedup and durable retry above the SDK. - Client.ExportSpans / ExportEvaluations over caller-built SpanEvent / EvaluationMetric payloads, lowered to the LLM Obs wire shape. - Caller-assigned trace/span/parent IDs preserved verbatim as opaque string payload fields; never routed into APM span IDs or sampling. - Chunking (50 spans, 1000 evals per request), best-effort 5MB size guard (truncates input/output), and per-request ExportResult with validation attribution. - Datadog route (DD-API-KEY) and Agent EVP route; reuses the internal LLM Obs transport for auth/routing/retry via a new transport.Post primitive. Multi-destination is modeled as one isolated Client per destination. A sibling otlp/export package is a stacked follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 0e9c636 commit a9279fb

9 files changed

Lines changed: 1305 additions & 0 deletions

File tree

CONTRIBUTING.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,10 @@ Sample PR: <https://github.com/DataDog/dd-trace-go/pull/3365>
296296

297297
Please view our contrib [README.md](contrib/README.md) for information on integrations. If you need support for a new integration, please file an issue to discuss before opening a PR.
298298

299+
### Offline export clients
300+
301+
The [`llmobs/export`](./llmobs/export) package provides an offline client for exporting already-built LLM Observability spans and evaluations directly to Datadog intake or through the Agent EVP proxy. It is an export/transport API — not live instrumentation — for tools that reconstruct telemetry offline and need the SDK to own endpoint derivation, auth, retry, size handling, and structured results. Caller-assigned IDs are preserved as opaque payload fields and are never routed into APM span IDs or sampling. A sibling `otlp/export` package for offline OTLP trace/metric/log export is planned as a follow-up.
302+
299303
### Working with environment variables
300304

301305
When working with environment variables, direct use of `os.Getenv` and `os.LookupEnv` is not permitted. Instead, all environment variables must be validated against an [allowed list](./internal/env/supported_configurations.gen.go) using `env.Get` and `env.Lookup` from the [`internal/env`](./internal/env.go) package (or [`instrumentation/env`](./instrumentation/env/env.go) when working on contrib packages). This validation system helps us automatically detect newly introduced variables and ensures they are properly documented and tracked.

internal/llmobs/transport/transport.go

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,15 @@ const (
4646
subdomainDNE = "api"
4747
)
4848

49+
// Exported endpoint paths and EVP subdomains for offline export clients
50+
// (see llmobs/export). They identify the LLM Obs span and evaluation intakes.
51+
const (
52+
PathLLMSpans = endpointLLMSpan
53+
SubdomainLLMSpans = subdomainLLMSpan
54+
PathEvalMetrics = endpointEvalMetric
55+
SubdomainEval = subdomainEvalMetric
56+
)
57+
4958
const (
5059
defaultSite = "datadoghq.com"
5160
defaultMaxRetries uint = 3
@@ -268,6 +277,86 @@ func (c *Transport) request(ctx context.Context, method, path, subdomain string,
268277
return backoff.Retry(ctx, doRequest, backoff.WithBackOff(backoffStrat), backoff.WithMaxTries(defaultMaxRetries))
269278
}
270279

280+
// Result reports the outcome of a single Post performed by an offline export
281+
// client. It surfaces the final HTTP status, the number of attempts made, the
282+
// (bounded) response body, and whether the failure class was retriable.
283+
type Result struct {
284+
StatusCode int
285+
Attempts int
286+
Body []byte
287+
Retriable bool
288+
}
289+
290+
// Post sends an already-encoded body to path under the given EVP subdomain using
291+
// the transport's routing (agentless direct intake vs. Agent EVP proxy), auth
292+
// headers, and bounded retry policy, and reports a structured Result.
293+
//
294+
// It is intended for offline export clients (llmobs/export) that build their own
295+
// payloads and need per-request outcomes. Retry classification matches the rest
296+
// of the transport (5xx/408/425/429 retriable, other 4xx permanent); callers
297+
// must not layer additional retry on top of it.
298+
func (c *Transport) Post(ctx context.Context, path, subdomain, contentType string, body []byte) (Result, error) {
299+
urlStr := c.baseURL(subdomain) + path
300+
backoffStrat := defaultBackoffStrategy()
301+
302+
var attempts int
303+
var last Result
304+
305+
op := func() (Result, error) {
306+
attempts++
307+
req, err := http.NewRequestWithContext(ctx, http.MethodPost, urlStr, bytes.NewReader(body))
308+
if err != nil {
309+
return Result{}, backoff.Permanent(err)
310+
}
311+
req.Header.Set("Content-Type", contentType)
312+
for key, val := range c.defaultHeaders {
313+
req.Header.Set(key, val)
314+
}
315+
if !c.agentless {
316+
req.Header.Set(headerEVPSubdomain, subdomain)
317+
}
318+
319+
timeoutCtx, cancel := context.WithTimeout(ctx, defaultTimeout)
320+
defer cancel()
321+
req = req.WithContext(timeoutCtx)
322+
323+
resp, err := c.httpClient.Do(req)
324+
if err != nil {
325+
// Network/transport error is retriable, but a caller-cancelled or
326+
// expired parent context is not a transient server condition.
327+
last = Result{Retriable: ctx.Err() == nil}
328+
return Result{}, err
329+
}
330+
defer resp.Body.Close()
331+
332+
code := resp.StatusCode
333+
respBody, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
334+
if code >= 200 && code <= 299 {
335+
last = Result{StatusCode: code, Body: respBody}
336+
return last, nil
337+
}
338+
if isRetriableStatus(code) {
339+
last = Result{StatusCode: code, Body: respBody, Retriable: true}
340+
return Result{}, fmt.Errorf("request failed with transient http status code: %d", code)
341+
}
342+
if code == http.StatusTooManyRequests {
343+
last = Result{StatusCode: code, Body: respBody, Retriable: true}
344+
wait := parseRetryAfter(resp.Header)
345+
return Result{}, backoff.RetryAfter(int(wait.Seconds()))
346+
}
347+
last = Result{StatusCode: code, Body: respBody}
348+
return Result{}, backoff.Permanent(fmt.Errorf("request failed with http status code: %d", code))
349+
}
350+
351+
res, err := backoff.Retry(ctx, op, backoff.WithBackOff(backoffStrat), backoff.WithMaxTries(defaultMaxRetries))
352+
if err != nil {
353+
last.Attempts = attempts
354+
return last, err
355+
}
356+
res.Attempts = attempts
357+
return res, nil
358+
}
359+
271360
func readErrorBody(resp *http.Response) string {
272361
if resp == nil || resp.Body == nil {
273362
return ""

llmobs/export/client.go

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
// Unless explicitly stated otherwise all files in this repository are licensed
2+
// under the Apache License Version 2.0.
3+
// This product includes software developed at Datadog (https://www.datadoghq.com/).
4+
// Copyright 2026 Datadog, Inc.
5+
6+
package export
7+
8+
import (
9+
"errors"
10+
"fmt"
11+
"net/http"
12+
"net/url"
13+
"strings"
14+
15+
"github.com/DataDog/dd-trace-go/v2/internal/llmobs/config"
16+
"github.com/DataDog/dd-trace-go/v2/internal/llmobs/transport"
17+
"github.com/DataDog/dd-trace-go/v2/internal/version"
18+
)
19+
20+
const (
21+
defaultSite = "datadoghq.com"
22+
defaultSpanBatchSize = 50
23+
defaultEvalBatchSize = 1000
24+
defaultMaxSpanBytes = 5 << 20 // 5 MiB
25+
)
26+
27+
// Config configures a [Client]. A client targets exactly one destination; build
28+
// several clients for multi-destination export.
29+
//
30+
// Routing:
31+
// - Datadog route (default): leave AgentURL empty and set Site + APIKey. The
32+
// client posts directly to the LLM Obs intake derived from Site with the
33+
// DD-API-KEY header.
34+
// - Agent route: set AgentURL to a Datadog Agent base URL. The client posts
35+
// through the Agent's EVP proxy and does not inject Datadog auth.
36+
type Config struct {
37+
// Site is the Datadog site (e.g. "datadoghq.com"). Defaults to datadoghq.com.
38+
Site string
39+
// APIKey is the Datadog API key, required for the Datadog route.
40+
APIKey string
41+
// AgentURL, if set, routes export through the Agent EVP proxy instead of the
42+
// direct Datadog intake (e.g. "http://localhost:8126").
43+
AgentURL string
44+
45+
// Service, Env and Version are stamped onto spans when absent (Service as the
46+
// top-level service field; Env and Version as env:/version: tags).
47+
Service string
48+
Env string
49+
Version string
50+
// MLApp is the default ml_app for evaluations when a metric omits it.
51+
MLApp string
52+
53+
// HTTPClient overrides the default HTTP client.
54+
HTTPClient *http.Client
55+
56+
// SpanBatchSize is the max spans per request (default 50).
57+
SpanBatchSize int
58+
// EvalBatchSize is the max evaluations per request (default 1000).
59+
EvalBatchSize int
60+
// MaxSpanPayloadBytes is the max encoded span-request size before input/output
61+
// values are truncated (default 5 MiB). Truncation is best-effort: only
62+
// input/output are shrunk, so payloads dominated by other fields may still
63+
// exceed the limit.
64+
MaxSpanPayloadBytes int
65+
}
66+
67+
// Client is an offline exporter for LLM Obs spans and evaluations. It is safe
68+
// for concurrent use.
69+
type Client struct {
70+
transport *transport.Transport
71+
72+
service string
73+
env string
74+
version string
75+
mlApp string
76+
77+
spanBatch int
78+
evalBatch int
79+
maxSpanBytes int
80+
}
81+
82+
// New builds a Client from cfg.
83+
func New(cfg Config) (*Client, error) {
84+
site := cfg.Site
85+
if site == "" {
86+
site = defaultSite
87+
}
88+
agentless := cfg.AgentURL == ""
89+
if agentless && cfg.APIKey == "" {
90+
return nil, errors.New("llmobs/export: APIKey is required for direct Datadog export; set AgentURL to route via the Agent")
91+
}
92+
93+
var agentURL *url.URL
94+
if cfg.AgentURL != "" {
95+
// Trim a trailing slash so the EVP proxy path is not doubled.
96+
u, err := url.Parse(strings.TrimRight(cfg.AgentURL, "/"))
97+
if err != nil {
98+
return nil, fmt.Errorf("llmobs/export: invalid AgentURL: %w", err)
99+
}
100+
agentURL = u
101+
}
102+
103+
icfg := &config.Config{
104+
ResolvedAgentlessEnabled: agentless,
105+
TracerConfig: config.TracerConfig{
106+
Site: site,
107+
APIKey: cfg.APIKey,
108+
AgentURL: agentURL,
109+
HTTPClient: cfg.HTTPClient,
110+
Service: cfg.Service,
111+
Env: cfg.Env,
112+
Version: cfg.Version,
113+
},
114+
}
115+
if icfg.TracerConfig.HTTPClient == nil {
116+
icfg.TracerConfig.HTTPClient = icfg.DefaultHTTPClient()
117+
}
118+
119+
c := &Client{
120+
transport: transport.New(icfg),
121+
service: cfg.Service,
122+
env: cfg.Env,
123+
version: cfg.Version,
124+
mlApp: cfg.MLApp,
125+
spanBatch: orDefault(cfg.SpanBatchSize, defaultSpanBatchSize),
126+
evalBatch: orDefault(cfg.EvalBatchSize, defaultEvalBatchSize),
127+
maxSpanBytes: orDefault(cfg.MaxSpanPayloadBytes, defaultMaxSpanBytes),
128+
}
129+
return c, nil
130+
}
131+
132+
func orDefault(v, def int) int {
133+
if v <= 0 {
134+
return def
135+
}
136+
return v
137+
}
138+
139+
// tracerVersion is the value stamped into _dd.tracer_version.
140+
func tracerVersion() string {
141+
return version.Tag
142+
}

llmobs/export/doc.go

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Unless explicitly stated otherwise all files in this repository are licensed
2+
// under the Apache License Version 2.0.
3+
// This product includes software developed at Datadog (https://www.datadoghq.com/).
4+
// Copyright 2026 Datadog, Inc.
5+
6+
// Package export provides an offline client for exporting already-built LLM
7+
// Observability spans and evaluations to Datadog.
8+
//
9+
// It is an offline export API, not a live instrumentation API: it does not start
10+
// the tracer, does not depend on the live LLM Obs lifecycle, and never replays
11+
// payloads through StartSpan/Finish. It is intended for tools that reconstruct
12+
// telemetry offline (for example, rebuilding finished agent sessions from logs)
13+
// and need the SDK to own the export mechanics — endpoint derivation, auth,
14+
// HTTP transport, retry classification, size handling, and structured results.
15+
//
16+
// Caller-assigned IDs (trace_id, span_id, parent_id) are preserved verbatim as
17+
// LLM Obs payload fields. They are opaque strings (decimal uint64 strings are
18+
// valid; 128-bit hex is not required) and are never routed into APM span IDs,
19+
// APM trace IDs, or sampling.
20+
//
21+
// Multi-destination routing is modeled as one isolated [Client] per destination:
22+
// construct N clients, each with its own site, API key, service and defaults.
23+
// The client does not deduplicate, spool, or durably retry — callers retain
24+
// ownership of reconstruction, projection, privacy filtering, deterministic ID
25+
// generation, dedup, and outbox/backfill behavior.
26+
package export

0 commit comments

Comments
 (0)