feat: Add FFI for trace exporter (#1952)

# What does this PR do?

Add span-building and trace-sending FFI functions to `libdd-data-pipeline-ffi`, enabling language tracers to construct spans field-by-field through the C API instead of passing pre-serialized msgpack.

New types:
- `TracerSpan`: opaque handle wrapping `Span<BytesData>`
- `TracerTraceChunks`: opaque handle wrapping `Vec<Vec<SpanBytes>>`

New functions:
- `ddog_tracer_span_new`, `_free`, `_set_meta`, `_set_metric`
- `ddog_tracer_trace_chunks_new`, `_free`, `_begin_chunk`, `_push_span`
- `ddog_trace_exporter_send_trace_chunks`

# Motivation

The Ruby tracer needs a native trace export path that bypasses Ruby-side msgpack serialization. This provides the FFI surface for it; the corresponding C extension in `dd-trace-rb` calls these functions.

Continuation of the prototype in #1661, reworked to stay close to `main`: the generic `TraceExporter<H>` and `SharedRuntime` are untouched; this is purely additive.

# Additional Notes

Spans are consumed on push, chunks are consumed on send — single-ownership enforced at the API level.

`libdd-trace-utils` is promoted from dev-dependency to regular dependency since the FFI crate now needs `SpanBytes` at build time.

Sibling PR for Ruby is at: DataDog/dd-trace-rb#5690

# How to test the change?

`cargo test -p libdd-data-pipeline-ffi --lib` — 39 tests, all passing. The existing `trace_exporter` tests are unaffected.


Co-authored-by: edmund.kump <edmund.kump@datadoghq.com>

Author: lloeki

libdd-data-pipeline-ffi/Cargo.toml

@@ -35,4 +35,5 @@ libdd-data-pipeline = { path = "../libdd-data-pipeline" }
 libdd-shared-runtime = { version = "1.0.0", path = "../libdd-shared-runtime" }
 libdd-common-ffi = { path = "../libdd-common-ffi", default-features = false }
 libdd-tinybytes = { path = "../libdd-tinybytes" }
+libdd-trace-utils = { path = "../libdd-trace-utils" }
 tracing = { version = "0.1", default-features = false }

libdd-data-pipeline-ffi/cbindgen.toml

@@ -11,12 +11,14 @@ include_guard = "DDOG_DATA_PIPELINE_H"
 includes = ["common.h"]
 after_includes = """
 typedef struct ddog_TraceExporter ddog_TraceExporter;
+typedef struct ddog_TracerSpan ddog_TracerSpan;
+typedef struct ddog_TracerTraceChunks ddog_TracerTraceChunks;
 """
 
 [export]
 prefix = "ddog_"
 renaming_overrides_prefixing = true
-exclude = ["TraceExporter"]
+exclude = ["TraceExporter", "TracerSpan", "TracerTraceChunks"]
 
 [export.rename]
 "ByteSlice" = "ddog_ByteSlice"
@@ -27,6 +29,9 @@ exclude = ["TraceExporter"]
 "ExporterResponse" = "ddog_TraceExporterResponse"
 "ExporterErrorCode" = "ddog_TraceExporterErrorCode"
 "ExporterError" = "ddog_TraceExporterError"
+"TracerSpan" = "ddog_TracerSpan"
+"TracerSpanFields" = "ddog_TracerSpanFields"
+"TracerTraceChunks" = "ddog_TracerTraceChunks"
 
 [export.mangle]
 rename_types = "PascalCase"
@@ -40,4 +45,4 @@ must_use = "DDOG_CHECK_RETURN"
 
 [parse]
 parse_deps = true
-include = ["libdd-common", "libdd-common-ffi", "libdd-shared-runtime", "libdd-data-pipeline"]
+include = ["libdd-common", "libdd-common-ffi", "libdd-shared-runtime", "libdd-data-pipeline", "libdd-trace-utils"]

libdd-data-pipeline-ffi/src/lib.rs

@@ -9,6 +9,7 @@
 mod error;
 mod response;
 mod trace_exporter;
+mod tracer;
 
 #[cfg(all(feature = "catch_panic", panic = "unwind"))]
 macro_rules! catch_panic {

libdd-data-pipeline-ffi/src/trace_exporter.rs

@@ -14,7 +14,7 @@ use libdd_data_pipeline::trace_exporter::{
     TraceExporterInputFormat, TraceExporterOutputFormat,
 };
 
-type TraceExporter = GenericTraceExporter<NativeCapabilities>;
+pub(crate) type TraceExporter = GenericTraceExporter<NativeCapabilities>;
 
 use libdd_shared_runtime::SharedRuntime;
 use std::{ptr::NonNull, sync::Arc, time::Duration};