Skip to content

Commit 378cb97

Browse files
djs55claude
andcommitted
otel: add VM→host span relay for full container lifecycle tracing
Add a Traces.Stream ttrpc service that relays OTel spans from inside the VM back to the host, mirroring the existing Events.Stream pattern. VM-side: - Channel-backed SpanExporter buffers spans (cap 1024, silent drop) - TracerProvider with 100ms batch interval drains into the channel - Traces ttrpc service streams spans to the host on demand - Spans added to: task.Create/Start, runc.NewContainer, crun.create/ start, ctrnetworking.Connect, mountutil.All Host-side (shim): - After VM boot, subscribes to Traces.Stream - Converts proto spans to ReadOnlySpan via tracetest.SpanStub.Snapshot - Corrects VM-vs-host clock skew (RTC has only second-level resolution) using host boot time as sync point - Re-exports via OTLP HTTP with service.name=nerdbox-vm - Spans added to: shim.BundleLoad, shim.VMBoot, shim.BundleTransfer, shim.IOSetup, shim.TaskCreate, shim.Start The relay logic lives in internal/tracing so it can be reused by integration tests without duplication. Integration test extended to exercise Task.Create and Task.Start, producing a 24-span trace visible in Jaeger. Sample trace from TestTraceVMBoot (24 spans, ~80ms total): nerdbox VM.NewInstance+Start........................ 75.3ms nerdbox . libkrun.VMStart........................... 72.4ms nerdbox . libkrun.WaitForTTRPC...................... 72.4ms nerdbox . TTRPC.System.Info......................... 0.2ms nerdbox . . System/Info (interceptor)............... 0.2ms nerdbox . TTRPC.Bundle.Create....................... 0.2ms nerdbox . . Bundle/Create (interceptor)............. 0.2ms nerdbox . TTRPC.Task.Create......................... 1.9ms nerdbox . . Task/Create (interceptor)............... 1.8ms nerdbox-vm . . . task.Create........................... 1.7ms nerdbox-vm . . . . runc.NewContainer................... 1.5ms nerdbox-vm . . . . . mountutil.All..................... 0.0ms nerdbox-vm . . . . . crun.create....................... 1.5ms nerdbox-vm . . . . ctrnetworking.Connect............... 0.0ms nerdbox . TTRPC.Task.Start.......................... 1.3ms nerdbox . . Task/Start (interceptor)................ 1.3ms nerdbox-vm . . . task.Start............................ 1.2ms nerdbox-vm . . . . waitForCtrNetConnect................ 0.0ms nerdbox-vm . . . . container.Start..................... 1.2ms nerdbox-vm . . . . . crun.start........................ 1.2ms Signed-off-by: David Scott <dave@recoil.org> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 8ca50d9 commit 378cb97

File tree

18 files changed

+1822
-22
lines changed

18 files changed

+1822
-22
lines changed

api/next.txtpb

Lines changed: 730 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
/*
2+
Copyright The containerd Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
syntax = "proto3";
18+
19+
package nerdbox.services.traces.v1;
20+
21+
import "google/protobuf/empty.proto";
22+
23+
option go_package = "github.com/containerd/nerdbox/api/services/traces/v1;traces";
24+
25+
service Traces {
26+
// Stream exports spans from the VM to the host.
27+
rpc Stream(google.protobuf.Empty) returns (stream Span);
28+
}
29+
30+
message Span {
31+
bytes trace_id = 1;
32+
bytes span_id = 2;
33+
bytes parent_span_id = 3;
34+
string name = 4;
35+
int64 start_time_unix_nano = 5;
36+
int64 end_time_unix_nano = 6;
37+
int32 kind = 7;
38+
int32 status_code = 8;
39+
string status_message = 9;
40+
repeated KeyValue attributes = 10;
41+
}
42+
43+
message KeyValue {
44+
string key = 1;
45+
string value = 2;
46+
}

api/services/traces/v1/doc.go

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
/*
2+
Copyright The containerd Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
// Package traces relays OTel spans from the VM to the host.
18+
package traces

0 commit comments

Comments
 (0)