Commit 378cb97
otel: add VM→host span relay for full container lifecycle tracing
Add a Traces.Stream ttrpc service that relays OTel spans from inside
the VM back to the host, mirroring the existing Events.Stream pattern.
VM-side:
- Channel-backed SpanExporter buffers spans (cap 1024, silent drop)
- TracerProvider with 100ms batch interval drains into the channel
- Traces ttrpc service streams spans to the host on demand
- Spans added to: task.Create/Start, runc.NewContainer, crun.create/
start, ctrnetworking.Connect, mountutil.All
Host-side (shim):
- After VM boot, subscribes to Traces.Stream
- Converts proto spans to ReadOnlySpan via tracetest.SpanStub.Snapshot
- Corrects VM-vs-host clock skew (RTC has only second-level resolution)
using host boot time as sync point
- Re-exports via OTLP HTTP with service.name=nerdbox-vm
- Spans added to: shim.BundleLoad, shim.VMBoot, shim.BundleTransfer,
shim.IOSetup, shim.TaskCreate, shim.Start
The relay logic lives in internal/tracing so it can be reused by
integration tests without duplication.
Integration test extended to exercise Task.Create and Task.Start,
producing a 24-span trace visible in Jaeger.
Sample trace from TestTraceVMBoot (24 spans, ~80ms total):
nerdbox VM.NewInstance+Start........................ 75.3ms
nerdbox . libkrun.VMStart........................... 72.4ms
nerdbox . libkrun.WaitForTTRPC...................... 72.4ms
nerdbox . TTRPC.System.Info......................... 0.2ms
nerdbox . . System/Info (interceptor)............... 0.2ms
nerdbox . TTRPC.Bundle.Create....................... 0.2ms
nerdbox . . Bundle/Create (interceptor)............. 0.2ms
nerdbox . TTRPC.Task.Create......................... 1.9ms
nerdbox . . Task/Create (interceptor)............... 1.8ms
nerdbox-vm . . . task.Create........................... 1.7ms
nerdbox-vm . . . . runc.NewContainer................... 1.5ms
nerdbox-vm . . . . . mountutil.All..................... 0.0ms
nerdbox-vm . . . . . crun.create....................... 1.5ms
nerdbox-vm . . . . ctrnetworking.Connect............... 0.0ms
nerdbox . TTRPC.Task.Start.......................... 1.3ms
nerdbox . . Task/Start (interceptor)................ 1.3ms
nerdbox-vm . . . task.Start............................ 1.2ms
nerdbox-vm . . . . waitForCtrNetConnect................ 0.0ms
nerdbox-vm . . . . container.Start..................... 1.2ms
nerdbox-vm . . . . . crun.start........................ 1.2ms
Signed-off-by: David Scott <dave@recoil.org>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 8ca50d9 commit 378cb97
File tree
18 files changed
+1822
-22
lines changed- api
- proto/nerdbox/services/traces/v1
- services/traces/v1
- cmd/vminitd
- docs
- integration
- internal
- shim/task
- tracing
- vminit
- ctrnetworking
- process
- runc
- task
- tracing
- vm/libkrun
- plugins/vminit/tracing
18 files changed
+1822
-22
lines changedLarge diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
0 commit comments