Skip to content

Commit 73b5209

Browse files
RoyLinRoyLin
authored andcommitted
release: v0.2.1 — graceful probe degradation + k8s deployment
- Per-probe attach is non-fatal: a tracepoint missing on the running kernel warns and the collector continues with whatever attaches (bails only if zero). One kernel-version difference no longer crashes the whole collector. Validated on KVM: all 12 core probes attach (attached=12 total=12), events flow. - deploy/Dockerfile (multi-stage, eBPF toolchain) + deploy/daemonset.yaml (privileged DaemonSet, NDJSON to stdout, no k8s API/RBAC) make it deployable; README deploy pointer.
1 parent f635327 commit 73b5209

10 files changed

Lines changed: 121 additions & 28 deletions

File tree

CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,21 @@
22

33
All notable changes to a3s-observer will be documented in this file.
44

5+
## [0.2.1] — robustness + deployability
6+
7+
### Changed
8+
9+
- **Graceful probe degradation:** per-probe attach is now non-fatal — a tracepoint missing
10+
on the running kernel logs a warning and the collector continues with whatever attaches
11+
(bails only if zero attach). One kernel-version difference no longer takes down the whole
12+
collector. Validated: all 12 core probes attach, events flow.
13+
14+
### Added
15+
16+
- **Kubernetes deployment:** `deploy/Dockerfile` (multi-stage build with the eBPF
17+
toolchain) + `deploy/daemonset.yaml` (privileged DaemonSet, NDJSON to stdout, no k8s
18+
API/RBAC — pod identity from `/proc/<pid>/cgroup`). Pairs with `deploy/otel-collector.yaml`.
19+
520
## [0.2.0] — production-ready
621

722
Closes the gap between the design and the implementation: every capability the docs

Cargo.lock

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "a3s-observer"
3-
version = "0.2.0"
3+
version = "0.2.1"
44
edition = "2021"
55
license = "MIT"
66
description = "General-purpose, language-agnostic eBPF observability for AI agents (LLM calls, tools, files, network egress)."

README.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -134,11 +134,14 @@ OTEL_EXPORTER_OTLP_ENDPOINT=http://otlp-backend:4317 \
134134
```
135135

136136
Every event is one valid-JSON line (verified: the `filelog` receiver's `json_parser`
137-
ingests them directly), so a3s-observer also drops straight into vector / Loki / `jq`. In
138-
Kubernetes, run a3s-observer as a DaemonSet writing to stdout and let a Collector DaemonSet
139-
tail the node logs. This keeps the always-on probe binary minimal and decoupled from
140-
backend availability; in-process OTLP push is intentionally **not** built — that's the
141-
Collector's job.
137+
ingests them directly), so a3s-observer also drops straight into vector / Loki / `jq`.
138+
139+
**Kubernetes:** build the image with [`deploy/Dockerfile`](deploy/Dockerfile) and deploy
140+
[`deploy/daemonset.yaml`](deploy/daemonset.yaml) (writes NDJSON to stdout); a node-level
141+
OTel Collector DaemonSet tails the container log. No k8s API/RBAC needed — pod identity
142+
comes from `/proc/<pid>/cgroup`. This keeps the always-on probe binary minimal and
143+
decoupled from backend availability; in-process OTLP push is intentionally **not** built —
144+
that's the Collector's job.
142145

143146
## License
144147

a3s-observer-collector/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "a3s-observer-collector"
3-
version = "0.2.0"
3+
version = "0.2.1"
44
edition = "2021"
55
license = "MIT"
66
description = "a3s-observer collector: loads the eBPF probes and exports enriched events."

a3s-observer-collector/src/main.rs

Lines changed: 33 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -27,24 +27,40 @@ async fn main() -> anyhow::Result<()> {
2727
)))
2828
.context("load eBPF object")?;
2929

30-
attach(&mut ebpf, "exec", "syscalls", "sys_enter_execve")?;
31-
attach(&mut ebpf, "tls_write", "syscalls", "sys_enter_write")?;
32-
attach(&mut ebpf, "tls_sendto", "syscalls", "sys_enter_sendto")?;
33-
attach(&mut ebpf, "connect", "syscalls", "sys_enter_connect")?;
34-
attach(&mut ebpf, "dns_query", "syscalls", "sys_enter_sendto")?;
35-
attach(&mut ebpf, "dns_sendmsg", "syscalls", "sys_enter_sendmsg")?;
36-
attach(&mut ebpf, "dns_sendmmsg", "syscalls", "sys_enter_sendmmsg")?;
3730
// File-write capture is opt-in: openat is a firehose on a busy node (e.g. containerd
3831
// unpacking images), and the agent's own writes need downstream identity filtering.
3932
let files = std::env::var_os("A3S_OBSERVER_FILES").is_some();
33+
let mut probes = vec![
34+
("exec", "sys_enter_execve"),
35+
("tls_write", "sys_enter_write"),
36+
("tls_sendto", "sys_enter_sendto"),
37+
("connect", "sys_enter_connect"),
38+
("dns_query", "sys_enter_sendto"),
39+
("dns_sendmsg", "sys_enter_sendmsg"),
40+
("dns_sendmmsg", "sys_enter_sendmmsg"),
41+
("read_enter", "sys_enter_read"),
42+
("recv_enter", "sys_enter_recvfrom"),
43+
("read_exit", "sys_exit_read"),
44+
("recv_exit", "sys_exit_recvfrom"),
45+
("sock_close", "sys_enter_close"),
46+
];
4047
if files {
41-
attach(&mut ebpf, "file_open", "syscalls", "sys_enter_openat")?;
48+
probes.push(("file_open", "sys_enter_openat"));
49+
}
50+
// Per-probe attach is non-fatal: kernels vary, and one missing tracepoint shouldn't take
51+
// down the whole collector — degrade to whatever attaches, fail only if nothing does.
52+
let mut attached = 0usize;
53+
for (prog, tp) in &probes {
54+
match attach(&mut ebpf, prog, "syscalls", tp) {
55+
Ok(()) => attached += 1,
56+
Err(e) => {
57+
tracing::warn!(probe = prog, error = %e, "probe failed to attach — continuing")
58+
}
59+
}
60+
}
61+
if attached == 0 {
62+
anyhow::bail!("no eBPF probes could be attached");
4263
}
43-
attach(&mut ebpf, "read_enter", "syscalls", "sys_enter_read")?;
44-
attach(&mut ebpf, "recv_enter", "syscalls", "sys_enter_recvfrom")?;
45-
attach(&mut ebpf, "read_exit", "syscalls", "sys_exit_read")?;
46-
attach(&mut ebpf, "recv_exit", "syscalls", "sys_exit_recvfrom")?;
47-
attach(&mut ebpf, "sock_close", "syscalls", "sys_enter_close")?;
4864

4965
// A3S_OBSERVER_JSON=1 → NDJSON (pipe to vector/Loki/jq); otherwise human-readable log.
5066
let exporter: Box<dyn Exporter> = if std::env::var_os("A3S_OBSERVER_JSON").is_some() {
@@ -82,9 +98,11 @@ async fn main() -> anyhow::Result<()> {
8298
)?;
8399

84100
tracing::info!(
101+
attached,
102+
total = probes.len(),
85103
files,
86-
"a3s-observer-collector: exec + TLS-SNI + connect + dns + LLM-metrics probes attached \
87-
(file-write capture: set A3S_OBSERVER_FILES=1); streaming (Ctrl-C to stop)"
104+
"a3s-observer-collector: probes attached (file-write capture: set A3S_OBSERVER_FILES=1); \
105+
streaming (Ctrl-C to stop)"
88106
);
89107

90108
let mut sigint = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::interrupt())?;

a3s-observer-common/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "a3s-observer-common"
3-
version = "0.2.0"
3+
version = "0.2.1"
44
edition = "2021"
55
license = "MIT"
66
description = "Shared no_std types crossing the eBPF <-> userspace boundary for a3s-observer."

a3s-observer-ebpf/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "a3s-observer-ebpf"
3-
version = "0.2.0"
3+
version = "0.2.1"
44
edition = "2021"
55
license = "MIT"
66
publish = false

deploy/Dockerfile

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Build the collector (with the eBPF toolchain) then ship just the binary.
2+
# Build from the crate root: docker build -f deploy/Dockerfile -t a3s-observer:0.2.0 .
3+
#
4+
# bpf-linker borrows rustc's bundled LLVM (aya-rustc-llvm-proxy), so no system LLVM is
5+
# needed; build.rs compiles the eBPF crate with the nightly toolchain + rust-src.
6+
7+
FROM rust:1-bookworm AS build
8+
RUN rustup toolchain install nightly --component rust-src \
9+
&& cargo install bpf-linker
10+
WORKDIR /src
11+
COPY . .
12+
RUN cargo build --release -p a3s-observer-collector
13+
14+
FROM debian:bookworm-slim
15+
COPY --from=build /src/target/release/a3s-observer-collector /usr/local/bin/a3s-observer-collector
16+
ENV A3S_OBSERVER_JSON=1
17+
ENTRYPOINT ["/usr/local/bin/a3s-observer-collector"]

deploy/daemonset.yaml

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Run a3s-observer on every node. It writes NDJSON to stdout; a node-level OpenTelemetry
2+
# Collector then tails the container log and ships OTLP (see deploy/otel-collector.yaml).
3+
#
4+
# docker build -f deploy/Dockerfile -t <registry>/a3s-observer:0.2.0 . && docker push ...
5+
# kubectl apply -f deploy/daemonset.yaml
6+
#
7+
# No k8s API access / RBAC is needed: pod-UID + container-id come from /proc/<pid>/cgroup.
8+
apiVersion: apps/v1
9+
kind: DaemonSet
10+
metadata:
11+
name: a3s-observer
12+
namespace: observability
13+
labels: { app: a3s-observer }
14+
spec:
15+
selector:
16+
matchLabels: { app: a3s-observer }
17+
template:
18+
metadata:
19+
labels: { app: a3s-observer }
20+
spec:
21+
hostPID: true # resolve /proc/<pid> for host processes (identity)
22+
containers:
23+
- name: a3s-observer
24+
image: <registry>/a3s-observer:0.2.0 # built from deploy/Dockerfile
25+
securityContext:
26+
# eBPF load + tracepoint attach. On kernels ≥5.8 you can drop `privileged` and
27+
# use capabilities: [BPF, PERFMON, SYS_RESOURCE] instead.
28+
privileged: true
29+
env:
30+
- { name: A3S_OBSERVER_JSON, value: "1" }
31+
# - { name: A3S_OBSERVER_FILES, value: "1" } # opt in to file-write capture
32+
volumeMounts:
33+
- { name: sys, mountPath: /sys, readOnly: true } # tracepoint ids under /sys
34+
resources:
35+
requests: { cpu: 50m, memory: 64Mi }
36+
limits: { memory: 256Mi }
37+
volumes:
38+
- { name: sys, hostPath: { path: /sys } }
39+
tolerations:
40+
- operator: Exists # run on every node, including control-plane

0 commit comments

Comments
 (0)