Skip to content

Commit 586503a

Browse files
RoyLinRoyLin
authored andcommitted
docs: refactor README from first principles, grounded in code facts (v0.6.0)
Single source of truth from the actual surface: 6 event types (ToolExec/Egress/Dns/LlmCall/ FileAccess/SslContent), 7 signals incl opt-in file + OpenSSL content, 15 providers, both guards (enforce: connect4 egress; fileguard: fanotify open+exec, hot-reloaded), 3 binaries, 4 env knobs, and the production/deploy surface (signed image, liveness, OTel). Updated the architecture diagram to show the egress + file/exec intervention paths.
1 parent e213129 commit 586503a

1 file changed

Lines changed: 78 additions & 76 deletions

File tree

README.md

Lines changed: 78 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -1,98 +1,91 @@
11
# a3s-observer
22

3-
Kernel-level **eBPF** observability — and optional intervention — for AI agents. It turns
3+
Kernel-level **eBPF observability — and optional intervention — for AI agents.** It turns
44
syscalls and network events into agent-semantic telemetry (which agent ran which tool, made
55
which LLM call, touched which files, reached which endpoint) with **zero changes to the agent
6-
and no per-language instrumentation**. The same kernel vantage point can also **intervene**
7-
deny an agent's egress or file access from an external policy.
6+
and no per-language instrumentation** — and the same kernel vantage point can **intervene**:
7+
deny an agent's egress, file access, or process execution from an external policy.
88

9-
Built and validated on Linux 6.8 (Aya, no uprobes). Observe-only by default; intervention is
10-
opt-in and never affects the observe path.
9+
Built and validated on Linux 6.8 (Aya). Observe-only by default; every intervention is opt-in
10+
and isolated from the observe path.
1111

1212
## Architecture
1313

1414
A **minimal core** — probes → loader → identity → correlation → export — with everything else a
15-
swappable trait. Two paths share one kernel vantage point: **observe** is always-on and passive;
16-
**intervene** is opt-in and isolated, so a policy mistake can never break observability.
15+
swappable trait. Two paths share one kernel vantage point: **observe** is always-on and passive
16+
(tracepoints can't block); **intervene** is opt-in (cgroup-BPF / fanotify), so a policy mistake
17+
can never break observability.
1718

1819
```
19-
AI agent + its tool subprocesses unmodified · any language
20-
21-
│ execve · connect · TLS ClientHello · DNS · openat · read / recv
22-
════════════╪══════════════════════════════════════════════ KERNEL · eBPF (no uprobes)
23-
24-
OBSERVE (passive, always on) INTERVENE (opt-in)
25-
exec connect sni dns connect4 egress guard (cgroup-scoped)
26-
llm-metrics file fanotify file guard
27-
│ ▲
28-
│ ring buffers │ allow / DENY → EPERM
29-
════════════╪═════════════════════════════════════════════╪═══════════ USERSPACE
30-
▼ │
31-
a3s-observer-collector (Aya) a3s-observer-enforce · fileguard
32-
· identity k8s pod / proc / comm ▲
33-
· correlate (pid,fd) → peer │ reads
34-
· export NDJSON / human log external policy file
35-
│ (your controller · OPA · a script)
36-
37-
OTel Collector → your backend
20+
AI agent + its tool subprocesses unmodified · any language
21+
22+
│ execve · connect · TLS ClientHello · DNS · openat · read/recv · SSL_*
23+
══════════╪═══════════════════════════════════════════════ KERNEL (eBPF + fanotify)
24+
25+
OBSERVE (passive, always-on) INTERVENE (opt-in, external policy)
26+
exec connect sni dns enforce → cgroup/connect4: deny egress
27+
llm-metrics file* ssl-content* fileguard → fanotify: deny open + exec
28+
29+
│ ring buffers
30+
══════════╪═══════════════════════════════════════════════ USERSPACE
31+
32+
a3s-observer-collector (Aya loader)
33+
identity (k8s pod / proc / comm) · correlate (pid,fd)→peer · export NDJSON
34+
35+
36+
OTel Collector → your backend * opt-in: A3S_OBSERVER_FILES / _SSL
3837
```
3938

40-
The four core pieces — **probes** (language-agnostic kernel hooks), **identity** (attribute an
41-
event to an agent), **correlation** (fuse provider + endpoint), **export** (NDJSON) — plus the
42-
opt-in **guards**, are detailed below.
43-
4439
## Observe — who / what / where
4540

46-
One event answers **who / what / where**: who (process or k8s pod), what (tool, file, or LLM
47-
provider + bytes/latency/TTFT), where (peer IP / hostname).
41+
One event answers **who** (process or k8s pod) / **what** (tool, file, LLM provider + bytes /
42+
latency / TTFT, or plaintext) / **where** (peer IP / hostname).
4843

49-
| probe | kernel hook | signal |
44+
| signal | kernel hook | event |
5045
|---|---|---|
51-
| `exec` | `sys_enter_execve` | tools / subprocesses (argv, comm, uid) |
52-
| `connect` | `sys_enter_connect` | peer IP:port |
53-
| `sni` | TLS ClientHello (plaintext `server_name`) | LLM provider + endpoint |
54-
| `dns` | `sendto` / `sendmsg` / `sendmmsg` to :53 | resolved hostnames |
55-
| LLM metrics | per-socket `read` / `recv` + `close` | req/resp wire bytes, latency, TTFT |
56-
| `file` | `sys_enter_openat` (write opens) | files written — **opt-in** (`A3S_OBSERVER_FILES=1`; high-volume) |
57-
| `ssl` content | uprobes on OpenSSL `SSL_write` / `SSL_read` | request/response **plaintext** (prompt/completion) — **opt-in** (`A3S_OBSERVER_SSL=1`; OpenSSL-only, not language-agnostic) |
46+
| `exec` | `sys_enter_execve` | `ToolExec` — tool / subprocess (argv, comm, uid) |
47+
| `connect` | `sys_enter_connect` | `Egress`peer IP:port |
48+
| `sni` | TLS ClientHello (plaintext `server_name`) | LLM **provider** + endpoint |
49+
| `dns` | `sendto` / `sendmsg` / `sendmmsg` to :53 | `Dns`resolved hostname |
50+
| llm metrics | per-socket `read`/`recv` + `close` | `LlmCall` req/resp wire bytes, latency, TTFT |
51+
| `file`\* | `sys_enter_openat` (write opens) | `FileAccess` — files written (`A3S_OBSERVER_FILES=1`) |
52+
| `ssl`\* | OpenSSL `SSL_write` / `SSL_read` uprobes | `SslContent`request/response plaintext (`A3S_OBSERVER_SSL=1`) |
5853

5954
Userspace enriches each event with **identity** (k8s cgroup→pod, `/proc` comm+ppid, or an
60-
in-kernel `comm` fallback for short-lived processes) and a `(pid,fd)→peer` **correlation**,
61-
then exports **NDJSON** (or a human-readable log).
55+
in-kernel `comm` fallback for short-lived processes), a `(pid,fd)→peer` **correlation**, and
56+
**provider** classification (SNI → 15 LLM providers); then exports **NDJSON** (or a human log).
6257

63-
## Intervene (opt-in)
58+
## Intervene — egress / file / exec (opt-in)
6459

65-
The same vantage point can enforce an **external** policy: the policy lives outside the binary
66-
(a plain file any controller can write), and the kernel asks a guard allow/deny per action.
67-
The observe-only core is untouched.
60+
The same vantage point enforces an **external policy** — a plain file any controller writes; the
61+
kernel asks a guard allow/deny per action. The observe-only core is untouched. Both guards are
62+
**hot-reloaded** and **KVM-validated** (a denied action returns `EPERM`):
6863

6964
| guard | mechanism | denies |
7065
|---|---|---|
71-
| `a3s-observer-enforce` | `cgroup/connect4` eBPF | egress to policy IPs/hosts — **cgroup-scoped**, fail-open |
72-
| `a3s-observer-fileguard` | fanotify `FAN_OPEN_PERM` + `FAN_OPEN_EXEC_PERM` | `open()` **and** `exec` of policy-listed files; policy **hot-reloaded** |
66+
| `a3s-observer-enforce` | eBPF `cgroup/connect4` | `connect()` to policy IPs/hosts — cgroup-scoped, fail-open, DNS-re-resolved |
67+
| `a3s-observer-fileguard` | fanotify `FAN_OPEN_PERM` + `FAN_OPEN_EXEC_PERM` | `open()` **and** `exec` of policy-listed files |
7368

74-
Both KVM-validated: a denied connect / file-open returns `EPERM`, everything else is
75-
untouched. Drive it in-process (the `Policy` trait) or out-of-process — `scripts/example-controller.py`
69+
Drive it in-process (the `Policy` trait) or out-of-process — `scripts/example-controller.py`
7670
turns observed events into a deny-list. See [`docs/enforcement.md`](docs/enforcement.md).
7771

7872
## Why eBPF, and the boundary
7973

80-
- **Zero-instrumentation, language-agnostic** — observe or guard any agent
81-
(Python/Node/Go/Rust) without touching its code, including its tool subprocesses.
82-
- **Sees what the app won't report** — real execs, file I/O, network egress.
83-
- **Kernel hooks only in the always-on core, no uprobes.** The deliberate trade-off: the core
84-
gives **no LLM prompt / completion content**. That content *is* available via an **opt-in**
85-
OpenSSL uprobe extension (`A3S_OBSERVER_SSL=1`) that captures `SSL_write`/`SSL_read`
86-
plaintext — kept out of the core because a uprobe binds to a library symbol, so it's OpenSSL
87-
only, not language-agnostic. (ECH will eventually hide SNI → fall back to IP/DNS.)
88-
- **a3s-box:** a box is a separate guest kernel, so host-side eBPF sees box **egress** (it
89-
flows through the host net path) but not in-guest exec/file — those need an in-guest
90-
collector (phase 2).
74+
- **Zero-instrumentation, language-agnostic** — observe or guard any agent (Python/Node/Go/Rust)
75+
without touching its code, including its tool subprocesses.
76+
- **Kernel hooks only in the always-on core, no uprobes** — so the core gives **no LLM
77+
prompt/completion content**. That content is available via an **opt-in** OpenSSL uprobe
78+
extension (`A3S_OBSERVER_SSL=1`) — OpenSSL only (Python/Node/curl …, not Go `crypto/tls`),
79+
kept out of the universal core because a uprobe binds to a library symbol. (ECH will
80+
eventually hide SNI → fall back to IP/DNS.)
81+
- **a3s-box** — a box is a separate guest kernel, so host-side eBPF sees box **egress** (it
82+
flows through the host net path) but not in-guest exec/file — those need an in-guest collector
83+
(phase 2).
9184

9285
## Build & run
9386

94-
eBPF needs nightly + `rust-src` + [`bpf-linker`](https://github.com/aya-rs/bpf-linker)
95-
(it borrows rustc's bundled LLVM — no system LLVM required):
87+
eBPF needs nightly + `rust-src` + [`bpf-linker`](https://github.com/aya-rs/bpf-linker) (it
88+
borrows rustc's bundled LLVM — no system LLVM required):
9689

9790
```bash
9891
rustup toolchain install nightly --component rust-src
@@ -101,38 +94,47 @@ cargo build --release -p a3s-observer-collector # build.rs compiles + links t
10194

10295
sudo ./target/release/a3s-observer-collector # human-readable log
10396
A3S_OBSERVER_JSON=1 sudo -E ./target/release/a3s-observer-collector # NDJSON
104-
A3S_OBSERVER_FILES=1 A3S_OBSERVER_JSON=1 sudo -E ./target/release/a3s-observer-collector # + file writes
10597
```
10698

107-
Linux only; needs root (CAP_BPF + CAP_PERFMON).
99+
Linux only; needs root (CAP_BPF + CAP_PERFMON). Env knobs: `A3S_OBSERVER_JSON` (NDJSON),
100+
`A3S_OBSERVER_FILES` (file writes — high-volume), `A3S_OBSERVER_SSL` (OpenSSL content),
101+
`A3S_OBSERVER_HEARTBEAT` (liveness file path).
102+
103+
Opt-in enforcement — run against an agent's cgroup and/or a deny-list file:
104+
105+
```bash
106+
sudo ./target/release/a3s-observer-enforce /sys/fs/cgroup/<agent> egress-deny.txt
107+
sudo ./target/release/a3s-observer-fileguard file-exec-deny.txt
108+
```
108109

109110
## Deploy
110111

111-
a3s-observer captures and emits NDJSON; shipping it (batch / retry / route to a backend) is
112-
the OpenTelemetry Collector's job, so in-process OTLP is intentionally **not** built:
112+
a3s-observer emits NDJSON; shipping it (batch / retry / route to a backend) is the OpenTelemetry
113+
Collector's job, so in-process OTLP is intentionally **not** built:
113114

114115
```
115116
a3s-observer → NDJSON → OTel Collector (filelog → OTLP) → your backend
116117
```
117118

118-
- Collector config: [`deploy/otel-collector.yaml`](deploy/otel-collector.yaml). Every event
119-
is one valid-JSON line, so it also drops into vector / Loki / `jq`.
120-
- **Kubernetes:** CI publishes `ghcr.io/a3s-lab/observer:<tag>` on each tag
121-
([`image.yml`](.github/workflows/image.yml) from [`deploy/Dockerfile`](deploy/Dockerfile));
122-
deploy [`deploy/daemonset.yaml`](deploy/daemonset.yaml) (NDJSON to stdout, pod identity from
123-
`/proc/<pid>/cgroup` — no k8s API/RBAC). Mirror the image to a cluster-local registry for
124-
nodes that can't reach ghcr.io.
119+
- Collector config: [`deploy/otel-collector.yaml`](deploy/otel-collector.yaml) (`memory_limiter`
120+
+ a retrying sending queue). Every event is one valid-JSON line — also drops into vector /
121+
Loki / `jq`.
122+
- **Kubernetes:** CI publishes `ghcr.io/a3s-lab/observer:<tag>` (Trivy-scanned, cosign-signed,
123+
with SBOM + SLSA provenance); deploy [`deploy/daemonset.yaml`](deploy/daemonset.yaml) (NDJSON
124+
to stdout, pod identity from `/proc/<pid>/cgroup` — no k8s API/RBAC; liveness probe,
125+
`system-node-critical`). Mirror the image to a cluster-local registry for nodes that can't
126+
reach ghcr.io.
125127

126128
## Workspace
127129

128130
| crate | role |
129131
|---|---|
130132
| `a3s-observer` | contracts + data model (`IdentityResolver` / `ServiceClassifier` / `Exporter` / `Policy`) — host-buildable |
131133
| `a3s-observer-common` | `no_std` types shared with the eBPF probes |
132-
| `a3s-observer-ebpf` | the probes + egress guard, compiled to BPF bytecode |
134+
| `a3s-observer-ebpf` | probes + the `connect4` egress guard, compiled to BPF bytecode |
133135
| `a3s-observer-collector` | loader, correlation, export; plus the `enforce` and `fileguard` binaries |
134136

135-
Rust + [Aya](https://aya-rs.dev).
137+
Rust + [Aya](https://aya-rs.dev). Validated on Linux 6.8.
136138

137139
## Security
138140

0 commit comments

Comments
 (0)