Skip to content

Commit 976e90b

Browse files
committed
[CWS] Add io_uring socket/splice/ftruncate hooks + tests
Adds a SKILL.md to automate io_uring coverage auditing, and closes the three gaps where CWS observed the syscall but emitted no event via io_uring by adding eBPF hooks for the opcodes (and tests): - IORING_OP_SOCKET: io_socket hooks; domain/type/protocol read via BTF-resolved LOAD_CONSTANT offsets on struct io_socket - IORING_OP_SPLICE: io_splice hooks, reusing the get_pipe_info path. We have to hook on io_issue_sqe instead because io_splice is inlined in some kernel versions. - IORING_OP_FTRUNCATE: io_ftruncate hooks, reusing do_truncate (observed as an open+O_TRUNC event; needs kernel 6.9+)
1 parent 1aa758e commit 976e90b

21 files changed

Lines changed: 519 additions & 8 deletions

File tree

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
name: cws-iouring-coverage
3+
description: Audit CWS (runtime-security) io_uring functional-test coverage and add a functional test for any io_uring opcode whose operation CWS observes but that is not exercised through io_uring. Test-driven — coverage is judged by tests, never by reading eBPF/hook internals. Use when auditing io_uring test coverage or after new IORING_OP_ opcodes appear.
4+
---
5+
6+
# CWS io_uring test coverage audit
7+
8+
For every io_uring operation CWS observes, there must be a functional test that issues the
9+
operation *through io_uring* and asserts the event fires. This skill finds the gaps and fills
10+
them.
11+
12+
## Principle: tests are the only arbiter
13+
14+
Coverage is decided by tests, nothing else. **Do not read eBPF programs or C hooks** (anything
15+
under `pkg/security/ebpf/`) to judge coverage — implementation is out of scope and misleads.
16+
The only question is: *does an io_uring test exist for this operation, and does it pass?*
17+
18+
The one io_uring-specific fact (it's a test assertion, not implementation): io_uring completes
19+
asynchronously, so **every io_uring test must assert `event.async == true`**.
20+
21+
## The audit: build three lists, then intersect
22+
23+
**List A — io_uring opcodes.** `WebFetch https://man7.org/linux/man-pages/man2/io_uring_enter.2.html`
24+
and list every `IORING_OP_*` with the syscall it performs (e.g. `IORING_OP_SOCKET``socket(2)`).
25+
Offline fallback: the iouring-go fork is `replace`d in `go.mod` — import path stays
26+
`github.com/iceber/iouring-go`, but source on disk is under
27+
`$(go env GOMODCACHE)/github.com/lebauce/iouring-go@*/syscall/types.go`.
28+
29+
**List B — tested syscalls.** Read `pkg/security/tests/`. A syscall is *tested* when a test
30+
issues it **as the triggering action** (inside the `func() error {…}` passed to
31+
`WaitSignal`/`runSyscallTester`) **and asserts an event** — not when it only appears as setup.
32+
Build the list from test bodies, not event names: the asserted event may be named differently.
33+
34+
**List C — tested io_uring opcodes.** `grep -rn 'iouring\.\|io_uring' pkg/security/tests/*.go`.
35+
Mostly `t.Run("io_uring", …)` subtests plus a few standalone funcs.
36+
37+
**Conclude, per opcode in A:**
38+
- syscall ∉ B → **out of scope** (CWS doesn't observe it; no io_uring test expected).
39+
- syscall ∈ B → **in scope**; then opcode ∈ C → **tested**, else → **gap**.
40+
41+
Write a test for every gap.
42+
43+
## Writing the io_uring test
44+
45+
Model on an existing subtest (`open_test.go``t.Run("io_uring", …)`). For each gap:
46+
47+
1. **Rule scope.** The test process is `testsuite`. Many rules are scoped to
48+
`process.file.name == "syscall_tester"`; reuse the parent rule only if it already admits
49+
`testsuite` (e.g. `in [ "syscall_tester", "testsuite" ]`), otherwise add a new rule scoped to
50+
`process.file.name == "testsuite"`. Check the parent's `ruleDefs` first.
51+
2. **Submit.** `iour, err := iouring.New(1)`; submit the op; read the result; in the validation
52+
callback assert event type, key fields, and `event.async == true`.
53+
3. **Kernel gate, not errno skip.** Gate "kernel too old for this opcode" deterministically at
54+
the top of the subtest:
55+
`checkKernelCompatibility(t, "io_uring <op> needs Linux X.Y", func(kv *kernel.Version) bool { return kv.Code < kernel.VersionCode(X, Y, 0) })`.
56+
Don't skip on a negative errno — a malformed raw SQE returns one too, so skipping would hide
57+
the gap behind a green test. On a supported kernel, treat an unexpected negative result as a
58+
**failure** (`return fmt.Errorf(...)`). (A library prep helper can't be malformed, so an
59+
errno skip there is harmless.)
60+
4. **ebpfless.** Only matters if the parent test is in the `available` list (`~`-prefixed
61+
entries) in `pkg/security/tests/main_linux.go` — those prefix-match subtests and pull yours
62+
into the ebpfless run, where io_uring is unsupported. If so, add an `exclude` entry. The
63+
match is exact on the full `t.Name()`, so prefer a flat sibling name
64+
(`TestOpen/io_uring_ftruncate`, not a nested `TestOpen/io_uring/ftruncate`).
65+
66+
**No prep helper in the fork?** The fork wraps only a subset. For other opcodes build a raw SQE
67+
with a custom `iouring.PrepRequest` using the helpers in `pkg/security/tests/iouring_test.go`
68+
(extend them as needed). This is the one place you may touch an opcode's low-level shape.
69+
70+
## Verify
71+
72+
- `gofmt -l <your files>` (no output = OK).
73+
- Run the test via the harness:
74+
`dda inv security-agent.functional-tests --skip-linters --testflags="-test.run <YourTest>"`.
75+
Green = covered, red = the gap is real.
76+
77+
## Report
78+
79+
Report the per-opcode classification (out of scope / tested / gap) and the test files
80+
added or changed.

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
/.claude/skills/create-runtime-setting/ @DataDog/agent-runtimes @DataDog/agent-configuration
3434
/.claude/skills/create-status-provider/ @DataDog/agent-configuration
3535
/.claude/skills/create-subcommand/ @DataDog/agent-configuration
36+
/.claude/skills/cws-iouring-coverage/ @DataDog/agent-security
3637
/.claude/skills/explain-lading-config @DataDog/single-machine-performance
3738
/.claude/skills/quality-gate-size-analysis/ @DataDog/agent-build
3839
/.claude/skills/review-pr-comments/ @DataDog/agent-devx

pkg/security/ebpf/c/include/constants/offsets/filesystem.h

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -580,6 +580,12 @@ static __attribute__((always_inline)) u64 get_iokiocb_ctx_offset() {
580580
return iokiocb_ctx_offset;
581581
}
582582

583+
static __attribute__((always_inline)) u64 get_iokiocb_opcode_offset() {
584+
u64 iokiocb_opcode_offset;
585+
LOAD_CONSTANT("iokiocb_opcode_offset", iokiocb_opcode_offset);
586+
return iokiocb_opcode_offset;
587+
}
588+
583589
static __attribute__((always_inline)) u64 get_getattr2() {
584590
u64 getattr2;
585591
LOAD_CONSTANT("getattr2", getattr2);

pkg/security/ebpf/c/include/constants/offsets/network.h

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,24 @@ __attribute__((always_inline)) struct sock* get_sock_from_socket(struct socket *
4141
return sk;
4242
}
4343

44+
__attribute__((always_inline)) u64 get_io_socket_domain_offset() {
45+
u64 offset;
46+
LOAD_CONSTANT("io_socket_domain_offset", offset);
47+
return offset;
48+
}
49+
50+
__attribute__((always_inline)) u64 get_io_socket_type_offset() {
51+
u64 offset;
52+
LOAD_CONSTANT("io_socket_type_offset", offset);
53+
return offset;
54+
}
55+
56+
__attribute__((always_inline)) u64 get_io_socket_protocol_offset() {
57+
u64 offset;
58+
LOAD_CONSTANT("io_socket_protocol_offset", offset);
59+
return offset;
60+
}
61+
4462
__attribute__((always_inline)) u64 get_flowi4_saddr_offset() {
4563
u64 flowi4_saddr_offset;
4664
LOAD_CONSTANT("flowi4_saddr_offset", flowi4_saddr_offset);

pkg/security/ebpf/c/include/hooks/network/socket.h

Lines changed: 36 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
#ifndef _HOOKS_SOCKET_H_
22
#define _HOOKS_SOCKET_H_
33

4-
static long __attribute__((always_inline)) trace__sys_socket(void *ctx, u8 async, u16 domain, u16 type, u16 protocol) {
4+
#include "constants/offsets/network.h"
5+
#include "helpers/iouring.h"
6+
7+
static long __attribute__((always_inline)) trace__sys_socket(void *ctx, u16 domain, u16 type, u16 protocol, u64 pid_tgid) {
58
if (is_discarded_by_pid()) {
69
return 0;
710
}
@@ -10,11 +13,12 @@ static long __attribute__((always_inline)) trace__sys_socket(void *ctx, u8 async
1013
struct syscall_cache_t syscall = {
1114
.type = EVENT_SOCKET,
1215
.policy = policy,
13-
.async = async,
16+
.async = pid_tgid ? ASYNC_SYSCALL : SYNC_SYSCALL,
1417
.socket = {
1518
.domain = domain,
1619
.type = type,
1720
.protocol = protocol,
21+
.pid_tgid = pid_tgid,
1822
}
1923
};
2024

@@ -39,12 +43,18 @@ static int __attribute__((always_inline)) sys_socket_ret(void *ctx, int retval)
3943

4044
struct socket_event_t event = {
4145
.syscall.retval = retval,
46+
.event.flags = syscall->async ? EVENT_FLAGS_ASYNC : 0,
4247
.domain = syscall->socket.domain,
4348
.type = syscall->socket.type,
4449
.protocol = syscall->socket.protocol,
4550
};
4651

47-
struct proc_cache_t *entry = fill_process_context(&event.process);
52+
struct proc_cache_t *entry;
53+
if (syscall->socket.pid_tgid != 0) {
54+
entry = fill_process_context_with_pid_tgid(&event.process, syscall->socket.pid_tgid);
55+
} else {
56+
entry = fill_process_context(&event.process);
57+
}
4858
fill_cgroup_context(entry, &event.cgroup);
4959
fill_span_context(&event.span);
5060

@@ -56,14 +66,35 @@ HOOK_SYSCALL_ENTRY3(socket, int, domain, int, type, int, protocol) {
5666
// Mask out SOCK_CLOEXEC / SOCK_NONBLOCK flags to get just the socket type
5767
// (SOCK_STREAM=1, SOCK_DGRAM=2, ... fit in 4 bits)
5868
u16 socket_type = (u16)type & 0x0F;
59-
return trace__sys_socket(ctx, SYNC_SYSCALL, (u16)domain, socket_type, (u16)protocol);
69+
return trace__sys_socket(ctx, (u16)domain, socket_type, (u16)protocol, 0);
6070
}
6171

6272
HOOK_SYSCALL_EXIT(socket) {
6373
int retval = SYSCALL_PARMRET(ctx);
6474
return sys_socket_ret(ctx, retval);
6575
}
6676

77+
// io_socket (IORING_OP_SOCKET, kernel 5.19+).
78+
HOOK_ENTRY("io_socket")
79+
int hook_io_socket(ctx_t *ctx) {
80+
void *raw_req = (void *)CTX_PARM1(ctx);
81+
82+
int domain = 0, type = 0, protocol = 0;
83+
bpf_probe_read(&domain, sizeof(domain), raw_req + get_io_socket_domain_offset());
84+
bpf_probe_read(&type, sizeof(type), raw_req + get_io_socket_type_offset());
85+
bpf_probe_read(&protocol, sizeof(protocol), raw_req + get_io_socket_protocol_offset());
86+
87+
u64 pid_tgid = get_pid_tgid_from_iouring(raw_req);
88+
// Mask out SOCK_CLOEXEC / SOCK_NONBLOCK (same as above)
89+
u16 socket_type = (u16)type & 0x0F;
90+
return trace__sys_socket(ctx, (u16)domain, socket_type, (u16)protocol, pid_tgid);
91+
}
92+
93+
HOOK_EXIT("io_socket")
94+
int rethook_io_socket(ctx_t *ctx) {
95+
return sys_socket_ret(ctx, (int)CTX_PARMRET(ctx));
96+
}
97+
6798
TAIL_CALL_TRACEPOINT_FNC(handle_sys_socket_exit, struct tracepoint_raw_syscalls_sys_exit_t *args) {
6899
return sys_socket_ret(args, args->ret);
69100
}
@@ -98,4 +129,4 @@ static int hook_sock_release(struct bpf_sock *ctx)
98129
return 1;
99130
}
100131

101-
#endif
132+
#endif

pkg/security/ebpf/c/include/hooks/open.h

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,17 @@ int hook_io_openat2(ctx_t *ctx) {
221221
return trace_io_openat(ctx);
222222
}
223223

224+
// io_ftruncate (IORING_OP_FTRUNCATE, kernel 6.9+) calls do_ftruncate -> do_truncate, which
225+
// is already hooked (hook_do_truncate). As with the ftruncate syscall, we model it as an
226+
// open with O_TRUNC and let the do_truncate hook resolve the path from the file.
227+
HOOK_ENTRY("io_ftruncate")
228+
int hook_io_ftruncate(ctx_t *ctx) {
229+
void *raw_req = (void *)CTX_PARM1(ctx);
230+
u64 pid_tgid = get_pid_tgid_from_iouring(raw_req);
231+
int flags = O_CREAT | O_WRONLY | O_TRUNC;
232+
return trace__sys_openat2(ctx, NULL, flags, 0, pid_tgid);
233+
}
234+
224235
// used by both tail call callback and directly for tracepoints
225236
int __attribute__((always_inline)) _sys_open_ret(void *ctx, struct syscall_cache_t *syscall) {
226237
if (IS_UNHANDLED_ERROR(syscall->retval)) {
@@ -307,6 +318,16 @@ HOOK_SYSCALL_COMPAT_EXIT(ftruncate) {
307318
return sys_open_ret(ctx);
308319
}
309320

321+
HOOK_EXIT("io_ftruncate")
322+
int rethook_io_ftruncate(ctx_t *ctx) {
323+
struct syscall_cache_t *syscall = pop_syscall(EVENT_OPEN);
324+
if (!syscall || !syscall->open.dentry) {
325+
return 0;
326+
}
327+
syscall->retval = CTX_PARMRET(ctx);
328+
return _sys_open_ret(ctx, syscall);
329+
}
330+
310331
HOOK_SYSCALL_COMPAT_EXIT(open) {
311332
return sys_open_ret(ctx);
312333
}

pkg/security/ebpf/c/include/hooks/splice.h

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,17 @@
11
#ifndef _HOOKS_SPLICE_H_
22
#define _HOOKS_SPLICE_H_
33

4+
#include <uapi/linux/io_uring.h>
5+
46
#include "constants/offsets/splice.h"
57
#include "constants/syscall_macro.h"
68
#include "helpers/approvers.h"
79
#include "helpers/discarders.h"
810
#include "helpers/filesystem.h"
11+
#include "helpers/iouring.h"
912
#include "helpers/syscalls.h"
1013

11-
HOOK_SYSCALL_ENTRY0(splice) {
14+
int __attribute__((always_inline)) sys_splice(void *ctx, u64 pid_tgid) {
1215
if (is_discarded_by_pid()) {
1316
return 0;
1417
}
@@ -17,12 +20,33 @@ HOOK_SYSCALL_ENTRY0(splice) {
1720
struct syscall_cache_t syscall = {
1821
.type = EVENT_SPLICE,
1922
.policy = policy,
23+
.async = pid_tgid ? ASYNC_SYSCALL : SYNC_SYSCALL,
24+
.splice = {
25+
.pid_tgid = pid_tgid,
26+
},
2027
};
2128

2229
cache_syscall_update_cgroup(ctx, &syscall);
2330
return 0;
2431
}
2532

33+
HOOK_SYSCALL_ENTRY0(splice) {
34+
return sys_splice(ctx, 0);
35+
}
36+
37+
// io_splice is a static function that the compiler may inline into its sole caller
38+
// (io_issue_sqe). Hook io_issue_sqe instead and filter by opcode == IORING_OP_SPLICE.
39+
HOOK_ENTRY("io_issue_sqe")
40+
int hook_io_issue_sqe(ctx_t *ctx) {
41+
void *raw_req = (void *)CTX_PARM1(ctx);
42+
u8 opcode;
43+
int ret = bpf_probe_read(&opcode, sizeof(opcode), raw_req + get_iokiocb_opcode_offset());
44+
if (ret < 0 || opcode != IORING_OP_SPLICE) {
45+
return 0;
46+
}
47+
return sys_splice(ctx, get_pid_tgid_from_iouring(raw_req));
48+
}
49+
2650
HOOK_ENTRY("get_pipe_info")
2751
int hook_get_pipe_info(ctx_t *ctx) {
2852
struct syscall_cache_t *syscall = peek_syscall(EVENT_SPLICE);
@@ -98,13 +122,19 @@ int __attribute__((always_inline)) sys_splice_ret(void *ctx, int retval) {
98122

99123
struct splice_event_t event = {
100124
.syscall.retval = retval,
125+
.event.flags = syscall->async ? EVENT_FLAGS_ASYNC : 0,
101126
.file = syscall->splice.file,
102127
.pipe_entry_flag = syscall->splice.pipe_entry_flag,
103128
.pipe_exit_flag = syscall->splice.pipe_exit_flag,
104129
};
105130
fill_file(syscall->splice.dentry, &event.file);
106131

107-
struct proc_cache_t *entry = fill_process_context(&event.process);
132+
struct proc_cache_t *entry;
133+
if (syscall->splice.pid_tgid != 0) {
134+
entry = fill_process_context_with_pid_tgid(&event.process, syscall->splice.pid_tgid);
135+
} else {
136+
entry = fill_process_context(&event.process);
137+
}
108138
fill_cgroup_context(entry, &event.cgroup);
109139
fill_span_context(&event.span);
110140

@@ -117,6 +147,12 @@ HOOK_SYSCALL_EXIT(splice) {
117147
return sys_splice_ret(ctx, (int)SYSCALL_PARMRET(ctx));
118148
}
119149

150+
HOOK_EXIT("io_issue_sqe")
151+
int rethook_io_issue_sqe(ctx_t *ctx) {
152+
// pop_syscall inside sys_splice_ret returns NULL if no splice entry was cached
153+
return sys_splice_ret(ctx, (int)CTX_PARMRET(ctx));
154+
}
155+
120156
TAIL_CALL_TRACEPOINT_FNC(handle_sys_splice_exit, struct tracepoint_raw_syscalls_sys_exit_t *args) {
121157
return sys_splice_ret(args, args->ret);
122158
}

pkg/security/ebpf/c/include/structs/syscalls.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,7 @@ struct syscall_cache_t {
224224
u32 file_found;
225225
u32 pipe_entry_flag;
226226
u32 pipe_exit_flag;
227+
u64 pid_tgid;
227228
} splice;
228229

229230
struct {
@@ -252,6 +253,7 @@ struct syscall_cache_t {
252253
u16 domain;
253254
u16 type;
254255
u16 protocol;
256+
u64 pid_tgid;
255257
} socket;
256258

257259
struct {

pkg/security/ebpf/kernel/kernel.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,8 @@ var (
105105
Kernel6_7 = kernel.VersionCode(6, 7, 0)
106106
// Kernel6_8 is the KernelVersion representation of kernel version 6.8
107107
Kernel6_8 = kernel.VersionCode(6, 8, 0)
108+
// Kernel6_9 is the KernelVersion representation of kernel version 6.9
109+
Kernel6_9 = kernel.VersionCode(6, 9, 0)
108110
// Kernel6_10 is the KernelVersion representation of kernel version 6.10
109111
Kernel6_10 = kernel.VersionCode(6, 10, 0)
110112
// Kernel6_11 is the KernelVersion representation of kernel version 6.11

pkg/security/ebpf/probes/event_types.go

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,8 @@ func GetSelectorsPerEventType(hasFentry bool, hasCgroupSocket bool) map[eval.Eve
232232
hookFunc("hook_io_openat"),
233233
hookFunc("hook_io_openat2"),
234234
hookFunc("rethook_io_openat2"),
235+
hookFunc("hook_io_ftruncate"),
236+
hookFunc("rethook_io_ftruncate"),
235237
}},
236238
&manager.OneOf{Selectors: []manager.ProbesSelector{
237239
hookFunc("hook_terminate_walk"),
@@ -535,6 +537,10 @@ func GetSelectorsPerEventType(hasFentry bool, hasCgroupSocket bool) map[eval.Eve
535537
&manager.AllOf{Selectors: []manager.ProbesSelector{
536538
hookFunc("hook_get_pipe_info"),
537539
hookFunc("rethook_get_pipe_info"),
540+
}},
541+
&manager.BestEffort{Selectors: []manager.ProbesSelector{
542+
hookFunc("hook_io_issue_sqe"),
543+
hookFunc("rethook_io_issue_sqe"),
538544
}}},
539545

540546
// List of probes required to capture accept events
@@ -568,6 +574,10 @@ func GetSelectorsPerEventType(hasFentry bool, hasCgroupSocket bool) map[eval.Eve
568574
// List of probes required to capture socket events
569575
"socket": {
570576
&manager.BestEffort{Selectors: ExpandSyscallProbesSelector(SecurityAgentUID, "socket", hasFentry, EntryAndExit)},
577+
&manager.BestEffort{Selectors: []manager.ProbesSelector{
578+
hookFunc("hook_io_socket"),
579+
hookFunc("rethook_io_socket"),
580+
}},
571581
},
572582

573583
// List of probes required to capture chdir events

0 commit comments

Comments
 (0)