Skip to content

Commit 55a84f3

Browse files
committed
feat(stack-coexist): route sendmsg/recvmsg/getpeername/getsockname/shutdown to kernel stack
Add 5 host bridges (ff_host_sendmsg/recvmsg/shutdown/getpeername/getsockname) and route them in ff_syscall_wrapper.c for managed kernel fds; ff_shutdown also dual-drives the dual-stack map fd like ff_close. All under FF_KERNEL_COEXIST. Verified: macro-on build rc=0 with symbols present, macro-off rc=0 size 6539682 byte-identical to baseline (zero regression); cmocka dual-mode (host 27/24, epoll 21, config 54/50) incl. new loopback bridge test; real-machine kernel-fd selftest PASS (getpeername/getsockname/sendmsg/recvmsg/shutdown end-to-end) and F-Stack fast path curl 200. Spec D8 converged (zh+en): these 5 now routed, readv/writev/ioctl remain known limitations.
1 parent 3e71f46 commit 55a84f3

9 files changed

Lines changed: 188 additions & 7 deletions

File tree

docs/kernel_event_support_spec/02-current-state-analysis.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
> - **hook cross-reference**: `fstack_kernel_fd_map[65536]` (`ff_hook_syscall.c:258`, bare array, lock-free, single-threaded poll); epoll dual-build merge (create `:1990-2000`, wait `:2324+`, close linkage `:1871-1884`); **socket/listen are NOT dual-built**.
1313
> - **native vs hook**: isomorphic (map / epoll merge / close linkage); divergent (socket/bind/listen/connect auto dual-build is v6-only).
1414
> - **v6 map gap (D9, grep verified)**: `grep ff_native_fd_map / ff_native_map_get/set/clear lib/` → 0 hits; `FF_MAX_FREEBSD_FILES` only in `ff_hook_syscall.c:257` and these docs. The v6 native map and default dual-build/dual-drive are NOT yet landed — to-be-implemented at R7. v6 adds (HOST_CFLAGS, `#ifdef FF_KERNEL_COEXIST`): `static int ff_native_fd_map[FF_MAX_FREEBSD_FILES]` + `ff_native_map_get/set/clear` in `ff_host_interface.{c,h}`, lock-free (modeled on hook).
15-
> - **D1-D8** code-doc inconsistencies fixed in §6 (D2 config parse line `:1027-1031` not `:956`; D3 no `default_stack`, `ff_api.h:91` comment stale; D4 header `unsigned int` vs impl `socklen_t`; D5 `ff_stack_get_stats` NOT implemented; D6 encode-offset + `ff_epoll_pairs`, not enum/ownership-table; D8 routing covers 13 entries, NOT `ff_readv/writev/send/recv/getpeername/getsockname/shutdown/ioctl/sendmsg/recvmsg`).
15+
> - **D1-D8** code-doc inconsistencies fixed in §6 (D2 config parse line `:1027-1031` not `:956`; D3 no `default_stack`, `ff_api.h:91` comment stale; D4 header `unsigned int` vs impl `socklen_t`; D5 `ff_stack_get_stats` NOT implemented; D6 encode-offset + `ff_epoll_pairs`, not enum/ownership-table; D8 R8 adds kernel routing for `sendmsg/recvmsg/getpeername/getsockname/shutdown` (first four single-stack hot-route, shutdown kernel-route + dual-stack dual-drive); `ff_readv/writev/ioctl` remain unrouted (known limitation)).
1616
1717
---
1818

docs/kernel_event_support_spec/zh_cn/02-current-state-analysis.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@
129129
| D5 | `ff_stack_stats`/`ff_stack_get_stats` **未实现**,标注待定 |
130130
| D6 | fd 区分=`FF_KERNEL_FD_BASE` 偏移 + `ff_epoll_pairs` 配对;v6 增 `ff_native_fd_map` 双栈映射(仍非 enum/归属表) |
131131
| D7 | 原生 v5 共存已落地(编译宏已包裹);v6 在其上改默认语义为双栈 |
132-
| D8 | 路由仅 13 入口;`readv/writev/getpeername/getsockname/shutdown/ioctl/sendmsg/recvmsg` 等未加路由(已知限制)v6 双栈 fd 对这些接口默认仅 F-Stack 驱动 |
132+
| D8 | R8 补齐 `sendmsg/recvmsg/getpeername/getsockname/shutdown` 内核路由(前 4 单栈热路由,shutdown 内核路由+双栈双驱动);剩 `readv/writev/ioctl` 仍未加路由(已知限制)v6 双栈 fd 对其默认仅 F-Stack 驱动 |
133133
| **D9(v6)** | `ff_native_fd_map`/默认双建/双驱动**尚未实现**(§5.2 grep=0)——文档区分「v5 已实测」与「v6 待实现」,不当既成 |
134134
135135
---

docs/kernel_event_support_spec/zh_cn/05-interface-design.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,8 @@ ff_connect(s, name, namelen) s 为双栈 fd (ff_native_map_get(s)>0)
190190
| `ff_epoll_wait maxevents<1` | `-EINVAL``ff_epoll.c:221`|
191191
| 内核/F-Stack 地址端口冲突 | 返回原生 errno(双栈某栈 bind 失败按部分失败契约处理) |
192192
| close 双栈 fd | 两栈 close + `ff_native_map_clear`;kqueue fd 清 `ff_epoll_pairs`(避免内核 fd 泄漏,FR-7) |
193-
| 双栈 fd 调用未路由接口 | `ff_readv/writev/getpeername/getsockname/shutdown/ioctl/sendmsg/recvmsg` 默认**仅 F-Stack 驱动**(D8 延伸已知限制) |
193+
| 内核 fd 调用 `sendmsg/recvmsg/getpeername/getsockname/shutdown` | R8 已加内核路由:前 4 内核 fd 走 `ff_host_*` 单栈;`shutdown` 内核 fd 走 host,双栈 map fd 两侧双驱动半关闭(仿 `ff_close`|
194+
| 双栈 fd 调用未路由接口 | `ff_readv/writev/ioctl` 默认**仅 F-Stack 驱动**(D8 延伸已知限制) |
194195

195196
---
196197

docs/kernel_event_support_spec/zh_cn/07-test-spec.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@
113113
4. PERF-1/2/4 无回归。
114114
5. config 关/`SOCK_FSTACK`/宏关时与纯 F-Stack 一致(NFR-1/NFR-3)。
115115
6. **connect 契约确认**`05 §6` 草案经用户确认后 UT-14/IT-9 方可定稿判 PASS。
116-
7. D8 路由覆盖限制 + 双栈 fd 未路由接口仅 F-Stack 驱动须文档/用例明示。
116+
7. R8 新增 `sendmsg/recvmsg/getpeername/getsockname/shutdown` 内核 fd 路由须单测覆盖;剩余 `readv/writev/ioctl` 仍为 D8 已知限制(双栈 fd F-Stack 驱动),须文档/用例明示。
117117
- 任一项失败 → bounce 打回上一里程碑(同步骤≤3 次,超限转人工)。
118118

119119
---

example/helloworld_stacksel/main.c

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
#include <errno.h>
3333
#include <sys/types.h>
3434
#include <sys/socket.h>
35+
#include <sys/uio.h>
3536
#include <sys/wait.h>
3637
#include <netinet/in.h>
3738
#include <arpa/inet.h>
@@ -82,8 +83,31 @@ run_client(int port)
8283
ff_close(c);
8384
return 1;
8485
}
85-
if (ff_send(c, "ping", 4, 0) != 4) {
86-
perror("ff_send");
86+
87+
/* R8: exercise getpeername/getsockname on the kernel fd. */
88+
struct sockaddr_in pn;
89+
socklen_t pnlen = sizeof(pn);
90+
if (ff_getpeername(c, (struct linux_sockaddr *)&pn, &pnlen) < 0 ||
91+
ntohs(pn.sin_port) != (unsigned short)port) {
92+
fprintf(stderr, "client: ff_getpeername mismatch\n");
93+
ff_close(c);
94+
return 1;
95+
}
96+
pnlen = sizeof(pn);
97+
if (ff_getsockname(c, (struct linux_sockaddr *)&pn, &pnlen) < 0) {
98+
perror("ff_getsockname");
99+
ff_close(c);
100+
return 1;
101+
}
102+
103+
/* R8: send via ff_sendmsg. */
104+
struct iovec io = { (void *)"ping", 4 };
105+
struct msghdr mh;
106+
memset(&mh, 0, sizeof(mh));
107+
mh.msg_iov = &io;
108+
mh.msg_iovlen = 1;
109+
if (ff_sendmsg(c, &mh, 0) != 4) {
110+
perror("ff_sendmsg");
87111
ff_close(c);
88112
return 1;
89113
}
@@ -105,9 +129,29 @@ run_server_once(int lfd)
105129
perror("ff_accept");
106130
return 1;
107131
}
108-
ssize_t n = ff_recv(c, buf, sizeof(buf), 0);
132+
133+
/* R8: getpeername/getsockname on the accepted kernel fd. */
134+
struct sockaddr_in pn;
135+
socklen_t pnlen = sizeof(pn);
136+
if (ff_getpeername(c, (struct linux_sockaddr *)&pn, &pnlen) < 0 ||
137+
ff_getsockname(c, (struct linux_sockaddr *)&pn, &pnlen) < 0) {
138+
perror("ff_get*name");
139+
ff_close(c);
140+
return 1;
141+
}
142+
143+
/* R8: receive via ff_recvmsg. */
144+
struct iovec io = { buf, sizeof(buf) };
145+
struct msghdr mh;
146+
memset(&mh, 0, sizeof(mh));
147+
mh.msg_iov = &io;
148+
mh.msg_iovlen = 1;
149+
ssize_t n = ff_recvmsg(c, &mh, 0);
109150
if (n == 4 && memcmp(buf, "ping", 4) == 0)
110151
ff_send(c, "pong", 4, 0);
152+
153+
/* R8: half-close the write side via ff_shutdown. */
154+
ff_shutdown(c, SHUT_WR);
111155
ff_close(c);
112156
return (n == 4) ? 0 : 1;
113157
}

lib/ff_host_interface.c

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -399,6 +399,36 @@ ff_host_epoll_wait(int epfd, void *events, int maxevents, int timeout)
399399
{
400400
return epoll_wait(epfd, (struct epoll_event *)events, maxevents, timeout);
401401
}
402+
403+
ssize_t
404+
ff_host_sendmsg(int fd, const void *msg, int flags)
405+
{
406+
return sendmsg(fd, (const struct msghdr *)msg, flags);
407+
}
408+
409+
ssize_t
410+
ff_host_recvmsg(int fd, void *msg, int flags)
411+
{
412+
return recvmsg(fd, (struct msghdr *)msg, flags);
413+
}
414+
415+
int
416+
ff_host_shutdown(int fd, int how)
417+
{
418+
return shutdown(fd, how);
419+
}
420+
421+
int
422+
ff_host_getpeername(int fd, void *addr, socklen_t *addrlen)
423+
{
424+
return getpeername(fd, (struct sockaddr *)addr, addrlen);
425+
}
426+
427+
int
428+
ff_host_getsockname(int fd, void *addr, socklen_t *addrlen)
429+
{
430+
return getsockname(fd, (struct sockaddr *)addr, addrlen);
431+
}
402432
#endif /* FF_KERNEL_COEXIST */
403433

404434
void ff_os_errno(int error)

lib/ff_host_interface.h

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,11 @@ int ff_host_fcntl(int fd, int cmd, int arg);
167167
int ff_host_epoll_create1(int flags);
168168
int ff_host_epoll_ctl(int epfd, int op, int fd, void *event);
169169
int ff_host_epoll_wait(int epfd, void *events, int maxevents, int timeout);
170+
ssize_t ff_host_sendmsg(int fd, const void *msg, int flags);
171+
ssize_t ff_host_recvmsg(int fd, void *msg, int flags);
172+
int ff_host_shutdown(int fd, int how);
173+
int ff_host_getpeername(int fd, void *addr, unsigned int *addrlen);
174+
int ff_host_getsockname(int fd, void *addr, unsigned int *addrlen);
170175
#endif /* FF_KERNEL_COEXIST */
171176

172177
#endif

lib/ff_syscall_wrapper.c

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1397,6 +1397,11 @@ ff_sendmsg(int s, const struct msghdr *msg, int flags)
13971397
struct msghdr freebsd_msg;
13981398
struct cmsghdr *freebsd_cmsg = NULL;
13991399

1400+
#ifdef FF_KERNEL_COEXIST
1401+
if (ff_is_kernel_fd(s))
1402+
return ff_host_sendmsg(ff_kernel_fd_real(s), msg, flags);
1403+
#endif /* FF_KERNEL_COEXIST */
1404+
14001405
freebsd_msg.msg_name = &freebsd_sa;
14011406
if ((__DECONST(struct linux_msghdr *, msg))->msg_control) {
14021407
freebsd_cmsg = malloc((__DECONST(struct linux_msghdr *, msg))->msg_controllen, NULL, 0);
@@ -1490,6 +1495,11 @@ ff_recvmsg(int s, struct msghdr *msg, int flags)
14901495
int rc, ret;
14911496
struct msghdr freebsd_msg;
14921497

1498+
#ifdef FF_KERNEL_COEXIST
1499+
if (ff_is_kernel_fd(s))
1500+
return ff_host_recvmsg(ff_kernel_fd_real(s), msg, flags);
1501+
#endif /* FF_KERNEL_COEXIST */
1502+
14931503
ret = linux2freebsd_msghdr((struct linux_msghdr *)msg, &freebsd_msg, 0);
14941504
if (ret < 0) {
14951505
rc = EINVAL;
@@ -1738,6 +1748,11 @@ ff_getpeername(int s, struct linux_sockaddr * name,
17381748
struct sockaddr_storage pfss;
17391749
struct sockaddr *pf = (struct sockaddr *)&pfss;
17401750

1751+
#ifdef FF_KERNEL_COEXIST
1752+
if (ff_is_kernel_fd(s))
1753+
return ff_host_getpeername(ff_kernel_fd_real(s), name, namelen);
1754+
#endif /* FF_KERNEL_COEXIST */
1755+
17411756
if ((rc = kern_getpeername(curthread, s, pf)))
17421757
goto kern_fail;
17431758

@@ -1761,6 +1776,11 @@ ff_getsockname(int s, struct linux_sockaddr *name,
17611776
struct sockaddr_storage pfss;
17621777
struct sockaddr *pf = (struct sockaddr *)&pfss;
17631778

1779+
#ifdef FF_KERNEL_COEXIST
1780+
if (ff_is_kernel_fd(s))
1781+
return ff_host_getsockname(ff_kernel_fd_real(s), name, namelen);
1782+
#endif /* FF_KERNEL_COEXIST */
1783+
17641784
if ((rc = kern_getsockname(curthread, s, pf)))
17651785
goto kern_fail;
17661786

@@ -1781,13 +1801,26 @@ ff_shutdown(int s, int how)
17811801
{
17821802
int rc;
17831803

1804+
#ifdef FF_KERNEL_COEXIST
1805+
if (ff_is_kernel_fd(s))
1806+
return ff_host_shutdown(ff_kernel_fd_real(s), how);
1807+
#endif /* FF_KERNEL_COEXIST */
1808+
17841809
struct shutdown_args sa = {
17851810
.s = s,
17861811
.how = how,
17871812
};
17881813
if ((rc = sys_shutdown(curthread, &sa)))
17891814
goto kern_fail;
17901815

1816+
#ifdef FF_KERNEL_COEXIST
1817+
if (ff_global_cfg.stack.kernel_coexist) {
1818+
int hfd = ff_native_map_get(s);
1819+
if (hfd > 0)
1820+
ff_host_shutdown(hfd, how);
1821+
}
1822+
#endif /* FF_KERNEL_COEXIST */
1823+
17911824
return (rc);
17921825
kern_fail:
17931826
ff_os_errno(rc);

tests/unit/test_ff_host_interface.c

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -613,6 +613,11 @@ test_ff_update_current_ts_assert_fail_on_clock_error(void **state)
613613
}
614614

615615
#ifdef FF_KERNEL_COEXIST
616+
#include <sys/socket.h>
617+
#include <sys/uio.h>
618+
#include <netinet/in.h>
619+
#include <arpa/inet.h>
620+
616621
/* Kernel-stack coexistence fd-space: a managed kernel fd is encoded as
617622
* (host_fd + FF_KERNEL_FD_BASE) and must never collide with FreeBSD fds
618623
* (< FF_KERNEL_FD_BASE). Verify the ff_is_kernel_fd/encode/real helpers. */
@@ -651,6 +656,68 @@ test_ff_native_fd_map(void **state)
651656
ff_native_map_set(-1, 5);
652657
ff_native_map_set(1 << 30, 5);
653658
}
659+
660+
/*
661+
* R8 host bridges: ff_host_sendmsg/recvmsg/getpeername/getsockname/shutdown.
662+
* Drive them over a real loopback AF_INET TCP pair (no mock) so the bridge's
663+
* direct host-libc passthrough is exercised end to end.
664+
*/
665+
static void
666+
test_ff_host_msg_name_shutdown_bridges(void **state)
667+
{
668+
(void)state;
669+
int ln = socket(AF_INET, SOCK_STREAM, 0);
670+
assert_true(ln >= 0);
671+
struct sockaddr_in sa;
672+
memset(&sa, 0, sizeof(sa));
673+
sa.sin_family = AF_INET;
674+
sa.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
675+
sa.sin_port = 0;
676+
assert_int_equal(bind(ln, (struct sockaddr *)&sa, sizeof(sa)), 0);
677+
assert_int_equal(listen(ln, 1), 0);
678+
679+
socklen_t slen = sizeof(sa);
680+
assert_int_equal(getsockname(ln, (struct sockaddr *)&sa, &slen), 0);
681+
682+
int cli = socket(AF_INET, SOCK_STREAM, 0);
683+
assert_true(cli >= 0);
684+
assert_int_equal(connect(cli, (struct sockaddr *)&sa, sizeof(sa)), 0);
685+
int srv = accept(ln, NULL, NULL);
686+
assert_true(srv >= 0);
687+
688+
struct sockaddr_in pn;
689+
socklen_t pnlen = sizeof(pn);
690+
assert_int_equal(ff_host_getsockname(cli, &pn, &pnlen), 0);
691+
assert_int_equal(pn.sin_addr.s_addr, htonl(INADDR_LOOPBACK));
692+
693+
pnlen = sizeof(pn);
694+
assert_int_equal(ff_host_getpeername(cli, &pn, &pnlen), 0);
695+
assert_int_equal(pn.sin_port, sa.sin_port);
696+
697+
char wbuf[] = "ping";
698+
struct iovec wio = { wbuf, sizeof(wbuf) };
699+
struct msghdr wmsg;
700+
memset(&wmsg, 0, sizeof(wmsg));
701+
wmsg.msg_iov = &wio;
702+
wmsg.msg_iovlen = 1;
703+
assert_int_equal(ff_host_sendmsg(cli, &wmsg, 0), (ssize_t)sizeof(wbuf));
704+
705+
char rbuf[16] = {0};
706+
struct iovec rio = { rbuf, sizeof(rbuf) };
707+
struct msghdr rmsg;
708+
memset(&rmsg, 0, sizeof(rmsg));
709+
rmsg.msg_iov = &rio;
710+
rmsg.msg_iovlen = 1;
711+
assert_int_equal(ff_host_recvmsg(srv, &rmsg, 0), (ssize_t)sizeof(wbuf));
712+
assert_string_equal(rbuf, "ping");
713+
714+
assert_int_equal(ff_host_shutdown(cli, SHUT_WR), 0);
715+
assert_int_equal(ff_host_recvmsg(srv, &rmsg, 0), 0);
716+
717+
ff_host_close(srv);
718+
ff_host_close(cli);
719+
ff_host_close(ln);
720+
}
654721
#endif /* FF_KERNEL_COEXIST */
655722

656723
int
@@ -688,6 +755,7 @@ main(void)
688755
#ifdef FF_KERNEL_COEXIST
689756
cmocka_unit_test(test_ff_kernel_fd_encode_roundtrip),
690757
cmocka_unit_test(test_ff_native_fd_map),
758+
cmocka_unit_test(test_ff_host_msg_name_shutdown_bridges),
691759
#endif /* FF_KERNEL_COEXIST */
692760
};
693761
return cmocka_run_group_tests(tests, NULL, NULL);

0 commit comments

Comments
 (0)