|
3 | 3 | > **Document ID**: SPEC-KE-09 (implementation-phase plan) |
4 | 4 | > **Version**: v6 (native automatic dual-stack coexistence paradigm) |
5 | 5 | > **Date**: 2026-06-17 |
6 | | -> **Status**: In progress (R0-R6 done; **R7 v6 auto dual-stack to-be-implemented**) |
7 | | -> **Basis**: the v6 spec in this directory (00-08); line numbers are subject to the actual code, gatekeeper re-verified. v6 changes are to-be-implemented design. |
| 6 | +> **Status**: R0-R7 all done and gate PASS (**R7 v6 auto dual-stack measured, commit 13b418191**; real-machine single listen(80) dual-stack 9.134.214.176 + 127.0.0.1 each HTTP 200) |
| 7 | +> **Basis**: the v6 spec in this directory (00-08); line numbers are subject to the actual code, gatekeeper re-verified. v6 changes are landed; §3 below is implemented code. |
8 | 8 | > **Authoritative full text**: `zh_cn/09-impl-plan.md`. |
9 | 9 |
|
10 | | -> **v6 sync (key points; see `zh_cn/09-impl-plan.md` for full detail)**: **R7 per-file rework** (under `#ifdef FF_KERNEL_COEXIST`, runtime `kernel_coexist=0` short-circuit, `SOCK_FSTACK`/macro-off byte-for-byte zero regression): |
11 | | -> - (1) `ff_host_interface.h`: declare `ff_native_map_get/set/clear` (NO struct-header change). |
12 | | -> - (2) `ff_host_interface.c`: `static int ff_native_fd_map[FF_MAX_FREEBSD_FILES]`(=65536, lock-free) + accessors (bounds-checked). |
13 | | -> - (3) `ff_syscall_wrapper.c`: `ff_socket:915-947` default dual-build (`sys_socket`(s)+`ff_host_socket`(h)+`ff_native_map_set(s,h)`, return s; markers single-stack); `ff_bind:1607-1627` (`kern_bindat` then `ff_host_bind(map[s], raw linux addr)`); `ff_listen:1584-1605` (`sys_listen` then `ff_host_listen(map[s])`); `ff_close:1095-1112` (`kern_close` then `ff_host_close(map[fd])`+`ff_native_map_clear`); `ff_accept/accept4:1514-1582` single-stack ownership; `ff_setsockopt:999`/`ff_fcntl:1495` sync both; `ff_connect:1629-1649` §connect draft (pending user confirmation). recv/send/read/write/recvfrom/sendto unchanged (single-stack by `ff_is_kernel_fd`, NO map lookup). |
14 | | -> - (4) `ff_epoll.c`: `ff_epoll_ctl:99-115` dual-register a dual-stack listen (kqueue + `ff_host_epoll_ctl(host_ep, op, map[fd], event)`, pass `event.data`); `ff_epoll_wait:214-252` merge (existing skeleton); `ff_close` clear `ff_epoll_pairs` for kqueue fd. |
15 | | -> - (5) `lib/Makefile`: already in place (`:174-177` dual CFLAGS); no change. |
16 | | -> - (6) demo: default dual-stack `listen(80)` demo for real-machine IT-1/2/3. |
17 | | -> Then R7 tests (cmocka dual-mode + real-machine dual-stack + perf) and gate (`08 §4` V1-V12; MT-1/3 incl. `ff_native_fd_map`); English spec sync; English short commit; config local values NOT committed; clean full rebuild after any header change (ABI skew). |
18 | | -
|
19 | 10 | --- |
20 | 11 |
|
21 | 12 | ## 0. Scope and Gate |
22 | 13 |
|
23 | | -- **Do all of R0-R5**: revert wrong code → spec rewrite → hook coexistence solidification + demo → native unified-event coexistence → tests/performance → gate + commit. |
24 | | -- **Hard gate (unconditionally all green)**: compilation passes + cmocka unit tests all green + coverage met + **F-Stack business fast-path zero regression (NFR-1/NFR-2)**. |
25 | | -- **Coexistence iron rule (NFR-3)**: at every phase the F-Stack user-space stack always carries the business and is never bypassed by the kernel stack; violating it bounces. |
26 | | -- **Target gate**: best-effort to actually set up the DPDK runtime and run the same-process dual-stack integration; when a real NIC is unavailable, the business plane is skipped with measured evidence and the kernel side is still measured on loopback. |
| 14 | +- **R0-R6 done**: revert → spec → hook solidification → native per-fd coexistence → tests/perf → gate/commit → compile-macro gating (ba148589d). |
| 15 | +- **R7 done (commit 13b418191)**: native automatic dual-stack (default dual-build/dual-drive + `ff_native_fd_map` + dual-stack events + accept ownership + dual-fire connect) landed and gate PASS. |
| 16 | +- **Hard gate (met)**: dual-mode compilation passes + cmocka dual-mode all green + **macro-off `nm` coexist symbols=0 (size 6539682, byte-for-byte identical to baseline, zero regression)** + **F-Stack business fast path + single-stack connection hot path zero regression (NFR-1/NFR-2)**. |
| 17 | +- **Coexistence iron rule (NFR-3)**: the F-Stack fd is always built and always carries the business; satisfied. |
| 18 | +- **connect contract**: implemented per the user-confirmed Q2=B (dual-fire connect, F-Stack as the return/data primary path) in `05 §6`. |
27 | 19 |
|
28 | 20 | --- |
29 | 21 |
|
30 | | -## 1. Agent Team Topology (harness + spec-driven) |
| 22 | +## 1. Agent Team Topology |
31 | 23 |
|
32 | 24 | | Agent | Role | Responsibility | |
33 | 25 | |---|---|---| |
34 | | -| **Leader** | orchestration+authoring+arbitration | orchestration, coding, gate arbitration, commit, bounce counting | |
35 | | -| **arch-probe** | architecture probe (read-only) | measure hook FF_KERNEL_EVENT / nginx / native event layer | |
36 | | -| **spec-writer** | spec rewrite | v4 Chinese & English docs | |
37 | | -| **build** | compilation (actual build) | lib / libff_syscall.so / tests | |
38 | | -| **unit-test** | unit tests | cmocka coexistence cases | |
39 | | -| **review** | review (read-only) | minimal diff/zero regression/coexistence iron rule/conventions | |
40 | | -| **test** | integration/performance | same-process dual-stack end-to-end + no regression on the F-Stack fast path | |
41 | | -| **gatekeeper** | gate (read-only) | per-assertion + gate-item verification | |
| 26 | +| Leader | orchestration+authoring+arbitration | orchestration, coding, gate arbitration, commit, bounce counting | |
| 27 | +| arch-probe | architecture probe (read-only) | measure v5 current state + hook isomorphism/divergence | |
| 28 | +| spec-writer | spec upgrade | v6 Chinese & English docs (this round's output) | |
| 29 | +| build | compilation (actual build) | lib dual-build / tests | |
| 30 | +| unit-test | unit tests | cmocka v6 dual-mode cases | |
| 31 | +| review | review (read-only) | minimal diff/zero regression/coexistence iron rule/conventions | |
| 32 | +| test | integration/performance | one-listen-many-uses real-machine dual-stack + fast-path no regression | |
| 33 | +| gatekeeper | gate (read-only) | per-assertion + gate items | |
42 | 34 |
|
43 | 35 | **Gate rollback**: any phase failing bounces back to the previous step; ≤3 bounces for the same step, escalate to manual when exceeded; bounces recorded in `08`. |
44 | 36 |
|
45 | 37 | --- |
46 | 38 |
|
47 | | -## 2. Change Points (measured anchors, code is authoritative) |
| 39 | +## 2. R0-R6 Change Points (done, measured anchors) |
48 | 40 |
|
49 | 41 | | Milestone | File | Change | |
50 | 42 | |---|---|---| |
51 | | -| R0 | `lib/ff_syscall_wrapper.c`, `lib/ff_host_interface.{c,h}` | revert the ff_host_socket side path (**done 0748eff94**) | |
52 | | -| R2 | `adapter/syscall/` (Makefile FF_KERNEL_EVENT), new demo | build libff_syscall.so; a correct same-process dual-stack demo (modeled after `main_stack_epoll_kernel.c`), replacing the v3 pure-kernel helloworld_stacksel | |
53 | | -| R3 | `lib/ff_config.{c,h}` | change the v3 `stack.default_to_kernel`/`default_stack` to `stack.kernel_coexist` (`MATCH("stack","kernel_coexist")`, default 0) + accessor | |
54 | | -| R3 | `lib/ff_host_interface.{c,h}` | add the managed kernel-side bridge (host socket/bind/listen/accept/connect/close/epoll_*) for lib to create managed kernel fds | |
55 | | -| R3 | `lib/ff_syscall_wrapper.c` `ff_socket` and ff_bind/listen/accept/connect/close | when coexistence enabled, SOCK_KERNEL→managed kernel fd+register ownership; route by ownership; default/`SOCK_FSTACK` zero regression | |
56 | | -| R3 | `lib/ff_epoll.c` | `ff_epoll_create` also creates a kernel epoll; `ff_epoll_ctl` splits; `ff_epoll_wait` merges kqueue⊕epoll; close linkage | |
57 | | -| R3 | `config.ini` | `[stack] kernel_coexist=0` example section (replacing the v3 default_stack) | |
58 | | -| R4 | `tests/unit/`, `tests/integration/` | cmocka coexistence cases + same-process dual-stack integration | |
59 | | -| docs | `docs/kernel_event_support_spec/` (Chinese & English 00-10) | v4 coexistence paradigm | |
| 43 | +| R0 | `ff_syscall_wrapper.c`, `ff_host_interface.{c,h}` | revert the ff_host_socket side path (0748eff94) | |
| 44 | +| R2 | `adapter/syscall/`, demo | hook solidification + same-process dual-stack demo | |
| 45 | +| R3 | `ff_config.{c,h}` | `stack.kernel_coexist` (`:1027-1031`/`:1363`/`:321-323`) | |
| 46 | +| R3 | `ff_host_interface.{c,h}` | 18 `ff_host_*` bridges + `FF_KERNEL_FD_BASE` | |
| 47 | +| R3 | `ff_syscall_wrapper.c` | `ff_socket` per-fd either/or + 13-entry `ff_is_kernel_fd` routing | |
| 48 | +| R3 | `ff_epoll.c` | `ff_epoll_pairs` merge | |
| 49 | +| R6 | `lib/Makefile` + 7 files | `FF_KERNEL_COEXIST` gating (`:174-177` dual CFLAGS) | |
| 50 | + |
| 51 | +--- |
| 52 | + |
| 53 | +## 3. R7 native automatic dual-stack per-file rework (v6 core, implemented commit 13b418191) |
| 54 | + |
| 55 | +> All new code goes inside `#ifdef FF_KERNEL_COEXIST`; runtime `kernel_coexist=0` short-circuits; `SOCK_FSTACK`/macro-off is byte-for-byte zero regression. |
| 56 | +
|
| 57 | +### 3.1 `lib/ff_host_interface.h` (HOST_CFLAGS header) |
| 58 | +- Inside the `#ifdef FF_KERNEL_COEXIST` block (currently `:94-160`) add accessor declarations: |
| 59 | + ```c |
| 60 | + int ff_native_map_get(int fstack_fd); |
| 61 | + void ff_native_map_set(int fstack_fd, int host_fd); |
| 62 | + void ff_native_map_clear(int fstack_fd); |
| 63 | + ``` |
| 64 | +- **Do not touch the struct header** (avoid ABI skew, `10 §7`). |
| 65 | +
|
| 66 | +### 3.2 `lib/ff_host_interface.c` (HOST_CFLAGS) |
| 67 | +- In the existing 18-bridge block (inside `#ifdef FF_KERNEL_COEXIST`) add: |
| 68 | + ```c |
| 69 | + static int ff_native_fd_map[FF_MAX_FREEBSD_FILES]; /* =65536, mimics ff_hook_syscall.c:258, lock-free */ |
| 70 | + int ff_native_map_get(int fd){ return (fd>=0 && fd<FF_MAX_FREEBSD_FILES) ? ff_native_fd_map[fd] : 0; } |
| 71 | + void ff_native_map_set(int fd,int h){ if(fd>=0 && fd<FF_MAX_FREEBSD_FILES) ff_native_fd_map[fd]=h; } |
| 72 | + void ff_native_map_clear(int fd){ if(fd>=0 && fd<FF_MAX_FREEBSD_FILES) ff_native_fd_map[fd]=0; } |
| 73 | + ``` |
| 74 | +- `#define FF_MAX_FREEBSD_FILES` here if lib does not already define it (mimics adapter). |
| 75 | + |
| 76 | +### 3.3 `lib/ff_syscall_wrapper.c` (CFLAGS) |
| 77 | +- `ff_socket`(`:915-947`): refactor inside the `#ifdef FF_KERNEL_COEXIST` block — |
| 78 | + - `SOCK_KERNEL && !SOCK_FSTACK && coexist`: keep v5 (kernel only + encode). |
| 79 | + - **new default dual-stack**: `!SOCK_FSTACK && coexist` (no marker) → first `sys_socket` builds F-Stack fd `s`, then `ff_host_socket(...)` builds host `h`, `ff_native_map_set(s,h)`, return `s`; `ff_host_socket` failure per the `05 §7` contract. |
| 80 | + - `SOCK_FSTACK` / coexist-off: original `sys_socket` path (`:937-943` unchanged). |
| 81 | +- `ff_bind`(`:1607-1627`): keep the existing `#ifdef` `ff_is_kernel_fd` block; after `kern_bindat` succeeds (`:1620-1623`) add a `#ifdef` block: `int h=ff_native_map_get(s); if(h>0) ff_host_bind(h, addr, addrlen);` (use the **raw linux addr**, not bsdaddr). |
| 82 | +- `ff_listen`(`:1584-1605`): after `sys_listen` succeeds add `int h=ff_native_map_get(s); if(h>0) ff_host_listen(h, backlog);`. |
| 83 | +- `ff_close`(`:1095-1112`): after `kern_close` succeeds add `int h=ff_native_map_get(fd); if(h>0){ ff_host_close(h); ff_native_map_clear(fd); }`; and clear `ff_epoll_pairs` for a kqueue fd (cooperating with `ff_epoll.c`, see 3.4). |
| 84 | +- `ff_accept`/`ff_accept4`(`:1514-1582`): for a dual-stack listen fd (`ff_native_map_get(s)>0`) follow `05 §5` single-stack ownership (kern_accept→EAGAIN→ff_host_accept+encode). |
| 85 | +- `ff_setsockopt`(`:999`)/`ff_fcntl`(`:1495`): for a dual-stack fd, after the F-Stack path sync `ff_host_setsockopt/fcntl` to `map[s]`. |
| 86 | +- `ff_connect`(`:1629-1649`): implemented per Q2=B — `kern_connectat` primary, best-effort `ff_host_connect(map[s])` dual-fire (`05 §6`). |
| 87 | +- **Hot path unchanged**: recv/send/read/write/recvfrom/sendto keep the v5 single `ff_is_kernel_fd` check and do **not** add `ff_native_map_get` (NFR-2). |
| 88 | + |
| 89 | +### 3.4 `lib/ff_epoll.c` (HOST_CFLAGS) |
| 90 | +- `ff_epoll_ctl`(`:99-115`): keep the existing `ff_is_kernel_fd(fd)` branch (kernel-only fd); **add**: for a dual-stack listen fd with `ff_native_map_get(fd)>0` — register on the kqueue (original path, `:120+`) **and** `ff_host_epoll_ctl(ff_epoll_host_ep(epfd,1), op, ff_native_map_get(fd), event)` (pass `event.data`). |
| 91 | +- `ff_epoll_wait`(`:214-252`): the existing merge skeleton already supports it (host `timeout=0` first, then kqueue), no major change; confirm both stacks' listen events enter the merge. |
| 92 | +- `ff_close` cooperation: when a kqueue fd is closed, clear the `ff_epoll_pairs` pairing + `ff_host_close(host_ep)` (new helper, e.g. `ff_epoll_close_pair(kq)`). |
| 93 | + |
| 94 | +### 3.5 `lib/Makefile` |
| 95 | +- Already in place (`:174-177` dual CFLAGS); no change (`ff_native_fd_map` lives in HOST_CFLAGS `ff_host_interface.c`, the dual-drive branches in CFLAGS `ff_syscall_wrapper.c`, all covered). |
| 96 | + |
| 97 | +### 3.6 demo |
| 98 | +- Rework `example/helloworld_stacksel` or add a new one: a plain `ff_socket`+`bind(80)`+`listen` (no marker) + `ff_epoll` loop accepting/handling both stacks' connections; return a preset HTTP body. Used for IT-1/2/3 real-machine dual-stack. |
60 | 99 |
|
61 | 100 | --- |
62 | 101 |
|
63 | | -## 3. Key Design Decisions |
64 | | -- **Reuse first**: hook coexistence is implemented; R2 mainly re-verifies/solidifies + a correct demo. |
65 | | -- **Native coexistence core**: managed kernel fd (**not a raw bypass**; lib registers ownership and integrates it into unified events) + `ff_epoll_wait` merge; the default path is byte-for-byte zero regression. |
66 | | -- **fd-space distinction**: mimic the nginx `ngx_max_sockets` offset or the hook encoded offset; fixed in the implementation phase. |
67 | | -- **Reachability layering**: config/fd-ownership/event-merge go through cmocka unit tests (host compilation, no DPDK runtime); same-process dual-stack end-to-end goes through integration (DPDK runtime, prefer vdev+--no-huge to avoid a physical NIC). |
| 102 | +## 4. Key Design Decisions |
| 103 | +- **Reuse first**: `FF_KERNEL_FD_BASE`/`ff_host_*`/`ff_epoll_pairs` are reused from v5; v6 only layers on `ff_native_fd_map` + default dual-build/dual-drive. |
| 104 | +- **Lock-free map**: single-threaded polling model, mimics the hook `fstack_kernel_fd_map` (NFR-5). |
| 105 | +- **Zero hot-path overhead**: a connection fd is single-stack; recv/send do not consult the map (NFR-2). |
| 106 | +- **connect ambiguity**: the draft is pending user confirmation; a pure-kernel client uses `SOCK_KERNEL` to avoid the ambiguity. |
| 107 | +- **Reachability layering**: map/dual-build/dual-drive/accept-ownership/event-merge go through cmocka (host compilation + mocked `ff_host_*`); one-listen-many-uses goes through real-machine DPDK dual-stack integration. |
68 | 108 |
|
69 | 109 | --- |
70 | 110 |
|
71 | | -## 4. Execution Steps |
72 | | -1. R0 revert (done). |
73 | | -2. R1 spec rewrite (Chinese & English done). |
74 | | -3. R2 hook coexistence solidification + demo + build. |
75 | | -4. R3 native unified-event coexistence + config change + build. |
76 | | -5. R4 unit/integration/performance. |
77 | | -6. R5 gate + English spec + commit. |
| 111 | +## 5. Execution Steps (done) |
| 112 | +1. R0-R6 done (see §2). |
| 113 | +2. R7 spec upgraded to v6 (Chinese & English synced). |
| 114 | +3. **R7 implementation done** (connect contract Q2=B confirmed): 3.1→3.2 (map)→3.3 (socket dual-build + bind/listen/close/accept dual-drive + setsockopt/fcntl)→3.4 (epoll dual-register + close clears the pairing)→3.6 (demo). |
| 115 | +4. R7 tests done: cmocka dual-mode (macro-off P1 50/50; macro-on P1 incl. `test_ff_native_fd_map`/`test_ff_kernel_fd_encode_roundtrip`) + real-machine dual-stack (single listen(80): kernel `curl 127.0.0.1:80=200`, F-Stack `ssh f-stack-client→9.134.214.176:80=200`). **Note**: the v6 wrk throughput baseline for PERF-1/2/4 was not re-run, see `10 §10`. |
| 116 | +5. R7 gate PASS: `08 §4` V1-V12 measured; dual-build `nm` zero regression (macro-off coexist symbols=0, size 6539682 identical to baseline; macro-on incl. `ff_native_fd_map`); Chinese & English spec synced; English short commit `13b418191`; config local values not committed. bounce=1 (test_ff_epoll stub, fixed). |
78 | 117 |
|
79 | | -## 5. Workspace Script Conventions |
80 | | -Delete files via `/data/workspace/rm_tmp_file.sh`; stop processes via `/data/workspace/kill_process.sh`; change permissions via `/data/workspace/chmod_modify.sh`; `make install`-type (non-direct chmod) commands may be executed. |
| 118 | +## 6. Workspace Script Conventions |
| 119 | +Delete files via `/data/workspace/rm_tmp_file.sh`; stop processes via `/data/workspace/kill_process.sh`; change permissions via `/data/workspace/chmod_modify.sh`; after a header change, `clean` full lib rebuild (`10 §7`). |
0 commit comments