Skip to content

Commit c429be2

Browse files
committed
docs(stack-coexist): backfill kernel_event_support_spec 09/10 to R7 measured-PASS
Align 09/10 (zh_cn + English) with 08: R7 native auto dual-stack marked done/PASS (commit 13b4181). 10 adds a §10 v6 verdict that honestly separates measured/provable PASS (macro-off nm zero regression, dual-mode unit tests, real-machine one-listen dual-stack 200/200, PERF-4 hot-path-no-map by code) from the v6 wrk throughput baseline (PERF-1/2) that was not re-run.
1 parent 13b4181 commit c429be2

4 files changed

Lines changed: 156 additions & 64 deletions

File tree

docs/kernel_event_support_spec/09-impl-plan.md

Lines changed: 87 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -3,78 +3,117 @@
33
> **Document ID**: SPEC-KE-09 (implementation-phase plan)
44
> **Version**: v6 (native automatic dual-stack coexistence paradigm)
55
> **Date**: 2026-06-17
6-
> **Status**: In progress (R0-R6 done; **R7 v6 auto dual-stack to-be-implemented**)
7-
> **Basis**: the v6 spec in this directory (00-08); line numbers are subject to the actual code, gatekeeper re-verified. v6 changes are to-be-implemented design.
6+
> **Status**: R0-R7 all done and gate PASS (**R7 v6 auto dual-stack measured, commit 13b418191**; real-machine single listen(80) dual-stack 9.134.214.176 + 127.0.0.1 each HTTP 200)
7+
> **Basis**: the v6 spec in this directory (00-08); line numbers are subject to the actual code, gatekeeper re-verified. v6 changes are landed; §3 below is implemented code.
88
> **Authoritative full text**: `zh_cn/09-impl-plan.md`.
99
10-
> **v6 sync (key points; see `zh_cn/09-impl-plan.md` for full detail)**: **R7 per-file rework** (under `#ifdef FF_KERNEL_COEXIST`, runtime `kernel_coexist=0` short-circuit, `SOCK_FSTACK`/macro-off byte-for-byte zero regression):
11-
> - (1) `ff_host_interface.h`: declare `ff_native_map_get/set/clear` (NO struct-header change).
12-
> - (2) `ff_host_interface.c`: `static int ff_native_fd_map[FF_MAX_FREEBSD_FILES]`(=65536, lock-free) + accessors (bounds-checked).
13-
> - (3) `ff_syscall_wrapper.c`: `ff_socket:915-947` default dual-build (`sys_socket`(s)+`ff_host_socket`(h)+`ff_native_map_set(s,h)`, return s; markers single-stack); `ff_bind:1607-1627` (`kern_bindat` then `ff_host_bind(map[s], raw linux addr)`); `ff_listen:1584-1605` (`sys_listen` then `ff_host_listen(map[s])`); `ff_close:1095-1112` (`kern_close` then `ff_host_close(map[fd])`+`ff_native_map_clear`); `ff_accept/accept4:1514-1582` single-stack ownership; `ff_setsockopt:999`/`ff_fcntl:1495` sync both; `ff_connect:1629-1649` §connect draft (pending user confirmation). recv/send/read/write/recvfrom/sendto unchanged (single-stack by `ff_is_kernel_fd`, NO map lookup).
14-
> - (4) `ff_epoll.c`: `ff_epoll_ctl:99-115` dual-register a dual-stack listen (kqueue + `ff_host_epoll_ctl(host_ep, op, map[fd], event)`, pass `event.data`); `ff_epoll_wait:214-252` merge (existing skeleton); `ff_close` clear `ff_epoll_pairs` for kqueue fd.
15-
> - (5) `lib/Makefile`: already in place (`:174-177` dual CFLAGS); no change.
16-
> - (6) demo: default dual-stack `listen(80)` demo for real-machine IT-1/2/3.
17-
> Then R7 tests (cmocka dual-mode + real-machine dual-stack + perf) and gate (`08 §4` V1-V12; MT-1/3 incl. `ff_native_fd_map`); English spec sync; English short commit; config local values NOT committed; clean full rebuild after any header change (ABI skew).
18-
1910
---
2011

2112
## 0. Scope and Gate
2213

23-
- **Do all of R0-R5**: revert wrong code → spec rewrite → hook coexistence solidification + demo → native unified-event coexistence → tests/performance → gate + commit.
24-
- **Hard gate (unconditionally all green)**: compilation passes + cmocka unit tests all green + coverage met + **F-Stack business fast-path zero regression (NFR-1/NFR-2)**.
25-
- **Coexistence iron rule (NFR-3)**: at every phase the F-Stack user-space stack always carries the business and is never bypassed by the kernel stack; violating it bounces.
26-
- **Target gate**: best-effort to actually set up the DPDK runtime and run the same-process dual-stack integration; when a real NIC is unavailable, the business plane is skipped with measured evidence and the kernel side is still measured on loopback.
14+
- **R0-R6 done**: revert → spec → hook solidification → native per-fd coexistence → tests/perf → gate/commit → compile-macro gating (ba148589d).
15+
- **R7 done (commit 13b418191)**: native automatic dual-stack (default dual-build/dual-drive + `ff_native_fd_map` + dual-stack events + accept ownership + dual-fire connect) landed and gate PASS.
16+
- **Hard gate (met)**: dual-mode compilation passes + cmocka dual-mode all green + **macro-off `nm` coexist symbols=0 (size 6539682, byte-for-byte identical to baseline, zero regression)** + **F-Stack business fast path + single-stack connection hot path zero regression (NFR-1/NFR-2)**.
17+
- **Coexistence iron rule (NFR-3)**: the F-Stack fd is always built and always carries the business; satisfied.
18+
- **connect contract**: implemented per the user-confirmed Q2=B (dual-fire connect, F-Stack as the return/data primary path) in `05 §6`.
2719

2820
---
2921

30-
## 1. Agent Team Topology (harness + spec-driven)
22+
## 1. Agent Team Topology
3123

3224
| Agent | Role | Responsibility |
3325
|---|---|---|
34-
| **Leader** | orchestration+authoring+arbitration | orchestration, coding, gate arbitration, commit, bounce counting |
35-
| **arch-probe** | architecture probe (read-only) | measure hook FF_KERNEL_EVENT / nginx / native event layer |
36-
| **spec-writer** | spec rewrite | v4 Chinese & English docs |
37-
| **build** | compilation (actual build) | lib / libff_syscall.so / tests |
38-
| **unit-test** | unit tests | cmocka coexistence cases |
39-
| **review** | review (read-only) | minimal diff/zero regression/coexistence iron rule/conventions |
40-
| **test** | integration/performance | same-process dual-stack end-to-end + no regression on the F-Stack fast path |
41-
| **gatekeeper** | gate (read-only) | per-assertion + gate-item verification |
26+
| Leader | orchestration+authoring+arbitration | orchestration, coding, gate arbitration, commit, bounce counting |
27+
| arch-probe | architecture probe (read-only) | measure v5 current state + hook isomorphism/divergence |
28+
| spec-writer | spec upgrade | v6 Chinese & English docs (this round's output) |
29+
| build | compilation (actual build) | lib dual-build / tests |
30+
| unit-test | unit tests | cmocka v6 dual-mode cases |
31+
| review | review (read-only) | minimal diff/zero regression/coexistence iron rule/conventions |
32+
| test | integration/performance | one-listen-many-uses real-machine dual-stack + fast-path no regression |
33+
| gatekeeper | gate (read-only) | per-assertion + gate items |
4234

4335
**Gate rollback**: any phase failing bounces back to the previous step; ≤3 bounces for the same step, escalate to manual when exceeded; bounces recorded in `08`.
4436

4537
---
4638

47-
## 2. Change Points (measured anchors, code is authoritative)
39+
## 2. R0-R6 Change Points (done, measured anchors)
4840

4941
| Milestone | File | Change |
5042
|---|---|---|
51-
| R0 | `lib/ff_syscall_wrapper.c`, `lib/ff_host_interface.{c,h}` | revert the ff_host_socket side path (**done 0748eff94**) |
52-
| R2 | `adapter/syscall/` (Makefile FF_KERNEL_EVENT), new demo | build libff_syscall.so; a correct same-process dual-stack demo (modeled after `main_stack_epoll_kernel.c`), replacing the v3 pure-kernel helloworld_stacksel |
53-
| R3 | `lib/ff_config.{c,h}` | change the v3 `stack.default_to_kernel`/`default_stack` to `stack.kernel_coexist` (`MATCH("stack","kernel_coexist")`, default 0) + accessor |
54-
| R3 | `lib/ff_host_interface.{c,h}` | add the managed kernel-side bridge (host socket/bind/listen/accept/connect/close/epoll_*) for lib to create managed kernel fds |
55-
| R3 | `lib/ff_syscall_wrapper.c` `ff_socket` and ff_bind/listen/accept/connect/close | when coexistence enabled, SOCK_KERNEL→managed kernel fd+register ownership; route by ownership; default/`SOCK_FSTACK` zero regression |
56-
| R3 | `lib/ff_epoll.c` | `ff_epoll_create` also creates a kernel epoll; `ff_epoll_ctl` splits; `ff_epoll_wait` merges kqueue⊕epoll; close linkage |
57-
| R3 | `config.ini` | `[stack] kernel_coexist=0` example section (replacing the v3 default_stack) |
58-
| R4 | `tests/unit/`, `tests/integration/` | cmocka coexistence cases + same-process dual-stack integration |
59-
| docs | `docs/kernel_event_support_spec/` (Chinese & English 00-10) | v4 coexistence paradigm |
43+
| R0 | `ff_syscall_wrapper.c`, `ff_host_interface.{c,h}` | revert the ff_host_socket side path (0748eff94) |
44+
| R2 | `adapter/syscall/`, demo | hook solidification + same-process dual-stack demo |
45+
| R3 | `ff_config.{c,h}` | `stack.kernel_coexist` (`:1027-1031`/`:1363`/`:321-323`) |
46+
| R3 | `ff_host_interface.{c,h}` | 18 `ff_host_*` bridges + `FF_KERNEL_FD_BASE` |
47+
| R3 | `ff_syscall_wrapper.c` | `ff_socket` per-fd either/or + 13-entry `ff_is_kernel_fd` routing |
48+
| R3 | `ff_epoll.c` | `ff_epoll_pairs` merge |
49+
| R6 | `lib/Makefile` + 7 files | `FF_KERNEL_COEXIST` gating (`:174-177` dual CFLAGS) |
50+
51+
---
52+
53+
## 3. R7 native automatic dual-stack per-file rework (v6 core, implemented commit 13b418191)
54+
55+
> All new code goes inside `#ifdef FF_KERNEL_COEXIST`; runtime `kernel_coexist=0` short-circuits; `SOCK_FSTACK`/macro-off is byte-for-byte zero regression.
56+
57+
### 3.1 `lib/ff_host_interface.h` (HOST_CFLAGS header)
58+
- Inside the `#ifdef FF_KERNEL_COEXIST` block (currently `:94-160`) add accessor declarations:
59+
```c
60+
int ff_native_map_get(int fstack_fd);
61+
void ff_native_map_set(int fstack_fd, int host_fd);
62+
void ff_native_map_clear(int fstack_fd);
63+
```
64+
- **Do not touch the struct header** (avoid ABI skew, `10 §7`).
65+
66+
### 3.2 `lib/ff_host_interface.c` (HOST_CFLAGS)
67+
- In the existing 18-bridge block (inside `#ifdef FF_KERNEL_COEXIST`) add:
68+
```c
69+
static int ff_native_fd_map[FF_MAX_FREEBSD_FILES]; /* =65536, mimics ff_hook_syscall.c:258, lock-free */
70+
int ff_native_map_get(int fd){ return (fd>=0 && fd<FF_MAX_FREEBSD_FILES) ? ff_native_fd_map[fd] : 0; }
71+
void ff_native_map_set(int fd,int h){ if(fd>=0 && fd<FF_MAX_FREEBSD_FILES) ff_native_fd_map[fd]=h; }
72+
void ff_native_map_clear(int fd){ if(fd>=0 && fd<FF_MAX_FREEBSD_FILES) ff_native_fd_map[fd]=0; }
73+
```
74+
- `#define FF_MAX_FREEBSD_FILES` here if lib does not already define it (mimics adapter).
75+
76+
### 3.3 `lib/ff_syscall_wrapper.c` (CFLAGS)
77+
- `ff_socket`(`:915-947`): refactor inside the `#ifdef FF_KERNEL_COEXIST` block —
78+
- `SOCK_KERNEL && !SOCK_FSTACK && coexist`: keep v5 (kernel only + encode).
79+
- **new default dual-stack**: `!SOCK_FSTACK && coexist` (no marker) → first `sys_socket` builds F-Stack fd `s`, then `ff_host_socket(...)` builds host `h`, `ff_native_map_set(s,h)`, return `s`; `ff_host_socket` failure per the `05 §7` contract.
80+
- `SOCK_FSTACK` / coexist-off: original `sys_socket` path (`:937-943` unchanged).
81+
- `ff_bind`(`:1607-1627`): keep the existing `#ifdef` `ff_is_kernel_fd` block; after `kern_bindat` succeeds (`:1620-1623`) add a `#ifdef` block: `int h=ff_native_map_get(s); if(h>0) ff_host_bind(h, addr, addrlen);` (use the **raw linux addr**, not bsdaddr).
82+
- `ff_listen`(`:1584-1605`): after `sys_listen` succeeds add `int h=ff_native_map_get(s); if(h>0) ff_host_listen(h, backlog);`.
83+
- `ff_close`(`:1095-1112`): after `kern_close` succeeds add `int h=ff_native_map_get(fd); if(h>0){ ff_host_close(h); ff_native_map_clear(fd); }`; and clear `ff_epoll_pairs` for a kqueue fd (cooperating with `ff_epoll.c`, see 3.4).
84+
- `ff_accept`/`ff_accept4`(`:1514-1582`): for a dual-stack listen fd (`ff_native_map_get(s)>0`) follow `05 §5` single-stack ownership (kern_accept→EAGAIN→ff_host_accept+encode).
85+
- `ff_setsockopt`(`:999`)/`ff_fcntl`(`:1495`): for a dual-stack fd, after the F-Stack path sync `ff_host_setsockopt/fcntl` to `map[s]`.
86+
- `ff_connect`(`:1629-1649`): implemented per Q2=B — `kern_connectat` primary, best-effort `ff_host_connect(map[s])` dual-fire (`05 §6`).
87+
- **Hot path unchanged**: recv/send/read/write/recvfrom/sendto keep the v5 single `ff_is_kernel_fd` check and do **not** add `ff_native_map_get` (NFR-2).
88+
89+
### 3.4 `lib/ff_epoll.c` (HOST_CFLAGS)
90+
- `ff_epoll_ctl`(`:99-115`): keep the existing `ff_is_kernel_fd(fd)` branch (kernel-only fd); **add**: for a dual-stack listen fd with `ff_native_map_get(fd)>0` — register on the kqueue (original path, `:120+`) **and** `ff_host_epoll_ctl(ff_epoll_host_ep(epfd,1), op, ff_native_map_get(fd), event)` (pass `event.data`).
91+
- `ff_epoll_wait`(`:214-252`): the existing merge skeleton already supports it (host `timeout=0` first, then kqueue), no major change; confirm both stacks' listen events enter the merge.
92+
- `ff_close` cooperation: when a kqueue fd is closed, clear the `ff_epoll_pairs` pairing + `ff_host_close(host_ep)` (new helper, e.g. `ff_epoll_close_pair(kq)`).
93+
94+
### 3.5 `lib/Makefile`
95+
- Already in place (`:174-177` dual CFLAGS); no change (`ff_native_fd_map` lives in HOST_CFLAGS `ff_host_interface.c`, the dual-drive branches in CFLAGS `ff_syscall_wrapper.c`, all covered).
96+
97+
### 3.6 demo
98+
- Rework `example/helloworld_stacksel` or add a new one: a plain `ff_socket`+`bind(80)`+`listen` (no marker) + `ff_epoll` loop accepting/handling both stacks' connections; return a preset HTTP body. Used for IT-1/2/3 real-machine dual-stack.
6099

61100
---
62101

63-
## 3. Key Design Decisions
64-
- **Reuse first**: hook coexistence is implemented; R2 mainly re-verifies/solidifies + a correct demo.
65-
- **Native coexistence core**: managed kernel fd (**not a raw bypass**; lib registers ownership and integrates it into unified events) + `ff_epoll_wait` merge; the default path is byte-for-byte zero regression.
66-
- **fd-space distinction**: mimic the nginx `ngx_max_sockets` offset or the hook encoded offset; fixed in the implementation phase.
67-
- **Reachability layering**: config/fd-ownership/event-merge go through cmocka unit tests (host compilation, no DPDK runtime); same-process dual-stack end-to-end goes through integration (DPDK runtime, prefer vdev+--no-huge to avoid a physical NIC).
102+
## 4. Key Design Decisions
103+
- **Reuse first**: `FF_KERNEL_FD_BASE`/`ff_host_*`/`ff_epoll_pairs` are reused from v5; v6 only layers on `ff_native_fd_map` + default dual-build/dual-drive.
104+
- **Lock-free map**: single-threaded polling model, mimics the hook `fstack_kernel_fd_map` (NFR-5).
105+
- **Zero hot-path overhead**: a connection fd is single-stack; recv/send do not consult the map (NFR-2).
106+
- **connect ambiguity**: the draft is pending user confirmation; a pure-kernel client uses `SOCK_KERNEL` to avoid the ambiguity.
107+
- **Reachability layering**: map/dual-build/dual-drive/accept-ownership/event-merge go through cmocka (host compilation + mocked `ff_host_*`); one-listen-many-uses goes through real-machine DPDK dual-stack integration.
68108

69109
---
70110

71-
## 4. Execution Steps
72-
1. R0 revert (done).
73-
2. R1 spec rewrite (Chinese & English done).
74-
3. R2 hook coexistence solidification + demo + build.
75-
4. R3 native unified-event coexistence + config change + build.
76-
5. R4 unit/integration/performance.
77-
6. R5 gate + English spec + commit.
111+
## 5. Execution Steps (done)
112+
1. R0-R6 done (see §2).
113+
2. R7 spec upgraded to v6 (Chinese & English synced).
114+
3. **R7 implementation done** (connect contract Q2=B confirmed): 3.1→3.2 (map)→3.3 (socket dual-build + bind/listen/close/accept dual-drive + setsockopt/fcntl)→3.4 (epoll dual-register + close clears the pairing)→3.6 (demo).
115+
4. R7 tests done: cmocka dual-mode (macro-off P1 50/50; macro-on P1 incl. `test_ff_native_fd_map`/`test_ff_kernel_fd_encode_roundtrip`) + real-machine dual-stack (single listen(80): kernel `curl 127.0.0.1:80=200`, F-Stack `ssh f-stack-client→9.134.214.176:80=200`). **Note**: the v6 wrk throughput baseline for PERF-1/2/4 was not re-run, see `10 §10`.
116+
5. R7 gate PASS: `08 §4` V1-V12 measured; dual-build `nm` zero regression (macro-off coexist symbols=0, size 6539682 identical to baseline; macro-on incl. `ff_native_fd_map`); Chinese & English spec synced; English short commit `13b418191`; config local values not committed. bounce=1 (test_ff_epoll stub, fixed).
78117

79-
## 5. Workspace Script Conventions
80-
Delete files via `/data/workspace/rm_tmp_file.sh`; stop processes via `/data/workspace/kill_process.sh`; change permissions via `/data/workspace/chmod_modify.sh`; `make install`-type (non-direct chmod) commands may be executed.
118+
## 6. Workspace Script Conventions
119+
Delete files via `/data/workspace/rm_tmp_file.sh`; stop processes via `/data/workspace/kill_process.sh`; change permissions via `/data/workspace/chmod_modify.sh`; after a header change, `clean` full lib rebuild (`10 §7`).

0 commit comments

Comments
 (0)