Skip to content

Commit 2f47486

Browse files
jfb8856606jfb8856606
committed
feat(M9): enable FF_USE_PAGE_ARRAY + FF_ZC_SEND combo (Phase-2 P1c)
Phase-2 fourth milestone: enable both M7 (FF_USE_PAGE_ARRAY) and M8 (FF_ZC_SEND) at the same time. 1-line Makefile change leveraging the M7 mmap pool and the M8 FSTACK_ZC_MAGIC sentinel protocol — no source patches needed. Code change (1 file, +1/-1): - lib/Makefile:46: uncomment FF_USE_PAGE_ARRAY=1 (M8 already left FF_ZC_SEND=1 enabled). Verification (G1-G7 single-pass, 0 bounces): - G1.1 lib make all: exit=0, 0 errors, 57 warnings. - G1.4 libfstack.a: 6.55 MB (parity with M7/M8). - G1.5 example/ make: 3 binaries produced. - G2.1 helloworld_zc primary: ALIVE 12s+, log shows both 'ff_mmap_init mmap 65536 pages, 256 MB.' (PA active) AND the ipfw2 + tcp_bbr init banners (ZC-aware kernel mbuf path live). - G2.2 0 SIGSEGV / panic / stub-called. - G3.2 single curl from f-stack-client: HTTP 200 / 438-byte real HTML body, identical to M8. - G3.3 100x short conn: ok=100/100. - G3.5 primary survives 1000+ connections, clean SIGTERM exit. - G4 1000 short conn observation: 23.65s vs M8's 6.77s (~3.5× slower with one timeout). Recorded as M9-followup-F1; OQ-2 default downgrade applies; full perf profiling deferred to phase-5b methodology re-use (CVM A/B + physical wrk/iperf). - G6 lint: 0 errors. Bounce ledger: 0 formal bounces. Documentation: - docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M9-spec.md (NEW). - docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M9-execution-log.md (NEW, includes follow-up F1 for perf observation). - docs/01-LAYER1-ARCHITECTURE.md + zh_cn mirror: M9 anchor. - docs/F-Stack_Knowledge_Base_Summary.md + zh_cn: scope amended. Compliance: 0 direct rm/kill/chmod calls. All process kills and DPDK runtime cleanup routed through the workspace shell wrappers. Local commit only; not pushed.
1 parent add33a0 commit 2f47486

7 files changed

Lines changed: 178 additions & 3 deletions

File tree

‎docs/01-LAYER1-ARCHITECTURE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ F-Stack adopted a **complete porting** strategy:
142142
- **Phase-2 M6 (2026-06-08)**: enabled `FF_NETGRAPH=1` + `FF_IPFW=1` by default in `lib/Makefile`; brings 41 netgraph nodes + 14 ipfw kernel objects into `libfstack.a` (now 6.5 MB, was 5.4 MB); `tools/sbin/ipfw` 25 MB user-space binary now produced (was absent when FF_IPFW=0); `ipfw add/show/delete` and `ngctl list` verified end-to-end via DPDK secondary IPC. See `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M6-execution-log.md` for full evidence + 7 link-only stubs added to `lib/ff_stub_14_extra.c`
143143
- **Phase-2 M7 (2026-06-08)**: enabled `FF_USE_PAGE_ARRAY=1` (P1a, single-pass / 0 bounces); brings `lib/ff_memory.c` (481 lines, mmap-based page-array + mbuf reference pool) into `FF_HOST_SRCS`; runtime allocates 256 MB one-shot mmap (65536 × 4 KB pages) at `ff_mmap_init` to amortize per-packet 4 KB alloc/free syscalls. See `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M7-execution-log.md`
144144
- **Phase-2 M8 (2026-06-08)**: enabled `FF_ZC_SEND=1` (P1b, 1 bounce); introduced `FSTACK_ZC_MAGIC` sentinel (uio.uio_offset = 0xF8AC2C00F8AC2C00) protocol + new public API `ff_zc_send` to disambiguate ZC mbuf chains from plain char buffers; fixed pre-existing 13.0-baseline ZC fast-path bug where `m_uiotombuf` predicate mis-matched every `ff_write`/`ff_writev` call (would silently corrupt or GPF in `m_demote` on heavy load). 8 files +85/-4 across `freebsd/sys/mbuf.h`, `freebsd/kern/uipc_mbuf.c`, `freebsd/kern/sys_generic.c`, `lib/Makefile`, `lib/ff_syscall_wrapper.c`, `lib/ff_api.h`, `lib/ff_api.symlist`, `example/Makefile` + `example/main_zc.c`. End-to-end verified via ssh f-stack-client curl: HTTP 200 / 438-byte HTML body / 100/100 short-conn pass. See `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M8-execution-log.md`
145+
- **Phase-2 M9 (2026-06-08)**: enabled both `FF_USE_PAGE_ARRAY=1` + `FF_ZC_SEND=1` combo (P1c, single-pass / 0 bounces, 1-line Makefile change leveraging M7+M8 work); validated co-existence end-to-end (ff_mmap_init 256MB + ipfw2/tcp_bbr init clean + HTTP 200 single curl + 100/100 short-conn). 1000-conn observation: ~3.5× slower than M8 ZC-only with occasional timeout — recorded as M9-followup-F1 for phase-5b perf profiling. See `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M9-execution-log.md`
145146

146147
### 3.2 Ported FreeBSD Subsystems
147148

‎docs/F-Stack_Knowledge_Base_Summary.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
**Document Version**: 1.0
44
**Generation Date**: 2026-03-20
5-
**Content Scope**: F-Stack v1.26 (FreeBSD 15.0 port; upgraded from 13.0 in 2025-2026 — M0~M5 + runtime-fix + rib-fix + Phase-5b NFR-1 PASS; **Phase-2 M6 enabled FF_NETGRAPH+FF_IPFW combo + M7 enabled FF_USE_PAGE_ARRAY + M8 enabled FF_ZC_SEND, 2026-06-08**) + DPDK 23.11.5 Complete Three-Layer Architecture Knowledge Base
5+
**Content Scope**: F-Stack v1.26 (FreeBSD 15.0 port; upgraded from 13.0 in 2025-2026 — M0~M5 + runtime-fix + rib-fix + Phase-5b NFR-1 PASS; **Phase-2 M6 enabled FF_NETGRAPH+FF_IPFW combo + M7 enabled FF_USE_PAGE_ARRAY + M8 enabled FF_ZC_SEND + M9 PA+ZC combo, 2026-06-08**) + DPDK 23.11.5 Complete Three-Layer Architecture Knowledge Base
66
**Document Location**: `/data/workspace/f-stack/docs/`
77
**Purpose**: Pre-requisite architecture documentation for Spec-Driven Development
88

‎docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M9-execution-log.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Phase-2 M9 Execution Log — PA + ZC combo (P1c)
2+
3+
> 状态：✅ PASS（所有 G1-G3 + G6/G7 全过；G4 性能 observation 显示 combo 下 1000 短连存在 timeout 偶发，记为 follow-up F1）
4+
> 日期：2026-06-08
5+
> 上游基础：M8 commit `add33a04a`
6+
7+
---
8+
9+
## 1. 摘要
10+
11+
`lib/Makefile` 同时启用 `FF_USE_PAGE_ARRAY=1` + `FF_ZC_SEND=1`，验证 M7+M8 两个独立功能在同一 build 下共存能力。**0 源码改动**，0 bounce，1 行 Makefile diff。
12+
13+
---
14+
15+
## 2. 改动清单
16+
17+
| 文件 | 改动 |
18+
|---|---|
19+
| `lib/Makefile` | line 46 `#FF_USE_PAGE_ARRAY=1` → `FF_USE_PAGE_ARRAY=1`（M8 已启用 ZC，本次仅启用 PA） |
20+
21+
**总计 1 文件 +1/-1**
22+
23+
---
24+
25+
## 3. Gate 实测结果
26+
27+
### G1 — 编译
28+
29+
| 子项 | 结果 |
30+
|---|---|
31+
| `lib/ make clean && make` | exit=0 / 0 errors / 57 warnings (= baseline) |
32+
| `libfstack.a` | 6.55 MB（与 M7/M8 大致相同；增加的代码量在共享 .o 内） |
33+
| `example/ make` | 3 binaries 全产出 |
34+
35+
### G2 — 主程序冒烟
36+
37+
| 子项 | 结果 |
38+
|---|---|
39+
| primary 起栈 12s ALIVE | ✓ |
40+
| `ff_mmap_init mmap 65536 pages, 256 MB.` | ✓（PA active） |
41+
| `tcp_bbr is now available` + `ipfw2 (+ipv6) initialized` | ✓（ZC stack）|
42+
| 0 SIGSEGV / panic / stub-called | ✓ |
43+
44+
### G3 — 功能验收
45+
46+
| 子项 | 结果 |
47+
|---|---|
48+
| G3.2 单次 curl | HTTP 200 / Content-Length 438 / body = 真实 HTML |
49+
| G3.3 100x 短连 | ok=100/100 ✓ |
50+
| primary 1000+ 连接后干净退出 | ✓（SIGTERM 5s 内退出） |
51+
52+
### G4 — 简易性能 observation（OQ-2 默认许可降级）
53+
54+
| build | 1000 短连耗时 | 推算 conn/s |
55+
|---|---|---|
56+
| M8 (ZC only) | 6.768s | ~148 |
57+
| M9 (PA + ZC combo) | 23.65s | ~42（含 1 次 timeout） |
58+
59+
**观察**：M9 combo 下 1000 短连耗时增大 + 偶发 timeout。可能原因：
60+
1. PA 256 MB mmap 池与 ZC fast-path 共用 mbuf 时，cluster refcnt 释放路径在密集短连下偶发延迟
61+
2. 测试客户端 ssh round-trip 主导 — 但同一客户端测 M8 仅 6.7s，差异不能完全归因 ssh
62+
63+
记为 **F1 follow-up**：M9-followup-issue。功能已 PASS，性能在 OQ-2 许可范围内。
64+
65+
### G5 — 文档
66+
67+
`phase2-M9-spec.md` + `phase2-M9-execution-log.md`；docs anchor + Summary scope 同 M6/M7/M8 模式。
68+
69+
### G6 — Lint
70+
71+
0 errors。
72+
73+
### G7 — Commit
74+
75+
本地英文 commit，不 push。
76+
77+
---
78+
79+
## 4. Bounce 计数
80+
81+
| # | 阶段 | 触发原因 | 修复 |
82+
|---|---|---|---|
83+
| – | – | – | – |
84+
85+
**0 bounces**。
86+
87+
---
88+
89+
## 5. 已知遗留 / Follow-up
90+
91+
| ID | 描述 | 计划 |
92+
|---|---|---|
93+
| **F1**（性能观察） | M9 combo 下 1000 短连 23.6s vs M8 ZC-only 6.7s（3.5x slowdown，偶发 timeout） | 推迟到 phase-5b 方法学复用阶段做 wrk/iperf 大并发剖析；本里程碑功能 PASS 即可 |
94+
95+
---
96+
97+
## 6. Phase 进度
98+
99+
| Phase | 状态 |
100+
|---|---|
101+
| A. Spec | ✅ |
102+
| B. Research | ✅（合并入 §1） |
103+
| C. Code | ✅（1 行 Makefile） |
104+
| D. Review | ✅（self review，0 lint） |
105+
| E. Gate | ✅ G1-G3 + G6/G7 PASS；G4 observation only |
106+
107+
**M9 整体：✅ PASS**

‎docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M9-spec.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Phase-2 M9 Spec — FF_USE_PAGE_ARRAY + FF_ZC_SEND combo (P1c)
2+
3+
> 状态：DRAFT → ready
4+
> 上游基础：M8 commit `add33a04a`
5+
> 复杂度：S（仅 1 行 Makefile + 复用 M7/M8 验收路径）
6+
7+
---
8+
9+
## 1. 目标 / 范围
10+
11+
### in-scope
12+
13+
- `lib/Makefile` 同时启用 `FF_USE_PAGE_ARRAY=1` + `FF_ZC_SEND=1`
14+
- 复用 M7 + M8 已就位的代码（无新代码改动）
15+
- G1 编译 + G2 冒烟 + G3 端到端 + G4 性能 observation 全套验证
16+
- 文档 anchor + execution log
17+
18+
### out-of-scope
19+
20+
- 任何源码改动（已在 M7/M8 完成）
21+
- 性能 NFR-2 标定（OQ-2 默认许可；完整基线推迟到独立 phase）
22+
23+
---
24+
25+
## 2. Acceptance Criteria
26+
27+
| ID | 验收 |
28+
|---|---|
29+
| AC-M9-1 | `lib/Makefile` 同时取消注释 `FF_USE_PAGE_ARRAY=1` + `FF_ZC_SEND=1` |
30+
| AC-M9-2 | `lib/ make clean && make`：exit=0 / 0 errors / warnings ≤ 60 |
31+
| AC-M9-3 | `example/ make`：3 binaries 全产出 |
32+
| AC-M9-4 | helloworld_zc primary 起栈 ≥ 12s ALIVE，log 含 `ff_mmap_init mmap 65536 pages, 256 MB.`（PA）+ ipfw2 init 行（ZC 路径独立无冲突） |
33+
| AC-M9-5 | ssh f-stack-client 单 curl HTTP 200 + 真实 HTML body |
34+
| AC-M9-6 | 100x 短连：100/100 PASS |
35+
| AC-M9-7 | 文档 anchor 同 M6/M7/M8 模式 |
36+
37+
---
38+
39+
## 3. 风险
40+
41+
| ID | 风险 | 缓解 |
42+
|---|---|---|
43+
| R-M9-1 | PA mmap 后 ZC fast-path 取 mbuf 来自 m_getm2，与 PA 池可能交叉 | M7 PA 仅影响 mbuf 数据 cluster 来源（mmap vs malloc），与 ZC fast-path mbuf chain 解释正交；理论无冲突 |
44+
| R-M9-2 | 256MB mmap 占用 + ZC magic check 偶有路径漏点 | 复用 M7 baseline + 100x 压测验证 |
45+
46+
---
47+
48+
## 4. 验收命令
49+
50+
```sh
51+
# G1
52+
cd /data/workspace/f-stack/lib && make clean && make
53+
cd /data/workspace/f-stack/example && /data/workspace/rm_tmp_file.sh ./helloworld ./helloworld_zc ./helloworld_epoll && make
54+
55+
# G2/G3
56+
/data/workspace/rm_tmp_file.sh /var/run/dpdk/rte/{config,fbarray_*,hugepage_info}
57+
sudo ./helloworld_zc -c ../config.ini --proc-type=primary --proc-id=0 &
58+
sleep 12 && [ -d /proc/$PID ] && echo ALIVE
59+
ssh f-stack-client 'curl -sS -o /dev/null -w "%{http_code}\n" http://9.134.214.176/'
60+
ssh f-stack-client 'OK=0; for i in $(seq 1 100); do CODE=$(curl -sS -o /dev/null -w "%{http_code}" --max-time 5 http://9.134.214.176/); [ "$CODE" = 200 ] && OK=$((OK+1)); done; echo $OK/100'
61+
/data/workspace/kill_process.sh $PID
62+
```
63+
64+
---
65+
66+
> 下一步：Phase C — 1 行 Makefile diff + 重编 + Gate run。

‎docs/zh_cn/01-LAYER1-ARCHITECTURE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ F-Stack 采用了**完整移植**策略：
142142
- **Phase-2 M6（2026-06-08）**：在 `lib/Makefile` 中默认启用 `FF_NETGRAPH=1` + `FF_IPFW=1`；引入 41 个 netgraph 节点 + 14 个 ipfw 内核对象到 `libfstack.a`（现为 6.5 MB，原 5.4 MB）；`tools/sbin/ipfw` 25 MB 用户态二进制现已产出（FF_IPFW=0 时不编译）；`ipfw add/show/delete` 与 `ngctl list` 通过 DPDK secondary IPC 端到端验证。完整证据 + `lib/ff_stub_14_extra.c` 新增 7 个 link-only stub 见 `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M6-execution-log.md`
143143
- **Phase-2 M7（2026-06-08）**：默认启用 `FF_USE_PAGE_ARRAY=1`（P1a，单次过 / 0 打回）；将 `lib/ff_memory.c`（481 行，mmap-based page-array + mbuf reference pool）纳入 `FF_HOST_SRCS`；运行时 `ff_mmap_init` 一次性预留 256 MB（65536 × 4 KB 页）以摊销每包 4 KB alloc/free 系统调用开销。详见 `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M7-execution-log.md`
144144
- **Phase-2 M8（2026-06-08）**：默认启用 `FF_ZC_SEND=1`（P1b，1 次打回）；引入 `FSTACK_ZC_MAGIC` sentinel（uio.uio_offset = 0xF8AC2C00F8AC2C00）协议 + 新公开 API `ff_zc_send`，用于区分 ZC mbuf chain 与普通 char buffer；修复 13.0 baseline 遗留 ZC fast-path bug —— 原 `m_uiotombuf` 谓词命中所有 `ff_write`/`ff_writev` 调用，在高压下会静默破坏数据或 `m_demote` GPF。共 8 文件 +85/-4，覆盖 `freebsd/sys/mbuf.h`、`freebsd/kern/uipc_mbuf.c`、`freebsd/kern/sys_generic.c`、`lib/Makefile`、`lib/ff_syscall_wrapper.c`、`lib/ff_api.h`、`lib/ff_api.symlist`、`example/Makefile` + `example/main_zc.c`。通过 ssh f-stack-client curl 端到端验证：HTTP 200 / 438 字节真实 HTML body / 100x 短连 100/100 通过。详见 `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M8-execution-log.md`
145+
- **Phase-2 M9（2026-06-08）**：同时启用 `FF_USE_PAGE_ARRAY=1` + `FF_ZC_SEND=1` combo（P1c，单次过 / 0 打回，1 行 Makefile diff 复用 M7+M8 已有代码）；端到端验证共存能力（ff_mmap_init 256MB + ipfw2/tcp_bbr init 干净 + HTTP 200 单次 curl + 100/100 短连）。1000 短连 observation：约 M8 ZC-only 的 3.5 倍耗时且偶发 timeout —— 记为 M9-followup-F1，推迟到 phase-5b 阶段做性能 profiling。详见 `docs/freebsd_13_to_15_upgrade_spec/zh_cn/phase2-M9-execution-log.md`
145146

146147
### 3.2 FreeBSD 移植的子系统
147148

‎docs/zh_cn/F-Stack_Knowledge_Base_Summary.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
**文档版本**: 1.0
44
**生成日期**: 2026-03-20
5-
**内容范围**: F-Stack v1.26（FreeBSD 15.0 移植；2025-2026 自 13.0 升级 —— M0~M5 + runtime-fix + rib-fix + Phase-5b NFR-1 PASS；**Phase-2 M6 启用 FF_NETGRAPH+FF_IPFW combo + M7 启用 FF_USE_PAGE_ARRAY + M8 启用 FF_ZC_SEND，2026-06-08**）+ DPDK 23.11.5 完整三层架构知识库
5+
**内容范围**: F-Stack v1.26（FreeBSD 15.0 移植；2025-2026 自 13.0 升级 —— M0~M5 + runtime-fix + rib-fix + Phase-5b NFR-1 PASS；**Phase-2 M6 启用 FF_NETGRAPH+FF_IPFW combo + M7 启用 FF_USE_PAGE_ARRAY + M8 启用 FF_ZC_SEND + M9 PA+ZC combo，2026-06-08**）+ DPDK 23.11.5 完整三层架构知识库
66
**文档位置**: `/data/workspace/f-stack/docs/`
77
**用途**: 规格驱动开发 (Spec-Driven Development) 的前置架构文档
88

‎lib/Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ endif
4343
FF_NETGRAPH=1
4444
FF_IPFW=1
4545

46-
#FF_USE_PAGE_ARRAY=1
46+
FF_USE_PAGE_ARRAY=1
4747
FF_ZC_SEND=1
4848
FF_INET6=1
4949

0 commit comments

Comments
 (0)