|
| 1 | +# 00 — Project Overview and Glossary |
| 2 | + |
| 3 | +> Chinese version: ./zh_cn/00-overview-and-glossary.md |
| 4 | +> |
| 5 | +> Document language: English (translated from zh_cn v0.1) |
| 6 | +> Series root directory: `/data/workspace/f-stack/docs/freebsd_13_to_15_upgrade_spec/` |
| 7 | +> Document version: v0.1 (2026-05-26) |
| 8 | +
|
| 9 | +--- |
| 10 | + |
| 11 | +## 1. One-line Project Definition |
| 12 | + |
| 13 | +Upgrade the **FreeBSD 13.0-RELEASE** kernel network stack and the matching user-space tool set that the **F-Stack** project currently depends on, all the way to **FreeBSD 15.0-RELEASE**, and produce an English Spec document set that engineering teams and downstream AI agents can execute directly. |
| 14 | + |
| 15 | +> This Spec phase only produces documents and does not modify code; the actual code migration is executed by the subsequent phases defined in `05-implementation-plan.md`. |
| 16 | +
|
| 17 | +--- |
| 18 | + |
| 19 | +## 2. Project Background |
| 20 | + |
| 21 | +F-Stack is the project that detaches the FreeBSD kernel network stack and runs it in DPDK user-space: |
| 22 | + |
| 23 | +- **Idea**: extract a subset of FreeBSD `sys/` (kern / net / netinet / netinet6 / netipsec / netgraph / vm / libkern / opencrypto / ...), use a set of glue files (`ff_*.c`) to replace the parts that are tightly coupled to the host kernel (VFS / process / signal / kvm / AIO / RACCT / kpilite / unmapped mbuf / extpg / SWI taskqueue, etc.), and then use DPDK to provide NIC I/O and lcore multi-threading, so that the network stack runs as a single-process library (`libff`). |
| 24 | +- **Status quo**: the current FreeBSD upstream version that F-Stack aligns with is **13.0-RELEASE-p2** (verified via `/data/workspace/freebsd-src-releng-13.0/sys/conf/newvers.sh`). |
| 25 | +- **Driving force**: FreeBSD 15.0-RELEASE was officially released in 2025 (`REVISION=15.0 BRANCH=RELEASE-p9`), spanning 6 versions in between (14.0/14.1/14.2/14.3/14.4). Many P0-grade KBI/KPI breakages happened between the two versions (pr_usrreqs merge / inpcb migrated to SMR / if_t opaque-ization / mbuf field adjustments / netlink added / mips removed / RACK as default), and at the same time the clang/llvm toolchain was upgraded from 11.x to 19.x. Staying on 13.0 will increasingly diverge from upstream security/performance/driver support. |
| 26 | + |
| 27 | +--- |
| 28 | + |
| 29 | +## 3. Scope Boundary |
| 30 | + |
| 31 | +### 3.1 IN-SCOPE (covered by this Spec series) |
| 32 | + |
| 33 | +| Scope | Measured size | |
| 34 | +|---|---| |
| 35 | +| `f-stack/freebsd/` (kernel-stack carved subset) | 25 top-level subdirectories; **102 file diffs** vs the 13.0 baseline backup, with ~50 files actually adapted | |
| 36 | +| `f-stack/tools/` (user-space tools, F-Stack-ized) | 22 top-level subdirectories; **163 file diffs** vs the 13.0 baseline backup | |
| 37 | +| `ff_*.c` and `ff_*.h` inside `f-stack/lib/` (glue files) | **30 `.c` + 14 `.h` = 44 files** (delineated by `FF_SRCS+FF_HOST_SRCS` in `f-stack/lib/Makefile`) | |
| 38 | + |
| 39 | +### 3.2 OUT-OF-SCOPE (NOT covered by this Spec series) |
| 40 | + |
| 41 | +| Scope | Reason for exclusion | |
| 42 | +|---|---| |
| 43 | +| Actual code migration and build verification | Spec phase only; later phases are tracked separately | |
| 44 | +| Performance benchmarks / stress tests | No binary is built; no tests are run | |
| 45 | +| Git commit / push | Only working-tree files are written | |
| 46 | +| English Spec | (Note: this is now satisfied by the English-translation pass; original v0.1 only delivered the Chinese version) | |
| 47 | +| "Capability extension" ports for FreeBSD 15.0 new capabilities (e.g. netlink port, ML-KEM post-quantum crypto, inotify, post-quantum TLS) | This pass only does "alignment", not "capability extension"; new capabilities are listed in "Future Enhancement Suggestions" only | |
| 48 | +| F-Stack `adapter/syscall/` (LD_PRELOAD ring IPC part) | Has no direct dependency on the FreeBSD kernel upgrade; covered by an independent spec series `docs/ld_preload_ring_spec/` | |
| 49 | + |
| 50 | +--- |
| 51 | + |
| 52 | +## 4. Three Source Roots (verified) |
| 53 | + |
| 54 | +| Path | Role | Version | |
| 55 | +|---|---|---| |
| 56 | +| `/data/workspace/freebsd-src-releng-13.0/` | Community FreeBSD 13.0 full source | `REVISION=13.0 BRANCH=RELEASE-p2` | |
| 57 | +| `/data/workspace/freebsd-src-releng-15.0/` | Community FreeBSD 15.0 full source | `REVISION=15.0 BRANCH=RELEASE-p9` | |
| 58 | +| `/data/workspace/f-stack/` | F-Stack project root | Adapted from freebsd-13.0 | |
| 59 | + |
| 60 | +### 4.1 13.0 original backup (already exists, read-only baseline) |
| 61 | + |
| 62 | +`/data/workspace/freebsd-src-releng-13.0/f-stack-lib/` |
| 63 | +- `f-stack-lib/freebsd/` — 25 subdirectories, mirroring F-Stack `freebsd/` top-level |
| 64 | +- `f-stack-lib/tools/` — 22 subdirectories, mirroring F-Stack `tools/` top-level |
| 65 | + |
| 66 | +### 4.2 15.0 original backup (created in Phase 1.4) |
| 67 | + |
| 68 | +`/data/workspace/freebsd-src-releng-15.0/f-stack-lib/` |
| 69 | +- `f-stack-lib/freebsd/` — 24 subdirectories + Makefile (23 freebsd-native + new netlink, missing mips) |
| 70 | +- `f-stack-lib/tools/` — 22 subdirectories (12 freebsd-native + 3 f-stack-shipped tool placeholders + 2 f-stack-lib helper directories + 5 helper files) |
| 71 | +- `f-stack-lib/INVENTORY.md` — full inventory |
| 72 | + |
| 73 | +--- |
| 74 | + |
| 75 | +## 5. Glossary |
| 76 | + |
| 77 | +| Term | Chinese | Explanation | |
| 78 | +|---|---|---| |
| 79 | +| **F-Stack** | F-Stack | Tencent open-source high-performance networking framework that runs the FreeBSD network stack in DPDK user-space | |
| 80 | +| **`f-stack-lib/`** | 13.0/15.0 original backup | Unmodified subset of freebsd-src copied at F-Stack initialization; serves as the baseline for diff and upgrade | |
| 81 | +| **`ff_*.c` / `ff_*.h`** | F-Stack glue files | F-Stack-implemented wrappers that replace host kernel facilities (44 files in total) | |
| 82 | +| **`FSTACK-stub`** | F-Stack short-circuit stub | Adaptation pattern: wrap the function body with `#ifndef FSTACK` to short-circuit to an empty implementation | |
| 83 | +| **`FSTACK-altimpl`** | F-Stack alternate implementation | Adaptation pattern: `#ifdef FSTACK ... #else` switches to a different implementation | |
| 84 | +| **`IPC-replace`** | IPC takeover | tools-side adaptation pattern: replace raw socket / sysctl with `ff_ipc_*` to talk to the f-stack instance | |
| 85 | +| **`FSTACK_ZC_SEND`** | F-Stack zero-copy send | A zero-copy send-path macro added by F-Stack in `uipc_mbuf.c` | |
| 86 | +| **DPDK** | Data Plane Development Kit | Intel open-source user-space network driver framework, F-Stack's runtime base | |
| 87 | +| **KPI** | Kernel Programming Interface | Source-level kernel API (function signatures, macros) | |
| 88 | +| **KBI** | Kernel Binary Interface | Binary-level kernel API (struct layout, syscall table) | |
| 89 | +| **VNET** | Virtualized Network Stack | FreeBSD's network-stack virtualization mechanism | |
| 90 | +| **SMR** | Safe Memory Reclamation | Mechanism used by FreeBSD 14/15 to replace some epoch scenarios | |
| 91 | +| **`pr_usrreqs`** | Protocol user-request vector table | 13.0-era socket protocol interface; 15.0 has merged it into `struct protosw` | |
| 92 | +| **`if_t`** | ifnet opaque access handle | 13.0 already has `typedef struct ifnet * if_t;` in `sys/net/if_var.h` but kernel APIs still expose `struct ifnet *`; 15.0 lifts the typedef up to `sys/net/if.h` (still `typedef struct ifnet *if_t`, **not** `void *`), changes kernel APIs such as `if_alloc()` to use `if_t`, and provides `if_get*/if_set*` accessors. "Opaque-ization" is API-contract semantics: external code should operate via accessors and should not depend on field layout. See `03-freebsd-15-changes.md §3.3`. | |
| 93 | +| **inpcbgroup** | inpcb group hash | 13.0 protected by epoch; 15.0 switched to SMR | |
| 94 | +| **netlink (FreeBSD)** | netlink-compatible subsystem | New in 15.0 (actually introduced in 14.0); a Linux netlink compatibility layer at sys/netlink/ | |
| 95 | +| **RACK** | Recent ACKnowledgment | TCP retransmission stack default-enabled in FreeBSD 14/15 | |
| 96 | +| **mips** | MIPS architecture | Removed from base in 14.0; affects F-Stack `freebsd/mips/` subdirectory | |
| 97 | +| **`__FreeBSD_version`** | FreeBSD version macro | 13.0 = `1300139`; 15.0 = `1500068` | |
| 98 | +| **`SYS_MAXSYSCALL`** | Highest syscall number | 13.0 = 580; 15.0 = 599 (13→15 net additions: 22 + deletions: 3; representative additions: `fspacectl`, `kqueuex`, `membarrier`, `timerfd_create/gettime/settime`, `inotify_add_watch_at`, `inotify_rm_watch`, `jail_remove_jd`, etc.; deletions: `gssd_syscall`, `sbrk`, `sstk`. Full list in `03-freebsd-15-changes.md` §2.4) | |
| 99 | +| **pkgbase** | base-system packaging | An optional 15.0 release form, unrelated to F-Stack | |
| 100 | +| **`m_pkthdr` / `m_ext`** | mbuf header / external storage pointer | Field adjustments between 14→15 | |
| 101 | +| **`mb_unmapped_*` / `pcpu_page_alloc`** | unmapped mbuf path | Host VM dependencies that F-Stack masks out | |
| 102 | +| **`kpilite.h`** | KPI lite header | Masked by F-Stack to avoid dependency on host module subsystem | |
| 103 | +| **`libff`** | F-Stack user-space library | Static library compiled from `ff_*.c` plus the linked-in freebsd `*_SRCS` | |
| 104 | +| **`KERN_SRCS` / `NET_SRCS` / `NETINET_SRCS` ...** | F-Stack Makefile link lists | Source-file lists in `f-stack/lib/Makefile` deciding "which freebsd source files actually participate in libff compilation" | |
| 105 | +| **Milestones M1-M5** | M1=infrastructure / M2=kern / M3=network stack / M4=edge subsystems / M5=tools+ff_* | See `05-implementation-plan.md` | |
| 106 | +| **P0 / P1 / P2 / P3** | Risk / priority tags | P0 must-fix blocking compile/run; P1 compiles but semantics need verification; P2 non-critical path; P3 informational | |
| 107 | + |
| 108 | +--- |
| 109 | + |
| 110 | +## 6. Relationship with Existing F-Stack Documents |
| 111 | + |
| 112 | +| Existing doc | How this Spec series reuses it | |
| 113 | +|---|---| |
| 114 | +| `docs/01-LAYER1-ARCHITECTURE.md` | Reused for source-tree panorama (in `02-architecture-analysis.md`) | |
| 115 | +| `docs/F-Stack_Architecture_Layer1_System_Overview.md` | Reused for system-layer architecture diagram (same as above) | |
| 116 | +| `docs/F-Stack_Architecture_Layer2_Interface_Specification.md` | Reused for interface inventory (same as above) | |
| 117 | +| `docs/KNOWLEDGE_GRAPH_WIKI.md` | Reused for dependency graph (same as above) | |
| 118 | +| `docs/ld_preload_ring_spec/` | **Not reused**: the LD_PRELOAD ring IPC is an independent workstream | |
| 119 | +| `adapter/syscall/README.md` | **Not directly reused**: but `ff_syscall_wrapper.c` is the intersection point and is referenced in `02-architecture-analysis.md` §3 | |
| 120 | + |
| 121 | +--- |
| 122 | + |
| 123 | +## 7. 8 Core Decisions (locked, confirmed via multi-round q&a in `plan.md`) |
| 124 | + |
| 125 | +| # | Decision | Outcome | Locked-in document | |
| 126 | +|---|---|---|---| |
| 127 | +| 1 | Spec output directory | `f-stack/docs/freebsd_13_to_15_upgrade_spec/zh_cn/` (English version added later, flat under parent) | plan.md §0 | |
| 128 | +| 2 | Coverage scope | Full f-stack/freebsd + full f-stack/tools + 30 .c + 14 .h under f-stack/lib | plan.md §1.5 | |
| 129 | +| 3 | Risk-record granularity | Full-set major changes for 13→14 plus 14→15 | plan.md §4 | |
| 130 | +| 4 | Execution mode | Hybrid Agent Team: Leader in main dialogue + 3 code-explorer subagents in parallel | plan.md §2 | |
| 131 | +| 5 | Disposition of mips/ | Skipped; not pulled into the 15.0 backup; the 04-diff phase plans to delete `f-stack/freebsd/mips/` | INVENTORY.md / plan.md Step 1.4 | |
| 132 | +| 6 | Disposition of netlink/ | Pulled into the 15.0 backup; spec DP-2 decides **not to introduce it into** `f-stack/freebsd/` for now (alignment only, no capability extension) | INVENTORY.md / plan.md §4.2 DP-2 | |
| 133 | +| 7 | Disposition of f-stack-shipped tools (knictl/traffic/top) | Placeholder-copied from the 13.0 backup; their upgrade is tracked under the dedicated milestone M5 | INVENTORY.md | |
| 134 | +| 8 | Copy strategy | `cp -a` preserves mtime / permissions / symlinks | INVENTORY.md | |
| 135 | + |
| 136 | +--- |
| 137 | + |
| 138 | +## 8. Reading-Order Suggestions |
| 139 | + |
| 140 | +| Role | Order | |
| 141 | +|---|---| |
| 142 | +| Project manager / architecture reviewer | 00 → 01 → 05 → 06 → 99 | |
| 143 | +| Implementation engineer | 00 → 03 → 04 → 05 → 02 | |
| 144 | +| Downstream AI agents (task pickup) | 04 → 05 → 02 → 06 | |
| 145 | +| Risk auditor | 01 → 03 → 99 | |
| 146 | + |
| 147 | +--- |
| 148 | + |
| 149 | +## 9. Document Metadata |
| 150 | + |
| 151 | +- **Author**: Agent Team Leader (executed inside the main dialogue) |
| 152 | +- **Collaboration**: Sub-Agents A/B/C (completed via code-explorer subagents) |
| 153 | +- **Review**: Reviewer (issued `99-review-report.md` on 2026-05-26; supplemented independent audit v0.2 `98-independent-audit-report.md` and 6 must-fix revisions on 2026-05-28) |
| 154 | +- **Verification**: all numeric figures come from measured command output; non-measured inferences are tagged with their source in the "7. 8 Core Decisions" table at the end |
| 155 | +- **Next step**: read `01-requirements-spec.md` to learn what specific problems this upgrade solves / does not solve |
0 commit comments