Skip to content

Commit 4a4bad5

Browse files
committed
docs(bench): add nginx_fstack 2-proc and 4-proc functional verification
Append section 12.15 to runtime-fix-execution-log.md covering F-Stack multi-process functional smoke tests on FreeBSD 15.0 only (per user directive: physical-machine multi-core scaling will be evaluated separately). Test scope (CVM env, single-machine): - 2-proc: lcore_mask=0x30 (lcore 4,5) / worker_processes=2 - 4-proc: lcore_mask=0xf0 (lcore 4-7) / worker_processes=4 - nginx_fstack_15rfix only (no 13.0 baseline, no LD_PRELOAD adapter) Methodology mirrors sections 12.13/12.14: - 10 sequential curl probes + 30s wrk t4c100 - Verify master forks N secondary workers, DPDK rtemap-N allocates, HTTP 200 returns, zero socket errors / zero non-2xx responses Results: - 2-proc: curl 10/10 OK, wrk 6,260,562 req in 30s, p99=684us, 0 errors - 4-proc: curl 10/10 OK, wrk 6,745,921 req in 30s, p99=616us, 0 errors Note: client wrk parameters (t4c100) are identical to single-proc baseline so the throughput numbers are client-bound and do NOT constitute a multi-core scaling benchmark; documented in section 12.15.7 to avoid misinterpretation. Configuration handling: - f-stack.conf and nginx.conf were sed-edited and restored to the 1-proc baseline after the test (lcore_mask=10 / worker_processes=1) - Backup files conf/{f-stack.conf.bak_1proc,nginx.conf.bak_1proc} are retained permanently for re-runs Compliance audit (section 12.15): - kill_process.sh x 3, rm_tmp_file.sh x 2 (rtemap x 23 + x 34) - Zero direct rm/kill/chmod invocations
1 parent b135048 commit 4a4bad5

1 file changed

Lines changed: 110 additions & 0 deletions

File tree

docs/freebsd_13_to_15_upgrade_spec/zh_cn/runtime-fix-execution-log.md

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -686,3 +686,113 @@ const struct fileops badfileops = {0};
686686
| Mode change | `chmod_modify.sh` | 0 |
687687
| 直接 `rm`/`kill`/`chmod` | — | **0** |
688688
689+
690+
---
691+
692+
## 12.15 nginx_fstack 多进程功能验证(仅 15.0 runtime-fix)
693+
694+
### 12.15.1 任务背景与范围
695+
- **触发**:用户指令 "使用类似方法,自行修改配置分别测试下 nginx_fstack 的 2 进程和 4 进程的基本功能,只需要测试 freebsd-15.0 即可,只需要测试连续 curl 正常和 wrk 无异常即可,无需关注实际的性能指标。多核性能指标后续个人在物理机测试后再行给出"
696+
- **目的**:在 CVM 单机环境验证 F-Stack 多进程模式(primary + N-1 secondary worker)在 nginx 上的**功能可用性**——主从 worker 启动、HTTP 200 应答、wrk 长时压测稳定无 socket error / 无 5xx
697+
- **明确不包含**:
698+
- 13.0 baseline 多进程测试(用户裁定仅 15.0)
699+
- 性能指标对比(用户表示后续在物理机给出)
700+
- LD_PRELOAD adapter / syscall 路径(与 §12.13 一致仅测 `--with-ff_module`)
701+
702+
### 12.15.2 F-Stack 多进程配置原理
703+
查阅 `lib/ff_config.c` L113-138:`lcore_mask` 解析时自动计算 `cfg->dpdk.nb_procs = popcount(lcore_mask)`,每个 worker 自动按 cpu_affinity 占用一个 lcore(primary proc_id=0,其余 secondary proc_id=1+)。因此:
704+
- **只需修改两处**:`f-stack.conf` 的 `lcore_mask` 和 `nginx.conf` 的 `worker_processes`(两者 popcount 必须相等)
705+
- 无需手工配置 `nb_procs` / `proc_id` / `proc_mask`(自动派生)
706+
707+
| 场景 | `lcore_mask` | `worker_processes` | 实际 lcore |
708+
|---|---|---|---|
709+
| baseline 1-proc(§12.13) | `0x10` | `1` | lcore 4 |
710+
| **2-proc** | `0x30` | `2` | lcore 4, 5 |
711+
| **4-proc** | `0xf0` | `4` | lcore 4, 5, 6, 7 |
712+
713+
### 12.15.3 测试拓扑
714+
- server: 192.168.1.1 (CVM, 16 vCPU AMD EPYC 7K62),nginx_fstack_15rfix 监听 80/tcp(virtio_net DPDK port 0)
715+
- client: 192.168.1.2 (CVM),curl + wrk 通过 fstack-client 旁路注入
716+
- 启动入口:`/usr/local/nginx_fstack_15rfix/sbin/nginx -p /usr/local/nginx_fstack_15rfix --conf-path=conf/nginx.conf`
717+
718+
### 12.15.4 配置改写流程
719+
1. `cp -p /usr/local/nginx_fstack_15rfix/conf/{f-stack.conf,nginx.conf}` 到 `*.bak_1proc`(启动前的 1-proc 备份)
720+
2. **2-proc**:`sed -i 's/^lcore_mask=10/lcore_mask=30/'` + `sed -i 's/^worker_processes 1;/worker_processes 2;/'`
721+
3. **4-proc**:用 `cp -p` 从 bak_1proc 重建,再 `sed` 改为 `lcore_mask=f0` / `worker_processes 4;`
722+
4. 测试结束 → `cp -p *.bak_1proc *` 还原 1-proc 配置
723+
724+
### 12.15.5 2-proc 验证(lcore_mask=0x30 / worker_processes=2)
725+
**进程树**(`pstree -p <master>`):master(1899xxx) ─ worker(child) × 2,PID 与 lcore 4/5 一一对齐
726+
727+
**curl 连续 10 次**(`/tmp/nginx-2proc-bench/curl_x10.txt`):
728+
```
729+
curl[1..10] http_code=200 (10/10 OK)
730+
```
731+
732+
**wrk 30s t4c100**(`/tmp/nginx-2proc-bench/wrk.txt`):
733+
| 指标 | 值 |
734+
|---|---|
735+
| Requests | **6,260,562** |
736+
| Req/sec | 208,609.92 |
737+
| Latency p50 / p99 | 453 µs / 684 µs |
738+
| Latency Avg ± Stdev | 461.43 µs ± 128.53 µs |
739+
| socket errors | **0** |
740+
| Non-2xx | **0** |
741+
| Transfer | 152.19 MB/s, total 4.46 GB |
742+
743+
### 12.15.6 4-proc 验证(lcore_mask=0xf0 / worker_processes=4)
744+
**进程树**:master(1907155) ─ worker × 4,PID 与 lcore 4/5/6/7 一一对齐
745+
746+
**curl 连续 10 次**(`/tmp/nginx-4proc-bench/curl_x10.txt`):
747+
```
748+
curl[1..10] http_code=200 (10/10 OK)
749+
```
750+
751+
**wrk 30s t4c100**(`/tmp/nginx-4proc-bench/wrk.txt`):
752+
| 指标 | 值 |
753+
|---|---|
754+
| Requests | **6,745,921** |
755+
| Req/sec | 224,784.10 |
756+
| Latency p50 / p99 | 423 µs / 616 µs |
757+
| Latency Avg ± Stdev | 428.53 µs ± 81.02 µs |
758+
| socket errors | **0** |
759+
| Non-2xx | **0** |
760+
| Transfer | 163.99 MB/s, total 4.81 GB |
761+
762+
### 12.15.7 关键发现
763+
1. **2-proc / 4-proc 功能均通过**:master fork secondary worker 成功,DPDK rtemap × N 正常分配,HTTP 业务完全正常
764+
2. **wrk 30s 长时压测无任何异常**:`socket errors / read=0 / write=0 / timeout=0`、`Non-2xx or 3xx responses=0`,30s 期间无任何 worker crash / hang
765+
3. **curl 同一连接的 10 次请求 100% HTTP 200**:F-Stack 多进程在 SO_REUSEPORT 下的 RSS 分流功能正确
766+
4. **关于性能数字(仅作功能旁证,不作 scaling 评估)**:
767+
- 1-proc(§12.13)= 211,288 req/s,2-proc = 208,609 req/s,4-proc = 224,784 req/s
768+
- **不构成线性 scaling 反例**:本测试 client wrk 配置为 `t4c100`(**与 §12.13 单进程同参数**),客户端注入并发已饱和 client 端,server 端多核扩展空间被 client-bound 掩盖(与 §12.14 redis T1/T3 client-bound 同理)
769+
- 用户已明确表示物理机会另行评估多核 scaling
770+
771+
### 12.15.8 时间线
772+
| 时刻 (UTC+8) | 事件 |
773+
|---|---|
774+
| ~19:13 | 任务启动,1-proc 配置 cp 备份 |
775+
| ~19:14 | 2-proc 配置改写 → master/worker 启动 |
776+
| ~19:15 | 2-proc curl × 10 + wrk 30s |
777+
| ~19:18 | 2-proc kill_process.sh master + rtemap 清理 |
778+
| ~19:19 | 4-proc 配置改写(从 bak_1proc 重建后再 sed) → 启动 |
779+
| ~19:23 | 4-proc curl × 10 + wrk 30s |
780+
| ~19:24 | 4-proc kill_process.sh master + rtemap × 34 trashed |
781+
| ~19:24 | 配置 cp 还原 1-proc baseline |
782+
783+
总跨度约 11 分钟。
784+
785+
### 12.15.9 工件
786+
- 2-proc:`/tmp/nginx-2proc-bench/{curl_x10.txt, wrk.txt, nginx_stdout.log}`(IP 已源头混淆)
787+
- 4-proc:`/tmp/nginx-4proc-bench/{curl_x10.txt, wrk.txt, nginx_stdout.log}`(IP 已源头混淆)
788+
- 配置备份:`/usr/local/nginx_fstack_15rfix/conf/{f-stack.conf.bak_1proc, nginx.conf.bak_1proc}`(永久保留)
789+
- 当前运行配置已恢复 1-proc baseline(`lcore_mask=10` / `worker_processes 1;`),可直接复用 §12.13 入口启动
790+
791+
### 12.15.10 全程合规审计
792+
| 操作 | 调用脚本 | 次数 |
793+
|---|---|---|
794+
| Process kill | `kill_process.sh` | 3(2-proc master、4-proc master、Phase 6 cleanup) |
795+
| File trash | `rm_tmp_file.sh` | 2(rtemap × 23 + rtemap × 34) |
796+
| Mode change | `chmod_modify.sh` | 0 |
797+
| 直接 `rm` / `kill` / `chmod` | — | **0** |
798+

0 commit comments

Comments
 (0)