Skip to content

Commit e3dbdb7

Browse files
committed
docs: add propagation performance report and mesh test workflow
1 parent 4527711 commit e3dbdb7

5 files changed

Lines changed: 363 additions & 0 deletions

File tree

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -295,10 +295,15 @@ just test-serve https://myapp.exe.dev
295295

296296
```bash
297297
just test
298+
299+
# 実機メッシュ検証(3ノードをローカル起動して伝搬確認)
300+
just test-multi-relay-mesh 19081
298301
```
299302

300303
## Benchmark
301304

305+
レポートは [docs/performance-reports.md](./docs/performance-reports.md) を参照。
306+
302307
```bash
303308
# 全シナリオ
304309
just bench https://bit-relay.mizchi.workers.dev

docs/performance-reports.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Performance Reports
2+
3+
bit-relay の性能検証結果を時系列で集約するドキュメント。
4+
5+
## 1. Cloudflare スケーリング計測(2026-02-22)
6+
7+
詳細レポート: [scaling.md](./scaling.md)
8+
9+
- 対象: `bit-relay.mizchi.workers.dev` (Cloudflare Workers + Durable Objects)
10+
- ツール: k6 v1.5.0
11+
- 署名検証: `RELAY_REQUIRE_SIGNATURE=false`
12+
13+
代表値(抜粋):
14+
15+
- Health (`GET /health`, `GET /`) は `500 VUs``p95=165ms`、エラー率 `0%`
16+
- Publish+Poll(分散 room)は `500 VUs``publish p95=387ms`, `poll p95=36ms`
17+
- WebSocket は `100 VUs``ready p95=328ms`, `ping/pong p95=27ms`
18+
19+
## 2. ローカル 3 ノード伝搬速度(2026-02-24)
20+
21+
### 計測条件
22+
23+
- ノード構成: `relay-a/b/c` の 3 ノード
24+
- 接続: 各ノードの `RELAY_PEERS` に他 2 ノードを設定
25+
- 同期間隔: `RELAY_PEER_SYNC_INTERVAL_SEC=1`
26+
- 署名検証: `RELAY_REQUIRE_SIGNATURE=false`
27+
- 試行回数: `notify``issue_sync` をそれぞれ 15 回
28+
29+
### シナリオ
30+
31+
- `notify`:
32+
- 各ノードに 2 ユーザー(合計 6 ユーザー)で publish
33+
- 全ノード `poll` で 6 件揃うまでの時間を計測
34+
- `issue_sync`:
35+
- `topic=issue``topic=issue.updated` を別ノードから publish
36+
- 全ノード `cache/issues/sync``upsert + updated` が揃うまでの時間を計測
37+
38+
### 結果
39+
40+
| Metric | Trials | Timeout | Min | Avg | p50 | p95 | Max |
41+
| ------------ | ------ | ------- | ----- | ------- | ----- | ----- | ----- |
42+
| `notify` | 15 | 0 | 762ms | 861.4ms | 854ms | 958ms | 980ms |
43+
| `issue_sync` | 15 | 0 | 859ms | 933.7ms | 960ms | 977ms | 983ms |
44+
45+
### 解釈
46+
47+
- 伝搬完了時間はほぼ 1 秒前後で、`RELAY_PEER_SYNC_INTERVAL_SEC=1` の設定に整合する。
48+
- `issue_sync``notify` よりやや遅いが、`p95` は 1 秒未満に収束した。
49+
50+
## 3. 再実行手順
51+
52+
機能検証(3ノードの伝搬可否):
53+
54+
```bash
55+
just test-multi-relay-mesh 19081
56+
```
57+
58+
負荷シナリオ(k6, multi-relay + cache hit/miss + issue sync):
59+
60+
```bash
61+
RELAY_URLS=https://relay-a.example,https://relay-b.example \
62+
just bench-scenario multi-relay-cache-issue-sync https://relay-a.example
63+
```

docs/scaling.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
bit-relay (Cloudflare Workers + Durable Objects) のスケーリング特性を k6
44
ベンチマークで計測した結果をまとめる。
55

6+
集約版レポート: [performance-reports.md](./performance-reports.md)
7+
68
2026-02-24 に、multi-relay + issue cache sync のベンチシナリオを追加した。
79

810
## 計測環境

justfile

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,9 @@ test-claim-watch target="http://localhost:8788":
5656
test-5agents target="http://localhost:8788":
5757
tools/test-5agents-claim.sh {{target}}
5858

59+
test-multi-relay-mesh base_port="19081":
60+
tools/test-multi-relay-mesh.sh {{base_port}}
61+
5962
bench target="http://localhost:8788":
6063
k6 run --env BASE_URL={{target}} bench/run-all.js
6164

tools/test-multi-relay-mesh.sh

Lines changed: 290 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,290 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# E2E integration check: multi-relay mesh propagation
5+
#
6+
# This script starts 3 local relay nodes with peer settings, attaches
7+
# multiple users to each node, and verifies:
8+
# 1) notify topic propagation across all nodes
9+
# 2) issue topic propagation via cache/issues/sync across all nodes
10+
#
11+
# Usage:
12+
# tools/test-multi-relay-mesh.sh [base_port]
13+
#
14+
# Examples:
15+
# tools/test-multi-relay-mesh.sh
16+
# tools/test-multi-relay-mesh.sh 19081
17+
#
18+
# Environment variables:
19+
# RELAY_SYNC_INTERVAL_SEC Sync interval sec for peer worker (default: 1)
20+
# RELAY_REQUIRE_SIGNATURE Pass-through for node startup (default: false)
21+
# KEEP_NODES Keep node processes after test (1 to keep)
22+
23+
BASE_PORT="${1:-19081}"
24+
SYNC_INTERVAL_SEC="${RELAY_SYNC_INTERVAL_SEC:-1}"
25+
REQUIRE_SIGNATURE="${RELAY_REQUIRE_SIGNATURE:-false}"
26+
KEEP_NODES="${KEEP_NODES:-0}"
27+
28+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
29+
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
30+
TMP_DIR="$(mktemp -d "/tmp/bit-relay-mesh.XXXXXX")"
31+
32+
NODES=("relay-a" "relay-b" "relay-c")
33+
PORTS=("${BASE_PORT}" "$((BASE_PORT + 1))" "$((BASE_PORT + 2))")
34+
URLS=(
35+
"http://127.0.0.1:${PORTS[0]}"
36+
"http://127.0.0.1:${PORTS[1]}"
37+
"http://127.0.0.1:${PORTS[2]}"
38+
)
39+
PIDS=()
40+
41+
pass=0
42+
fail=0
43+
44+
ok() {
45+
pass=$((pass + 1))
46+
echo " [OK] $1"
47+
}
48+
49+
ng() {
50+
fail=$((fail + 1))
51+
echo " [NG] $1" >&2
52+
}
53+
54+
require_cmd() {
55+
if ! command -v "$1" >/dev/null 2>&1; then
56+
echo "missing required command: $1" >&2
57+
exit 1
58+
fi
59+
}
60+
61+
uri_encode() {
62+
jq -rn --arg v "$1" '$v | @uri'
63+
}
64+
65+
cleanup() {
66+
if [ "${KEEP_NODES}" = "1" ]; then
67+
echo "KEEP_NODES=1: node processes are kept running."
68+
echo " logs: ${TMP_DIR}"
69+
return
70+
fi
71+
for pid in "${PIDS[@]}"; do
72+
kill "${pid}" >/dev/null 2>&1 || true
73+
wait "${pid}" >/dev/null 2>&1 || true
74+
done
75+
}
76+
trap cleanup EXIT
77+
78+
check_port_free() {
79+
local port="$1"
80+
if lsof -iTCP:"${port}" -sTCP:LISTEN -n -P >/dev/null 2>&1; then
81+
echo "port already in use: ${port}" >&2
82+
exit 1
83+
fi
84+
}
85+
86+
start_node() {
87+
local idx="$1"
88+
local node_id="${NODES[$idx]}"
89+
local port="${PORTS[$idx]}"
90+
local peer_urls=()
91+
local j
92+
for j in "${!NODES[@]}"; do
93+
if [ "${j}" -eq "${idx}" ]; then
94+
continue
95+
fi
96+
peer_urls+=("${URLS[$j]}")
97+
done
98+
local peers_csv
99+
peers_csv="$(IFS=,; echo "${peer_urls[*]}")"
100+
101+
check_port_free "${port}"
102+
103+
echo "start ${node_id} at ${URLS[$idx]} peers=${peers_csv}"
104+
(
105+
cd "${ROOT_DIR}"
106+
HOST=127.0.0.1 \
107+
PORT="${port}" \
108+
RELAY_NODE_ID="${node_id}" \
109+
RELAY_PEERS="${peers_csv}" \
110+
RELAY_PEER_SYNC_INTERVAL_SEC="${SYNC_INTERVAL_SEC}" \
111+
RELAY_REQUIRE_SIGNATURE="${REQUIRE_SIGNATURE}" \
112+
deno run --allow-net --allow-env src/deno_main.ts \
113+
>"${TMP_DIR}/${node_id}.log" 2>&1
114+
) &
115+
PIDS+=("$!")
116+
}
117+
118+
wait_for_health() {
119+
local base_url="$1"
120+
local label="$2"
121+
local attempt
122+
for attempt in $(seq 1 40); do
123+
local body
124+
body="$(curl -fsS "${base_url}/health" 2>/dev/null || true)"
125+
if echo "${body}" | jq -e '.status == "ok"' >/dev/null 2>&1; then
126+
ok "${label} health is ok"
127+
return 0
128+
fi
129+
sleep 0.25
130+
done
131+
ng "${label} failed to become healthy"
132+
echo "---- ${label} log ----" >&2
133+
cat "${TMP_DIR}/${label}.log" >&2 || true
134+
return 1
135+
}
136+
137+
publish_event() {
138+
local base_url="$1"
139+
local room="$2"
140+
local sender="$3"
141+
local msg_id="$4"
142+
local topic="$5"
143+
local payload="$6"
144+
145+
local url="${base_url}/api/v1/publish?room=$(uri_encode "${room}")&sender=$(uri_encode "${sender}")&id=$(uri_encode "${msg_id}")&topic=$(uri_encode "${topic}")"
146+
local body
147+
body="$(curl -fsS -X POST "${url}" -H 'content-type: application/json' -d "${payload}")"
148+
local accepted
149+
accepted="$(echo "${body}" | jq -r '.accepted')"
150+
if [ "${accepted}" != "true" ]; then
151+
ng "publish rejected sender=${sender} id=${msg_id}: ${body}"
152+
return 1
153+
fi
154+
ok "publish accepted sender=${sender} id=${msg_id}"
155+
}
156+
157+
verify_notify_propagation() {
158+
local room="$1"
159+
shift
160+
local expected_ids=("$@")
161+
local expected_json
162+
expected_json="$(printf '%s\n' "${expected_ids[@]}" | jq -R -s -c 'split("\n")[:-1] | sort')"
163+
164+
local attempt
165+
for attempt in $(seq 1 20); do
166+
local all_ok=1
167+
echo "notify propagation attempt=${attempt}"
168+
local idx
169+
for idx in "${!URLS[@]}"; do
170+
local base_url="${URLS[$idx]}"
171+
local body
172+
body="$(curl -fsS "${base_url}/api/v1/poll?room=$(uri_encode "${room}")&after=0&limit=100")"
173+
local count
174+
count="$(echo "${body}" | jq '.envelopes | length')"
175+
local ids_json
176+
ids_json="$(echo "${body}" | jq -c '[.envelopes[].id] | sort')"
177+
local ids_match
178+
ids_match="$(jq -n --argjson a "${ids_json}" --argjson b "${expected_json}" '$a == $b')"
179+
echo " ${base_url} count=${count} ids_match=${ids_match}"
180+
if [ "${ids_match}" != "true" ]; then
181+
all_ok=0
182+
fi
183+
done
184+
if [ "${all_ok}" -eq 1 ]; then
185+
ok "notify topic propagated to all nodes"
186+
return 0
187+
fi
188+
sleep 1
189+
done
190+
ng "notify propagation did not converge"
191+
return 1
192+
}
193+
194+
verify_issue_sync_propagation() {
195+
local room="$1"
196+
local attempt
197+
for attempt in $(seq 1 20); do
198+
local all_ok=1
199+
echo "issue sync propagation attempt=${attempt}"
200+
local idx
201+
for idx in "${!URLS[@]}"; do
202+
local base_url="${URLS[$idx]}"
203+
local body
204+
body="$(curl -fsS "${base_url}/api/v1/cache/issues/sync?room=$(uri_encode "${room}")&after=0&limit=20")"
205+
local event_count
206+
event_count="$(echo "${body}" | jq '.events | length')"
207+
local snapshot_count
208+
snapshot_count="$(echo "${body}" | jq '.snapshots | length')"
209+
local has_upsert
210+
has_upsert="$(echo "${body}" | jq '[.events[].action] | index("upsert") != null')"
211+
local has_updated
212+
has_updated="$(echo "${body}" | jq '[.events[].action] | index("updated") != null')"
213+
echo " ${base_url} events=${event_count} snapshots=${snapshot_count} upsert=${has_upsert} updated=${has_updated}"
214+
if [ "${event_count}" -lt 2 ] || [ "${has_upsert}" != "true" ] || [ "${has_updated}" != "true" ]; then
215+
all_ok=0
216+
fi
217+
done
218+
if [ "${all_ok}" -eq 1 ]; then
219+
ok "issue sync propagated to all nodes"
220+
return 0
221+
fi
222+
sleep 1
223+
done
224+
ng "issue sync propagation did not converge"
225+
return 1
226+
}
227+
228+
main() {
229+
require_cmd deno
230+
require_cmd curl
231+
require_cmd jq
232+
require_cmd lsof
233+
234+
echo "=== multi-relay mesh propagation test ==="
235+
echo "base_port=${BASE_PORT} sync_interval=${SYNC_INTERVAL_SEC}s require_signature=${REQUIRE_SIGNATURE}"
236+
echo "tmp_dir=${TMP_DIR}"
237+
238+
local idx
239+
for idx in "${!NODES[@]}"; do
240+
start_node "${idx}"
241+
done
242+
243+
for idx in "${!NODES[@]}"; do
244+
wait_for_health "${URLS[$idx]}" "${NODES[$idx]}"
245+
done
246+
247+
local notify_room="mesh-demo-$(date +%s)"
248+
local expected_ids=()
249+
local rows=(
250+
"0 alice-a1 a-msg-1"
251+
"0 alice-a2 a-msg-2"
252+
"1 bob-b1 b-msg-1"
253+
"1 bob-b2 b-msg-2"
254+
"2 carol-c1 c-msg-1"
255+
"2 carol-c2 c-msg-2"
256+
)
257+
258+
echo "=== publish notify events room=${notify_room} ==="
259+
local row
260+
for row in "${rows[@]}"; do
261+
local node_idx sender id_prefix msg_id
262+
read -r node_idx sender id_prefix <<<"${row}"
263+
msg_id="${id_prefix}-${notify_room}"
264+
publish_event "${URLS[$node_idx]}" "${notify_room}" "${sender}" "${msg_id}" "notify" \
265+
"{\"from\":\"${sender}\",\"message\":\"hello from ${sender}\"}"
266+
expected_ids+=("${msg_id}")
267+
done
268+
269+
verify_notify_propagation "${notify_room}" "${expected_ids[@]}"
270+
271+
local issue_room="mesh-issue-$(date +%s)"
272+
echo "=== publish issue events room=${issue_room} ==="
273+
publish_event "${URLS[0]}" "${issue_room}" "alice-a1" "issue-create-${issue_room}" "issue" \
274+
'{"issue_id":"issue-1","title":"initial"}'
275+
publish_event "${URLS[1]}" "${issue_room}" "bob-b1" "issue-update-${issue_room}" "issue.updated" \
276+
'{"issue_id":"issue-1","title":"updated"}'
277+
278+
verify_issue_sync_propagation "${issue_room}"
279+
280+
echo ""
281+
echo "=== Results: ${pass} passed, ${fail} failed ==="
282+
echo "notify_room=${notify_room}"
283+
echo "issue_room=${issue_room}"
284+
echo "logs=${TMP_DIR}"
285+
if [ "${fail}" -gt 0 ]; then
286+
exit 1
287+
fi
288+
}
289+
290+
main "$@"

0 commit comments

Comments
 (0)