Skip to content

Commit 61655d5

Browse files
MarcoPolojxssukunrt
authored
Test Partial Message in interop tester (#684)
* Test Partial Message in interop tester adds Go implementation of partial messages and changes to the test runner to support it. * add rust implementation for partial messages * remove instructions type field * fix PublishPartial group_id rename * gossipsub-interop: partial messages, special case 2 nodes * fixup add partial to gossipsub interop subscript tester * workaround for rust bug * gossipsub-interop(rust-libp2p): fix instruction parsing * gossipsub-interop(rust-libp2p): Fix cargo warning * gossipsub-interop(rust-libp2p): fix bitmap's available/missing methods * gossipsub-interop(rust-libp2p): implement republishing logic * debug print lines go-libp2p debugging lines debug print rust * update rust version to latest rust-libp2p * update rust-libp2p dep * update for latest rust-libp2p changes * update rust-libp2p sim to the updated scriptparams * gossipsub-interop: lint fixes * gossipsub-intero(go): update to latest partial message impl * gossipsub-interop: update go-libp2p to the latest version * gossipsub-interop: bump go-libp2p version * add missing MergeMedata func * properly return err on nested instruction * add partial_messages check * gossipsub-interop: add a new scenario for partial messages We create a chain topology 1 <-> 2 <-> ... <-> n. Provide each node with one part of the message. And then publish the message from one of the nodes. Eventually all the nodes should have all the messages. * gossipsub-interop: add partial message fanout scenario test * log "All parts received" only once per group id * gossipsub-interop: Add a basic `make test` command to run all tests * gossipsub-interop: Change rust log to match checker * Update rust-libp2p * update rust-libp2p * fix(rust): handle partial messages correctly in fanout scenario - treat None metadata as peer having nothing, send all available parts - create topic/group entries on-the-fly when receiving unknown partial messages instead of error'ing - log "All parts received" when local node has complete message * fix count occurences check * Expand makefile to add rust-go interop tests * avoid simultaneous connections * only connect to a node once in random mesh * echo command in make test * Add CI to gossipsub-interop tests * use jxs' branch of rust-libp2p * update to latest libp2p changes * fix typos on doc * Update go gossipsub library --------- Co-authored-by: João Oliveira <hello@jxs.pt> Co-authored-by: sukun <sukunrt@gmail.com>
1 parent 4c97236 commit 61655d5

23 files changed

Lines changed: 2021 additions & 880 deletions
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
name: Gossipsub Interoperability Tests (PR)
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- 'gossipsub-interop/**'
7+
- '.github/workflows/gossipsub-interop-pr.yml'
8+
workflow_dispatch:
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
jobs:
15+
test:
16+
runs-on: ubuntu-22.04
17+
steps:
18+
- name: Checkout code
19+
uses: actions/checkout@v4
20+
21+
- name: Set up Go
22+
uses: actions/setup-go@v5
23+
with:
24+
go-version-file: gossipsub-interop/go-libp2p/go.mod
25+
cache: true
26+
27+
- name: Set up Rust
28+
uses: dtolnay/rust-toolchain@stable
29+
30+
- name: Install uv
31+
run: |
32+
curl -LsSf https://astral.sh/uv/install.sh | sh
33+
echo "$HOME/.local/bin" >> $GITHUB_PATH
34+
35+
- name: Install Shadow simulator
36+
run: |
37+
# Install Shadow dependencies
38+
sudo apt-get update
39+
sudo apt-get install -y \
40+
cmake \
41+
findutils \
42+
libclang-dev \
43+
libc-dbg \
44+
libglib2.0-0 \
45+
libglib2.0-dev \
46+
make \
47+
netbase \
48+
python3 \
49+
python3-networkx \
50+
xz-utils
51+
52+
# Build and install Shadow v3.3.0 from source
53+
git clone --depth 1 --branch v3.3.0 https://github.com/shadow/shadow.git shadow-simulator
54+
cd shadow-simulator
55+
git checkout v3.3.0
56+
./setup build --clean
57+
./setup install
58+
echo "$HOME/.local/bin" >> $GITHUB_PATH
59+
60+
- name: Run gossipsub interop tests
61+
working-directory: gossipsub-interop
62+
run: make test

gossipsub-interop/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
*.swp
22
/shadow.data
3+
shadow-outputs/*
4+
latest
35
synctest*.data
46
/shadow.yaml
57
/graph.gml

gossipsub-interop/Makefile

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,46 @@ binaries:
77

88
# Clean all generated shadow simulation files
99
clean:
10-
rm -rf *.data || true
10+
rm -rf shadow-outputs || true
1111
rm plots/* || true
1212

13-
.PHONY: binaries all clean
13+
test:
14+
# Testing partial messages
15+
@echo "Testing partial messages"
16+
@uv run run.py --node_count 32 --composition "rust-and-go" --scenario "partial-messages" && uv run checks/partial_messages.py latest --count 1
17+
18+
@echo "Testing partial messages chain"
19+
@uv run run.py --node_count 8 --composition "rust-and-go" --scenario "partial-messages-chain" && uv run checks/partial_messages.py latest --count 16
20+
21+
@echo "Testing fanout"
22+
uv run run.py --node_count 8 --composition "rust-and-go" --scenario "partial-messages-fanout" && uv run checks/partial_messages.py latest/
23+
uv run run.py --node_count 8 --seed 1 --composition "rust-and-go" --scenario "partial-messages-fanout" && uv run checks/partial_messages.py latest/
24+
uv run run.py --node_count 8 --seed 2 --composition "rust-and-go" --scenario "partial-messages-fanout" && uv run checks/partial_messages.py latest/
25+
uv run run.py --node_count 8 --seed 3 --composition "rust-and-go" --scenario "partial-messages-fanout" && uv run checks/partial_messages.py latest/
26+
27+
test-go:
28+
# Testing partial messages
29+
@echo "Testing partial messages"
30+
@uv run run.py --node_count 8 --composition "all-go" --scenario "partial-messages" && uv run checks/partial_messages.py latest --count 1
31+
32+
@echo "Testing partial messages chain"
33+
@uv run run.py --node_count 8 --composition "all-go" --scenario "partial-messages-chain" && uv run checks/partial_messages.py latest --count 16
34+
35+
@echo "Testing fanout"
36+
@uv run run.py --node_count 2 --composition "all-go" --scenario "partial-messages-fanout" && uv run checks/partial_messages.py latest/
37+
38+
39+
test-rust-only:
40+
# Testing partial messages
41+
@echo "Testing partial messages"
42+
@uv run run.py --node_count 8 --composition "all-rust" --scenario "partial-messages" && uv run checks/partial_messages.py latest --count 1
43+
44+
@echo "Testing partial messages chain"
45+
@uv run run.py --node_count 8 --composition "all-rust" --scenario "partial-messages-chain" && uv run checks/partial_messages.py latest --count 16
46+
47+
@echo "Testing fanout"
48+
@uv run run.py --node_count 2 --composition "all-rust" --scenario "partial-messages-fanout" && uv run checks/partial_messages.py latest/
49+
50+
51+
52+
.PHONY: binaries all clean test

gossipsub-interop/README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,24 @@ After implementing it, make sure to add build commands in the Makefile's `binari
7676

7777
Finally, add it to the `composition` function in `experiment.py`.
7878

79+
## Examples
80+
81+
Minimal test of partial messages
82+
83+
```bash
84+
uv run run.py --node_count 2 --composition "all-go" --scenario "partial-messages" && uv run checks/partial_messages.py latest/
85+
```
86+
87+
That command runs the shadow simulation and then verifies the stdout logs have the expected message.
88+
89+
## Tests
90+
91+
```bash
92+
make test
93+
```
94+
95+
This runs various shadow simulations and checks.
96+
7997
## Future work (contributions welcome)
8098

8199
- Add more scenarios.
Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
#!/usr/bin/env python3
2+
"""Verify that each node stdout log contains the expected completion message."""
3+
4+
from __future__ import annotations
5+
6+
import argparse
7+
import sys
8+
from pathlib import Path
9+
10+
MESSAGE_SUBSTRING = '"msg":"All parts received"'
11+
12+
13+
def parse_args() -> argparse.Namespace:
14+
parser = argparse.ArgumentParser(
15+
description=(
16+
"Validate that every node stdout log inside a Shadow output directory "
17+
"contains the expected completion message."
18+
)
19+
)
20+
parser.add_argument(
21+
"shadow_output",
22+
help="Path to the Shadow output directory (the one containing the hosts/ folder).",
23+
)
24+
parser.add_argument(
25+
"--count",
26+
type=int,
27+
default=1,
28+
help="Minimum number of times each stdout log must contain the target message (default: 1).",
29+
)
30+
return parser.parse_args()
31+
32+
33+
def iter_stdout_logs(hosts_dir: Path):
34+
"""Yield all stdout log files under the given hosts directory."""
35+
for stdout_file in sorted(hosts_dir.rglob("*.stdout")):
36+
if stdout_file.is_file():
37+
yield stdout_file
38+
39+
40+
def count_occurrences(path: Path, needle: str) -> int:
41+
"""Count how many times the string appears inside the file."""
42+
if not needle:
43+
return 0
44+
total = 0
45+
with path.open("r", encoding="utf-8", errors="replace") as handle:
46+
for line in handle:
47+
total += line.count(needle)
48+
return total
49+
50+
51+
def main() -> int:
52+
args = parse_args()
53+
base_dir = Path(args.shadow_output).expanduser().resolve()
54+
if not base_dir.exists():
55+
print(f"shadow output directory does not exist: {base_dir}", file=sys.stderr)
56+
return 1
57+
58+
hosts_dir = base_dir / "hosts"
59+
if not hosts_dir.is_dir():
60+
print(f"hosts directory not found under: {base_dir}", file=sys.stderr)
61+
return 1
62+
63+
stdout_logs = list(iter_stdout_logs(hosts_dir))
64+
if not stdout_logs:
65+
print(f"no stdout logs found under: {hosts_dir}", file=sys.stderr)
66+
return 1
67+
68+
missing = []
69+
for log_path in stdout_logs:
70+
occurrences = count_occurrences(log_path, MESSAGE_SUBSTRING)
71+
if occurrences < args.count:
72+
missing.append((log_path, occurrences))
73+
74+
if missing:
75+
print(
76+
"The following stdout logs do not contain the required message:",
77+
file=sys.stderr,
78+
)
79+
for log_path, occurrences in missing:
80+
rel_path = log_path.relative_to(base_dir)
81+
print(
82+
f" - {rel_path}: found {occurrences} occurrences (expected >= {args.count})",
83+
file=sys.stderr,
84+
)
85+
print(
86+
f"{len(missing)} / {len(stdout_logs)} logs missing the message.",
87+
file=sys.stderr,
88+
)
89+
return 1
90+
91+
print(
92+
f"All {len(stdout_logs)} stdout logs under {hosts_dir} contain the required message."
93+
)
94+
return 0
95+
96+
97+
if __name__ == "__main__":
98+
sys.exit(main())

0 commit comments

Comments
 (0)