Skip to content

Commit ac2070f

Browse files
committed
Add echo-ws-pipeline test
1 parent fb2565c commit ac2070f

7 files changed

Lines changed: 81 additions & 1 deletion

File tree

site/content/_index.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ html.dark .test-card-endpoint { color: #64748b; }
6868
</style>
6969

7070
<div class="tests-section">
71-
<h2>27 Test Profiles Across H/1.1, H/2, H/3, gRPC and WebSocket</h2>
71+
<h2>28 Test Profiles Across H/1.1, H/2, H/3, gRPC and WebSocket</h2>
7272
<p class="tests-sub">Every framework is tested under diverse, realistic workloads — from raw throughput to JSON processing, gRPC unary calls, and WebSocket echo.</p>
7373

7474
<div class="tests-proto">
@@ -245,6 +245,11 @@ html.dark .test-card-endpoint { color: #64748b; }
245245
<div class="test-card-desc">WebSocket echo throughput — upgrade, send pipelined text messages, receive echoes. Measures frame processing performance.</div>
246246
<div class="test-card-endpoint">WS /ws (echo)</div>
247247
</a>
248+
<a class="test-card" href="docs/test-profiles/ws/echo-pipeline">
249+
<div class="test-card-title">Echo Pipelined</div>
250+
<div class="test-card-desc">Same WebSocket echo with 16 messages in flight per connection — measures frame batching and read-buffer draining under burst load.</div>
251+
<div class="test-card-endpoint">WS /ws (echo, p=16)</div>
252+
</a>
248253
</div>
249254
</div>
250255

site/content/docs/test-profiles/ws/_index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,5 @@ WebSocket test profiles measure framework performance for real-time bidirectiona
77

88
{{< cards >}}
99
{{< card link="echo" title="Echo" subtitle="WebSocket echo throughput — upgrade, send messages, receive echoes." icon="globe-alt" >}}
10+
{{< card link="echo-pipeline" title="Echo Pipelined (16x)" subtitle="WebSocket echo with 16 messages in flight per connection — measures frame batching and read-buffer draining." icon="fast-forward" >}}
1011
{{< /cards >}}
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
title: Echo Pipelined (WebSocket)
3+
---
4+
5+
Measures WebSocket echo throughput with pipelining. Each connection upgrades via HTTP/1.1, then sends 16 messages back-to-back before waiting for the echoes.
6+
7+
{{< cards >}}
8+
{{< card link="implementation" title="Implementation Guidelines" subtitle="Endpoint specification, expected request/response format, and type-specific rules." icon="code" >}}
9+
{{< card link="validation" title="Validation" subtitle="All checks executed by the validation script for this test profile." icon="check-circle" >}}
10+
{{< /cards >}}
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
---
2+
title: Implementation Guidelines
3+
---
4+
{{< type-rules production="Must use the framework standard WebSocket API with default buffer sizes. No custom batching or read-ahead optimizations." tuned="May optimize WebSocket frame handling, buffer sizes, and use custom frame parsers or batched read paths." engine="No specific rules. Ranked separately from frameworks." >}}
5+
6+
7+
Measures WebSocket echo throughput with pipelining. Each connection upgrades via HTTP/1.1, then sends 16 text messages back-to-back before waiting for the echoes. Each echo counts as one completed response.
8+
9+
**Connections:** 512, 4,096, 16,384
10+
**Pipeline:** 16 (16 messages in flight per connection — send batch, drain echoes, repeat)
11+
12+
## Workload
13+
14+
1. Open TCP connection to port 8080
15+
2. Send HTTP/1.1 upgrade request to `/ws`
16+
3. After receiving `101 Switching Protocols`, switch to WebSocket framing
17+
4. Send 16 text frames containing `"hello"` back-to-back, then read 16 echo frames
18+
5. Measure messages per second
19+
20+
## What it measures
21+
22+
- WebSocket frame parsing efficiency under burst load
23+
- Frameworks that drain multiple frames from a single read buffer gain a major advantage
24+
- Frameworks processing one frame at a time per connection see minimal improvement over the non-pipelined echo
25+
- Write coalescing and syscall reduction on the send path
26+
27+
## Expected upgrade request/response
28+
29+
```
30+
GET /ws HTTP/1.1
31+
Host: localhost:8080
32+
Upgrade: websocket
33+
Connection: Upgrade
34+
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
35+
Sec-WebSocket-Version: 13
36+
```
37+
38+
```
39+
HTTP/1.1 101 Switching Protocols
40+
Upgrade: websocket
41+
Connection: Upgrade
42+
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
43+
```
44+
45+
## Parameters
46+
47+
| Parameter | Value |
48+
|-----------|-------|
49+
| Endpoint | `/ws` (WebSocket upgrade) |
50+
| Connections | 512, 4,096, 16,384 |
51+
| Pipeline | 16 (16 messages in flight per connection) |
52+
| Message | `"hello"` (5 bytes, text frame) |
53+
| Duration | 5s |
54+
| Runs | 3 (best taken) |
55+
| Load generator | gcannon `--ws -p 16` |
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
title: Validation
3+
---
4+
5+
The same `validate-ws.py` checks executed for the [Echo](../echo/validation/) profile apply here — pipelining is a load-generator behavior, not a separate server contract. The endpoint and frame semantics are identical, so a server that passes the Echo validation passes for Echo Pipelined as well.
6+
7+
See the [Echo validation page](../echo/validation/) for the full list of checks (handshake, accept header, text/binary echo, multi-message echo, clean close, non-upgrade rejection, post-validation health check).

site/layouts/shortcodes/leaderboard-composite.html

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
(dict "id" "baseline-h3" "label" "Baseline" "sublabel" "H/3" "proto" "h3" "conns" (slice 64) "scored" true "engineScored" true "infraScored" false)
2727
(dict "id" "static-h3" "label" "Static" "sublabel" "H/3" "proto" "h3" "conns" (slice 64) "scored" true "engineScored" true "infraScored" false)
2828
(dict "id" "echo-ws" "label" "Echo" "sublabel" "WebSocket" "proto" "ws" "conns" (slice 512 4096 16384) "scored" true "engineScored" true "infraScored" false)
29+
(dict "id" "echo-ws-pipeline" "label" "Echo Pipelined" "sublabel" "WebSocket" "proto" "ws" "conns" (slice 512 4096 16384) "scored" true "engineScored" true "infraScored" false)
2930
}}
3031

3132
{{/* Build a clean JSON blob: array of {profile, framework, rps, language} entries */}}

site/layouts/shortcodes/leaderboard-ws.html

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{{ $profiles := slice
22
(dict "id" "echo-ws" "label" "Echo (WebSocket)" "conns" (slice 512 4096 16384) "desc" "WebSocket echo test — upgrade to WebSocket on <code>/ws</code>, then send one message at a time and wait for the echo before sending the next. Measures single-message round-trip throughput. Benchmarked with <code>gcannon --ws -p 1</code>. Container pinned to 64 CPU threads (cores 0-31, 64-95). <a href='/docs/test-profiles/ws/echo/' class='lb-desc-link'>Learn more &rarr;</a>")
3+
(dict "id" "echo-ws-pipeline" "label" "Echo Pipelined (WebSocket)" "conns" (slice 512 4096 16384) "desc" "WebSocket echo test with pipelining — upgrade to WebSocket on <code>/ws</code>, then send 16 messages back-to-back per connection before waiting for the echoes. Measures batched round-trip throughput and how efficiently a framework drains the read buffer. Benchmarked with <code>gcannon --ws -p 16</code>. Container pinned to 64 CPU threads (cores 0-31, 64-95). <a href='/docs/test-profiles/ws/echo/' class='lb-desc-link'>Learn more &rarr;</a>")
34
}}
45

56
{{/* Build JSON data blob for round switching */}}

0 commit comments

Comments
 (0)