Skip to content

WebSocket performance: vectorized masking + zero-copy codec paths (large messages 0.4x -> 2.5x of 1.x)#1304

Merged
quinnj merged 5 commits into
masterfrom
jq-ws-perf
Jun 12, 2026
Merged

WebSocket performance: vectorized masking + zero-copy codec paths (large messages 0.4x -> 2.5x of 1.x)#1304
quinnj merged 5 commits into
masterfrom
jq-ws-perf

Conversation

@quinnj

@quinnj quinnj commented Jun 12, 2026

Copy link
Copy Markdown
Member

Addresses the report that 2.x WebSockets are slow vs 1.x (large binary messages — the Bonito/WGLMakie regime — were up to 2.3× slower). Full analysis in bench/websocket/REPORT.md.

Result (vs HTTP 1.11.0, macOS aarch64, -t 4, medians of 3)

cell before this PR after vs 1.x
send_1m 433 MB/s (0.43× of 1.x) 2541 MB/s 2.54×
echo_64k 313 MB/s (0.61×) 888 MB/s 1.72×
send_4k 503 MB/s 862 MB/s 6.7×
small msgs (16B) 6–8× of 1.x unchanged-to-better, allocs/msg 11–15 → 2–5
latency p50/p99 56 / 89 µs 0.73× / 0.63×

send_1m now allocates payload + ~5 KB per message — the user-owned payload (receive hands ownership) is the API floor.

What changed

Masking (_ws_mask_into!): 8 bytes/iteration with the key broadcast into a UInt64 (phase-aware for chunked delivery, in-place capable, fuzz-validated in bench/websocket/proto_mask.jl). Replaces byte-at-a-time XOR with a per-byte modulo — masked decode was the server's receive path for all client traffic and ran at 0.53 GiB/s; now 7.1 GiB/s.

Decode: explicit payload_received cursor; exact-size presize for frames ≤ 4 MiB (capped so a forged length header can't allocate unbounded memory); per-byte push!copyto!/mask-into; frame payload ownership handoff instead of a copy per frame; reused frames_scratch; (buf, n) chunk API (no per-read view); callable-struct header guard (no boxed capture).

Encode: frames append directly into a persistent outgoing buffer (no per-frame buffer, no concat copy; capacity retained across flushes). The buffer has a leaf lock never held across socket I/O — this also fixes a latent race where the read task queued PONG/CLOSE replies into the outgoing queue unsynchronized with senders (socket writes stay serialized by the send lock as before). Masking key from rand(UInt32).

Correctness

  • Codec 88/88, client 20/20, server 35/35, integration 42/42.
  • Autobahn fuzzing suite: 463 cases, zero failures, run before and after the rewrite with identical verdicts (216 UNIMPLEMENTED = case 12/13.* permessage-deflate, not implemented by design).

Also in this PR

bench/websocket/: a version-agnostic benchmark suite that runs identically under 1.x and 2.x (send/push/echo throughput at 16B/4KiB/64KiB/1MiB, ping-pong latency, whole-process allocs+GC per message), codec microbenchmarks, the mask-kernel prototype + fuzz, baseline runner, comparison table, and REPORT.md with the full methodology and numbers.

🤖 Generated with Claude Code

quinnj and others added 3 commits June 11, 2026 17:53
…ecode

- _ws_mask_into!: 8-bytes-per-iteration XOR with the 4-byte key broadcast
  into a UInt64 (phase-aware for chunked delivery, in-place capable).
  Replaces byte-at-a-time XOR with a per-byte modulo on both the encode
  and decode paths: masked decode 0.53 -> 7.1 GiB/s (13x), masked encode
  1.35 -> 6.7 GiB/s (5x) at 1 MiB.
- Decoder payload assembly: explicit payload_received cursor, exact-size
  presize for frames <= 4 MiB (claimed-length capped so a forged length
  header cannot allocate unbounded memory), copyto!/mask-into instead of
  per-byte push!, and frame-payload ownership handoff (swap) instead of a
  payload-sized copy per frame.
- Client masking key from rand(UInt32) instead of a rand(UInt8, 4) Vector.

bench/websocket: version-agnostic 1.x-vs-2.x benchmark suite (send/push/echo/
latency with allocs+GC tracking), codec microbench, mask-kernel prototype
with fuzz validation, baseline runner + comparison table.

End-to-end (vs HTTP 1.11.0, -t 4, macOS aarch64, medians of 3):
  send_1m   0.43x -> 1.33x  (433 -> 1329 MB/s)
  echo_64k  0.61x -> 1.15x  (313 ->  590 MB/s)
  send_4k   3.9x  -> 5.1x; small-message cells unchanged (6-8x faster than 1.x)

Codec tests 88/88; WS client/server/integration suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mination

- Outgoing frames encode directly into a persistent WSConn buffer
  (_ws_append_frame!): removes the per-frame ByteMemory allocation and the
  full concat copy in ws_get_outgoing_data!, which now swaps buffers
  instead of copying. Steady-state sends allocate nothing in the codec.
- The buffer is guarded by a leaf out_lock never held across socket I/O —
  this also fixes a latent race where the read task queued PONG/CLOSE
  replies into the outgoing queue unsynchronized with senders. Socket
  writes remain serialized by the connection send lock as before.
- Decoder: reused frames_scratch vector (one less alloc per socket read);
  extended-length/masking-key caches filled without view allocations;
  ws_decoder_process!/ws_on_incoming_data! take an explicit datalen so the
  read loop passes (buf, n) without allocating a view per read.
- Read loop hoists its frame callback (one closure per connection, not per
  read); the configured-maximum header guard is a callable struct instead
  of a boxed-capture closure.

End-to-end (vs HTTP 1.11.0, -t 4, macOS aarch64, medians of 3):
  send_1m   433 -> 2566 MB/s cumulative (0.43x -> 2.56x of 1.x)
  echo_64k  313 ->  920 MB/s cumulative (0.61x -> 1.79x)
  send_4k   7.2x of 1.x; send_16b allocs/msg 11.4 -> 4.1 (whole process)

Codec 88/88, client 20/20, server 35/35, integration 42/42.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.07092% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.09%. Comparing base (6022a9f) to head (c1ebf9f).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
src/http_websocket_codec.jl 89.92% 14 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1304      +/-   ##
==========================================
+ Coverage   86.83%   87.09%   +0.26%     
==========================================
  Files          28       28              
  Lines       10799    10889      +90     
==========================================
+ Hits         9377     9484     +107     
+ Misses       1422     1405      -17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

quinnj and others added 2 commits June 11, 2026 18:32
…ffer swap

- >4 MiB masked frame delivered in odd-sized chunks (incremental-growth
  branch + key phase rotation across chunk boundaries)
- small masked frame split at every byte boundary (presize + cursor)
- max-length header guard: fragmented within/over budget, reset after fin,
  control-frame limit
- outgoing buffer swap: take/empty/re-take semantics

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@quinnj quinnj merged commit d48b520 into master Jun 12, 2026
8 checks passed
@quinnj quinnj deleted the jq-ws-perf branch June 12, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant