Benchmark safe_label_str hello and handshake-reject paths#1687
Conversation
Adds CodSpeed coverage for the three places `safe_label_str` runs on the wire-format hot path so any future regression to the printable filter or length-cap shows up as a measurable delta. - `test_noise_hello_parse` drives a hello frame through a fresh MockAPINoiseFrameHelper, exercising `_handle_hello` and its two safe_label_str calls (server name and MAC). - `test_noise_handshake_reject_parse` drives hello + reject frames so `_error_on_incorrect_preamble` runs the explanation sanitize. - `test_safe_label_str_throughput` is a parametrized direct benchmark of `safe_label_str` over realistic name / MAC / explanation inputs (clean and control-char-laden), catching algorithm-level regressions in the printable-filter / length-cap path.
Merging this PR will not alter performance
Performance Changes
Comparing 
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1687 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 25 25
Lines 4159 4159
=========================================
Hits 4159 4159 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds CodSpeed benchmarks to quantify the overhead of safe_label_str on the Noise “hello” and handshake-reject parsing paths, plus a direct throughput benchmark for safe_label_str with realistic inputs.
Changes:
- Add benchmarks that drive a hello frame through
MockAPINoiseFrameHelperto cover the server-name and MAC sanitization calls. - Add benchmarks that drive hello + a crafted handshake-reject frame to cover the explanation sanitization path.
- Add a parametrized throughput benchmark for
safe_label_strover clean/noisy representative strings and per-field length caps.
Comments suppressed due to low confidence (1)
tests/benchmarks/test_noise.py:284
- Same concern as the hello benchmark: each inner iteration constructs a new helper (including PSK decode +
NoiseConnectionsetup) and then triggers an error path that setsready_futureto an exception. Because the future’s exception is never awaited/consumed, repeated iterations can generate “Future exception was never retrieved” noise and affect benchmark stability. Preferbenchmark.pedanticwith per-iteration setup/teardown that consumes/cancelsready_future, and keep helper/proto setup outside the timed portion if the intent is to measure the handshake-reject sanitize path specifically.
@benchmark
def parse_reject_packets() -> None:
for _ in range(20):
connection, _ = _make_mock_connection()
helper = MockAPINoiseFrameHelper(
connection=connection,
noise_psk=noise_psk,
expected_name=None,
expected_mac=None,
client_info="my client",
log_name="test",
writer=_drop,
)
mock_data_received(helper, hello_pkt)
mock_data_received(helper, reject_frame)
helper.close()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @benchmark | ||
| def parse_hello_packets() -> None: | ||
| for _ in range(20): | ||
| connection, _ = _make_mock_connection() | ||
| helper = MockAPINoiseFrameHelper( | ||
| connection=connection, | ||
| noise_psk=noise_psk, | ||
| expected_name=None, | ||
| expected_mac=None, | ||
| client_info="my client", | ||
| log_name="test", | ||
| writer=_drop, | ||
| ) | ||
| mock_data_received(helper, hello_pkt) | ||
| helper.close() |
What does this implement/fix?
Adds CodSpeed benchmarks for the three sites where
safe_label_strruns on the noise wire-format hot path; this lands before #1669 so the cimport rewiring there has a measurable before/after baseline instead of going in blind.test_noise_hello_parsedrives a hello frame through a freshMockAPINoiseFrameHelper, covering_handle_helloand its two sanitize calls (server name, MAC).test_noise_handshake_reject_parsedrives hello + reject so_error_on_incorrect_preambleruns the explanation sanitize.test_safe_label_str_throughputis a parametrized direct benchmark over realistic name, MAC and explanation inputs, both clean and with embedded control bytes, so any regression in the printable filter or length cap shows up directly.No production code is touched, only
tests/benchmarks/test_noise.py.Types of changes
Related issue or feature (if applicable):
Pull request in esphome (if applicable):
Checklist:
tests/folder).