Skip to content

fix(message): rate-limit "Could not parse" warnings per NAME#1566

Merged
edenhaus merged 3 commits into
DeebotUniverse:devfrom
Beennnn:fix/warn-once-on-message-parse-failure
May 7, 2026
Merged

fix(message): rate-limit "Could not parse" warnings per NAME#1566
edenhaus merged 3 commits into
DeebotUniverse:devfrom
Beennnn:fix/warn-once-on-message-parse-failure

Conversation

@Beennnn

@Beennnn Beennnn commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

When a firmware version pushes messages whose on-wire format the lib does not yet support, every push triggers a Could not parse <NAME> WARNING via _handle_error_or_analyse. In one observed case a mower's spontaneous getMapTrace pushes produced 217 520 identical entries in 3 days (~70k/day, ~50/min during mowing), bloating the parent process' log file and any downstream sink (HA recorder DB, Loki, etc.).

The first few warnings are useful — they let an operator notice that something is wrong. Beyond that, identical repeats add no information and only make the rest of the log harder to read.

Change

Per-NAME counter with a default threshold of 3:

  • First 3 parse failures for a given message NAME → WARNING (as before)
  • 3rd warning is followed by a single notice: "Further 'Could not parse ' entries will be logged at DEBUG level"
  • 4th and beyond → DEBUG

Counters are isolated per NAME, so a new message class hitting a parse error still surfaces normally even if another NAME has been silenced.

The 3 existing emission sites all route through the new _log_parse_failure helper:

  • _handle_error_or_analyse exception branch (line 68)
  • _handle_error_or_analyse ERROR-state branch (line 84)
  • MessageBodyData.__handle_body_data exception branch (line 259)

Behaviour for non-throttled flows (SUCCESS, ANALYSE) is unchanged.

Refs

Test plan

  • test_warn_once_throttles_repeated_parse_failures — calls Message.handle() THRESHOLD + 2 times on a class whose _handle always raises, asserts THRESHOLD + 1 WARNINGs (incl. the "switching to DEBUG" notice) followed by 2 DEBUGs
  • test_warn_once_isolated_per_message_name — burns through the threshold for one NAME, verifies a sibling NAME still WARNs on its first parse failure
  • Full suite: 699/699 tests pass, no regression

Notes for review

  • The threshold (3) is module-level; happy to make it configurable via env var or Message class attribute if that fits the project style better.
  • Counter is module-level state; not reset across the lifetime of a process. A "Restart the parent process to reset" note is included in the threshold-reached message. If you'd rather have an admin API to reset, happy to add it.

When a firmware version pushes messages whose on-wire format the lib
does not yet support, every push triggers a `Could not parse <NAME>`
WARNING via `_handle_error_or_analyse`. In one observed case a mower's
spontaneous `getMapTrace` pushes produced 217 520 identical entries in
3 days (~70k/day, ~50/min during mowing), bloating the parent process'
log file and any downstream sink (HA recorder DB, Loki, etc.).

The first few warnings are useful — they let an operator notice that
something is wrong. Beyond that, identical repeats add no information
and only make the rest of the log harder to read.

This adds a per-NAME counter and a threshold (default 3): the first 3
parse failures for a given message NAME are still logged at WARNING,
followed by a single notice that further entries will be downgraded to
DEBUG. After that, all subsequent parse failures for that NAME log at
DEBUG. Counters are isolated per NAME, so a NEW message class hitting
a parse error still surfaces normally.

The 3 existing emission sites (`_handle_error_or_analyse` exception
path, `_handle_error_or_analyse` ERROR-state path, `MessageBodyData`
exception path) all route through the new helper. Behaviour for
non-throttled flows (success, ANALYSE) is unchanged.

Tests:

- `test_warn_once_throttles_repeated_parse_failures` — calls
  `Message.handle()` `THRESHOLD + 2` times on a class whose `_handle`
  always raises, asserts THRESHOLD + 1 WARNINGs (incl. the
  "switching to DEBUG" notice) followed by 2 DEBUGs.
- `test_warn_once_isolated_per_message_name` — burns through the
  threshold for one NAME, verifies a sibling NAME still WARNs on its
  first parse failure.
- Full suite: 699/699 pass.

Refs: DeebotUniverse#1376
Comment thread deebot_client/message.py Outdated
Comment thread deebot_client/message.py Outdated
Comment thread tests/test_message.py Outdated
Comment thread tests/test_message.py Outdated
Comment thread tests/test_message.py Outdated
Comment thread tests/test_message.py
…log assertions

Address review feedback on DeebotUniverse#1566:
- Remove the file-level explanatory comment block above the threshold
- Drop the "Restart Home Assistant" line — the lib does not know its host
- Replace per-NAME counter cleanup with a pytest fixture that wipes the
  whole dict before and after each test
- Compare ``caplog.records`` against the full expected level/message
  list in order (the previous count-only assertion would have passed
  with WARNING/DEBUG entries interleaved out of order)
Comment thread deebot_client/message.py Dismissed
Comment thread deebot_client/message.py Dismissed

@edenhaus edenhaus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure CI is passing

@codecov

codecov Bot commented May 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.99%. Comparing base (8980d8c) to head (b5d93ce).
⚠️ Report is 14 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1566      +/-   ##
==========================================
+ Coverage   94.92%   94.99%   +0.07%     
==========================================
  Files         156      158       +2     
  Lines        6105     6197      +92     
  Branches      350      352       +2     
==========================================
+ Hits         5795     5887      +92     
  Misses        248      248              
  Partials       62       62              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@edenhaus edenhaus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Beennnn 👍

Did some small refactoring in the tests

@edenhaus edenhaus enabled auto-merge (squash) May 7, 2026 22:20
@edenhaus edenhaus added the pr: bugfix PR, which fixes a bug label May 7, 2026
@edenhaus edenhaus disabled auto-merge May 7, 2026 22:21
@codspeed-hq

codspeed-hq Bot commented May 7, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 113 untouched benchmarks


Comparing Beennnn:fix/warn-once-on-message-parse-failure (b5d93ce) with dev (7266b58)

Open in CodSpeed

@edenhaus edenhaus merged commit f87b6c9 into DeebotUniverse:dev May 7, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr: bugfix PR, which fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants