Skip to content

Commit dd26cca

Browse files
committed
fix: audit remediation — auto-heal retries, persisted update dedup, hardening (v0.15.0)
All 16 items from the 2026-06-06 codebase audit: - Auto-heal re-enters heal() for stuck-unhealthy containers; storm guard bounds the loop; failed restarts send a distinct alert and count toward the guard; window uses monotonic clock - Image-update dedup persists to data/announced_updates.json - gather_context degrades gracefully on Docker hiccups - Unraid API error bodies redacted+truncated in all paths - fh_tog/set_containers validate container names - alert_callbacks parse boilerplate -> 4 shared helpers (935->838 lines) - pip-audit in dev deps + CI; idna/requests/urllib3 CVE bumps (aiohttp CVEs blocked on aiogram's <3.14 pin) - restart guard, version-store corrupt-file cleanup, misc hygiene - Fix broken PreToolUse prompt hook -> deterministic command hook Audit: audit-reports/audit-2026-06-06-1538.md (27/29 prior findings fixed)
1 parent 26cce4c commit dd26cca

40 files changed

Lines changed: 2455 additions & 326 deletions

.claude/settings.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,8 @@
2626
"matcher": "Edit|Write",
2727
"hooks": [
2828
{
29-
"type": "prompt",
30-
"prompt": "Check if the file being edited has a name matching any of these patterns: .env, .env.*, *.pem, *.key, credentials*, secrets*. If it matches, respond {\"decision\": \"block\", \"reason\": \"Blocked edit to secrets/credentials file. Modify these manually.\"}. Otherwise respond {\"decision\": \"allow\"}."
29+
"type": "command",
30+
"command": "f=$(jq -r \".tool_input.file_path // empty\" 2>/dev/null); b=$(basename \"$f\" 2>/dev/null); case \"$b\" in .env|.env.*|*.pem|*.key|credentials*|secrets*) echo \"{\\\"hookSpecificOutput\\\":{\\\"hookEventName\\\":\\\"PreToolUse\\\",\\\"permissionDecision\\\":\\\"deny\\\",\\\"permissionDecisionReason\\\":\\\"Blocked edit to secrets/credentials file. Modify these manually.\\\"}}\" ;; esac"
3131
}
3232
]
3333
}

.claude/structure/monitors.yaml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -214,17 +214,17 @@ files:
214214
items: [MemoryConfig]
215215

216216
image_update_monitor.py:
217-
description: Polls Docker registries for image digest changes, deduplicates across polls, sends batched update alert with Pull buttons
217+
description: Polls Docker registries for image digest changes, deduplicates across polls AND restarts (persisted to data/announced_updates.json), sends batched update alert with Pull buttons
218218
exports:
219219
- name: extract_local_digests
220220
kind: function
221221
description: Return sha256 digests from a container's RepoDigests
222-
params: [{name: container, type: Any}]
222+
params: [{name: container, type: Container}]
223223
returns: list[str]
224224

225225
- name: ImageUpdateMonitor
226226
kind: class
227-
description: Poll loop that checks opted-in containers for newer registry images and emits a batched digest alert
227+
description: Poll loop that checks opted-in containers for newer registry images and emits a batched digest alert; constructor takes optional state_path for the persisted dedup map, prunes entries for removed containers
228228
methods:
229229
- name: start
230230
params: []
@@ -237,6 +237,8 @@ files:
237237
returns: bool
238238

239239
imports:
240+
- from: docker.models.containers
241+
items: [Container]
240242
- from: src.config
241243
items: [ImageUpdatesConfig]
242244
- from: src.alerts.manager

.claude/structure/services.yaml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -201,18 +201,21 @@ files:
201201
items: [sanitize_container_name, sanitize_logs]
202202

203203
auto_healer.py:
204-
description: Auto-restarts opted-in containers that report HEALTHCHECK unhealthy, with per-container time-windowed storm guard
204+
description: Auto-restarts opted-in containers that report HEALTHCHECK unhealthy, with per-container time-windowed storm guard (monotonic clock); failed restarts count toward the guard
205205
exports:
206+
- name: HealOutcome
207+
kind: type-alias
208+
description: Literal["restarted", "failed", "gave_up"] — result of a heal attempt
206209
- name: AutoHealer
207210
kind: class
208-
description: Checks opt-in list and protected containers, tracks restart counts in a rolling window, restarts or escalates
211+
description: Checks opt-in list and protected containers, tracks restart attempts in a rolling window, restarts or escalates; checks ContainerController.restart result and sends failed-flavoured alert when the restart errors
209212
methods:
210213
- name: is_enabled
211214
params: [{name: container_name, type: str}]
212215
returns: bool
213216
- name: heal
214217
params: [{name: container_name, type: str}]
215-
returns: Awaitable[None]
218+
returns: Awaitable[HealOutcome]
216219

217220
imports:
218221
- from: src.config

.claude/structure/utils.yaml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,25 @@ files:
126126
- from: re
127127
items: [compile, sub, search, DOTALL, IGNORECASE, MULTILINE]
128128

129+
telegram_format.py:
130+
description: Convert LLM-generated Markdown into Telegram-safe HTML (escape-first, code blocks stashed) and strip tags for plain-text fallback
131+
exports:
132+
- name: markdown_to_telegram_html
133+
kind: function
134+
description: Render LLM markdown (bold/italic/code/links/headings) as Telegram HTML; html.escape runs before markdown conversion so model output cannot inject tags
135+
params: [{name: text, type: str}]
136+
returns: str
137+
- name: strip_html_tags
138+
kind: function
139+
description: Strip HTML tags and unescape entities for the plain-text fallback path
140+
params: [{name: text, type: str}]
141+
returns: str
142+
imports:
143+
- from: html
144+
items: [escape, unescape]
145+
- from: re
146+
items: [compile]
147+
129148
telegram_retry.py:
130149
description: Telegram API retry utilities for handling rate limits and transient failures.
131150
exports:

.github/workflows/ci.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,8 @@ jobs:
3232

3333
- name: Test (pytest)
3434
run: uv run pytest tests/
35+
36+
# Audits the project venv for known CVEs. Non-blocking is deliberate:
37+
# a new upstream CVE shouldn't break unrelated PRs — review the log.
38+
- name: Dependency audit (pip-audit)
39+
run: uv run pip-audit || true

CHANGELOG.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,32 @@
22

33
All notable changes to UnraidMonitor will be documented in this file.
44

5+
## [0.15.0] - 2026-06-06
6+
7+
Audit remediation release — all 16 items from the June 6 codebase audit (`audit-reports/audit-2026-06-06-1538.md`).
8+
9+
### Fixed
10+
- **Auto-heal now actually retries containers that stay unhealthy** — the `_unhealthy_alerted` dedup set blocked the post-restart unhealthy event from re-entering `heal()`, so a persistently-broken container got exactly one restart attempt and then silence: the documented storm guard (`max_restarts` per `window_minutes`) and the give-up escalation were only reachable for *flapping* containers. The dedup flag is now cleared after each heal attempt (unless the healer has given up), so the storm guard — not the dedup set — bounds the retry loop. `heal()` also checks the result of `controller.restart()` now: a failed restart sends a distinct "❌ Auto-heal failed" alert instead of the success-implying "attempt N/M". Failed attempts count toward the storm guard so a restart that keeps erroring escalates instead of retrying forever.
11+
- **Image-update alerts no longer repeat after a bot restart** — the dedup map (container → announced digest) was in-memory only, and the bot restarts itself routinely (setup wizard, `/manage` feature toggles, `/restart`). It now persists to `data/announced_updates.json` (atomic write, corrupt-file tolerant) and prunes entries for removed containers — never on a failed Docker poll, so a daemon outage can't wipe it.
12+
- **`/diagnose` no longer goes silent after "Analyzing…" when Docker hiccups**`gather_context()` only handled `NotFound`; an `APIError` from `containers.get` or any failure fetching logs crashed the handler. Daemon errors now degrade to "couldn't gather context", and unavailable logs degrade to a partial diagnosis with `(logs unavailable)`.
13+
- **Concurrent `/restart` triggers no longer send duplicate notices** — a module-level guard with a `finally` reset (a *failed* `os.execv` clears it so retries still work).
14+
- **Corrupted `data/announced_version.json` is removed on read** so "What's new" isn't re-shown on every boot after a bad write.
15+
16+
### Security
17+
- **Unraid API error bodies are now redacted and truncated in all paths**`_execute_query()` raised exceptions containing the full response body (the connectivity test already redacted; the query paths didn't). New `_safe_body()` helper redacts the API key *before* truncating so a key straddling the cut-off can't partially leak.
18+
- **Auto-heal picker validates container names**`fh_tog:` callback data is now checked against the live candidate list (every other callback family already validated), and `AutoHealConfig.set_containers()` defensively filters invalid names, so a spoofed callback can't persist junk into config.yaml.
19+
- **Dependency CVE scanning added** (`pip-audit` in dev deps + non-blocking CI step). First run found 6 CVEs: `idna`, `requests`, `urllib3` upgraded in the lockfile. `aiohttp` 3.13.5 (CVE-2026-34993, CVE-2026-47265, fixed in 3.14.0) remains pinned by aiogram 3.28.2 (`aiohttp<3.14`) — exposure is client-only; re-check when aiogram releases.
20+
- **Defense-in-depth**: diagnostic `alert_context` strings pass through `sanitize_for_prompt()`, and `markdown_to_telegram_html` now has explicit regression tests proving model output can't inject `<tg-spoiler>`/`<a href>` tags.
21+
22+
### Changed
23+
- **AutoHealer storm-guard window uses `time.monotonic()`** instead of wall-clock, so NTP steps can't stretch or shrink it; `_recent_count` renamed to `_prune_and_count` to reflect its mutation.
24+
- **`alert_callbacks.py` callback parsing deduplicated** — four shared helpers (`_parse_container_callback`, `_parse_valued_callback`, `_parse_metric_callback`, `_parse_minutes_callback`) replace ~230 lines of repeated parse→validate→lookup boilerplate across 12 handlers (935 → 838 lines). New alert buttons get name validation for free.
25+
- `atomic_yaml_write()` sets 0o644 on fresh files (matching `version_store.py`); `ResourceMonitor._violation_last_seen` initialised in `__init__` instead of lazily; `extract_local_digests` properly typed.
26+
27+
### Added
28+
- `send_autoheal_alert(..., failed=True)` variant across protocol, manager, and proxy with a distinct failure message.
29+
- 23 new tests (1169 total), including an end-to-end storm-guard test (unhealthy → 3 restarts → exactly one give-up escalation → silence) and a restart-survival test for image-update dedup.
30+
531
## [0.14.4] - 2026-06-05
632

733
### Fixed

CLAUDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ src/utils/version_store.py - read/write data/announced_version.json for startup
9292

9393
## Project Overview
9494

95-
Unraid Server Monitor Bot (v0.14.4) - A Docker-based Telegram bot for monitoring Unraid servers. Monitors Docker containers (events, logs, resources) and Unraid server health (CPU, memory, disks, array, UPS). Uses multi-provider LLM support (Anthropic, OpenAI, Ollama) for AI-powered diagnostics and natural language interaction. Sends alerts via Telegram with quick-action buttons.
95+
Unraid Server Monitor Bot (v0.15.0) - A Docker-based Telegram bot for monitoring Unraid servers. Monitors Docker containers (events, logs, resources) and Unraid server health (CPU, memory, disks, array, UPS). Uses multi-provider LLM support (Anthropic, OpenAI, Ollama) for AI-powered diagnostics and natural language interaction. Sends alerts via Telegram with quick-action buttons.
9696

9797
**Version string lives in TWO places**`pyproject.toml` (`version`) and `src/__init__.py` (`__version__`). Both must be bumped together on release; `tests/test_version.py` guards against drift. `BOT_VERSION` (in `src/bot/health_command.py`) resolves from installed package metadata when available, else falls back to `src.__version__` — the package is not installed in the Docker image, so the `__version__` fallback is what runs in production.
9898

@@ -278,7 +278,7 @@ Environment variables can also be set via `config/.env` (loaded by pydantic-sett
278278
- `image_updates` - Opt-in daily digest of containers with newer images (enabled, poll_interval_hours)
279279
- `auto_heal` - Opt-in auto-restart of unhealthy containers (enabled, containers, max_restarts, window_minutes)
280280

281-
Data files in `data/`: `mutes.json`, `server_mutes.json`, `array_mutes.json`, `ignored_errors.json`, `model_selection.json` (runtime LLM choice), `chat_ids.json` (persistent Telegram chat IDs for alert delivery), `announced_version.json` (last-announced bot version for startup What's new gate)
281+
Data files in `data/`: `mutes.json`, `server_mutes.json`, `array_mutes.json`, `ignored_errors.json`, `model_selection.json` (runtime LLM choice), `chat_ids.json` (persistent Telegram chat IDs for alert delivery), `announced_version.json` (last-announced bot version for startup What's new gate), `announced_updates.json` (image-update dedup map so restarts don't re-announce)
282282

283283
## Design Context
284284

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Implementation Plan — Codebase Audit 2026-04-23 12:27
2+
3+
## Overview
4+
5+
16 findings total (0 Critical, 2 High, 6 Medium, 8 Low). Estimated effort: ~9 S, ~5 M, ~2 L tasks. All prior audit findings (17/17) are resolved — this plan covers new findings only.
6+
7+
## Phase 1: Security & Race Conditions (Do First)
8+
9+
| # | Finding | Files | Effort | Description |
10+
|---|---------|-------|--------|-------------|
11+
| 1 | H1 — Unsanitized tool results fed to LLM | `src/services/nl_processor.py:285`, `src/services/nl_tools.py:462` | M | In `nl_processor.py:285`, wrap tool result: `from src.utils.sanitize import sanitize_for_prompt; result = sanitize_for_prompt(result, max_length=8000)` before appending to `tool_results`. Also sanitize error text in `_tool_get_recent_errors()` at nl_tools.py:462. |
12+
| 2 | H2 — MemoryMonitor kill/decline race | `src/monitors/memory_monitor.py:162-214` | M | Add `self._kill_lock = asyncio.Lock()` to `__init__`. Wrap `_execute_kill_countdown()` lines 162-207 in `async with self._kill_lock`. In `cancel_pending_kill()`, also acquire the lock. Re-check `_kill_cancel_event.is_set()` immediately before `_stop_container()` at line 196. |
13+
| 3 | M2 — Uncapped mute duration in callbacks | `src/bot/alert_callbacks.py:267-270` | S | After `minutes = int(parts[1])`, add `minutes = min(max(minutes, 1), 10080)` to cap at 7 days. |
14+
| 4 | M3 — World-writable ignore files | `src/alerts/ignore_manager.py:359` | S | Change `os.fchmod(fd, 0o666)` to `os.fchmod(fd, 0o644)`. |
15+
16+
## Phase 2: Data Integrity
17+
18+
| # | Finding | Files | Effort | Description |
19+
|---|---------|-------|--------|-------------|
20+
| 5 | M1 — ChatIdStore non-atomic writes | `src/alerts/manager.py:45-54` | M | Replace bare `json.dump()` with tempfile + `os.replace()` pattern matching `base_mute_manager.py:149-159`. Add `import os, tempfile`. Use `tempfile.mkstemp(dir=..., prefix=".tmp_chatids_", suffix=".json")`, write, `os.fchmod(fd, 0o644)`, `os.replace()`. |
21+
| 6 | M4 — BOT_VERSION stale | `src/bot/health_command.py:23` | S | Replace `BOT_VERSION = "0.9.2"` with dynamic version: `from importlib.metadata import version; BOT_VERSION = version("unraid-monitor-bot")` wrapped in try/except `PackageNotFoundError` with fallback to `"unknown"`. |
22+
23+
## Phase 3: Memory & Scalability
24+
25+
| # | Finding | Files | Effort | Description |
26+
|---|---------|-------|--------|-------------|
27+
| 7 | M5 — CrashTracker unbounded dict keys | `src/monitors/docker_events.py:35-103` | M | Add a `_cleanup_stale()` method that removes entries from `_crashes` where the list is empty, and entries from `_last_escalation`/`_last_recovery_alert` older than 24 hours. Call it from `record_crash()` every 100 calls (counter pattern, like RateLimiter). |
28+
| 8 | M6 — Pattern cache unbounded | `src/monitors/log_watcher.py:26-27` | S | Replace mutable default dict with a module-level `_PATTERN_CACHE: dict = {}` and add `_PATTERN_CACHE_MAX = 64`. In the function, check `if len(_PATTERN_CACHE) > _PATTERN_CACHE_MAX: _PATTERN_CACHE.clear()`. |
29+
30+
## Phase 4: Code Quality & Cleanup
31+
32+
| # | Finding | Files | Effort | Description |
33+
|---|---------|-------|--------|-------------|
34+
| 9 | L1 — Redundant asyncio imports | `src/monitors/resource_monitor.py:178, 205` | S | Delete `import asyncio` at lines 178 and 205. Module-level import at line 1 is sufficient. |
35+
| 10 | L2 — Redundant time import | `src/monitors/log_watcher.py:217` | S | Delete `import time` at line 217. Module-level import at line 4 is sufficient. |
36+
| 11 | L3 — Health command encapsulation | `src/bot/health_command.py`, `src/monitors/*.py` | L | Add public status properties to each monitor: `DockerEventMonitor.is_running`, `LogWatcher.is_running`, `LogWatcher.total_drops`, etc. Update health_command.py to use these instead of private attributes. |
37+
| 12 | L6 — Missing return in telegram_retry | `src/utils/telegram_retry.py:92` | S | Add explicit `return None` after the for loop at line 92 to satisfy mypy. |
38+
| 13 | L8 — parse_duration unbounded | `src/alerts/mute_manager.py:33` | S | After `if value <= 0: return None`, add `if value > 10080: return None` to cap at 7 days (10080 minutes). |
39+
| 14 | L4 — NLProcessor._user_locks TTL | `src/services/nl_processor.py:203-206` | S | Add timestamp tracking to cleanup: only delete locks for users inactive >30 minutes, regardless of lock state. |
40+
41+
## Phase 5: Type Safety (Incremental)
42+
43+
| # | Finding | Files | Effort | Description |
44+
|---|---------|-------|--------|-------------|
45+
| 15 | L7 — mypy errors (191) | Various (30 files) | L | Address incrementally. Priority order: (a) Add `types-PyYAML` to dev dependencies to fix `import-untyped`. (b) Add return type annotations to all factory functions in `src/bot/`. (c) Add generic type parameters to `dict`, `list`, `Task` uses. (d) Fix `InaccessibleMessage` union handling by adding runtime isinstance checks in callback handlers. |
46+
| 16 | L5 — OpenAI prompt caching | `src/services/llm/openai_provider.py` | M | OpenAI auto-caches repeated prompts server-side (no code change needed for basic caching). For explicit caching, would need to use their `cached_tokens` API — defer until cost becomes an issue. |
47+
48+
## Dependencies & Ordering Notes
49+
50+
- **H1 must come first** — it's a security fix that prevents prompt injection via crafted container logs.
51+
- **H2 (kill/decline race)** is independent and can be done in parallel with H1.
52+
- **M1 (atomic writes)** and **M4 (BOT_VERSION)** are independent quick wins.
53+
- **Items 9-14** (Phase 4) are all independent and can be done in parallel or batched into a single commit.
54+
- **L7 (mypy errors)** is best done incrementally over multiple sessions — don't try to fix all 191 at once.
55+
56+
## Quick Wins (S effort + High/Medium severity)
57+
58+
These should be done first within each phase:
59+
1. M2 — Cap mute duration (S) — one line change
60+
2. M3 — Fix file permissions (S) — one line change
61+
3. M4 — Fix BOT_VERSION (S) — two line change
62+
4. M6 — Bound pattern cache (S) — five line change

0 commit comments

Comments
 (0)