Automate manual post-install steps + install hardening (HAM-120)#79
Open
pbitzer wants to merge 7 commits into
Open
Automate manual post-install steps + install hardening (HAM-120)#79pbitzer wants to merge 7 commits into
pbitzer wants to merge 7 commits into
Conversation
Bring-up repeatedly landed without documented-but-unautomated steps, causing silent runtime degradation caught only months later. Wire the small ones into unified_install as best-effort steps and add matching verify_deployment checks: - HAM-84: pin gpiozero<2.0 on Python 3.7 (2.0 breaks relay.py on Buster) - HAM-118: fetch .googlechat key so state_monitor notifications work - HAM-80: provision the local datasync user (key install still manual) - HAM-71: add the timezone verify check (install step already in bootstrap) Post-install steps run in a new Phase 7 (--skip-postinstall to opt out) and never abort the install on failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review found two unguarded commands (mkdir&&chmod, chown) in the "best-effort" datasync step: under set -e a failure there aborted the whole install at finalize — the opposite of the documented contract. Guard both, and add a sandboxed regression test that proves the Phase 7 steps survive a failing privileged command (fails against the pre-fix code). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fetch_notification_key: capture scp stderr so the warning shows the real cause (auth vs DNS vs unreachable) instead of swallowing it - align the .googlechat host to www.hamma.dev (matches README + verify script) - harden the skip test to assert the skip branch ran, add a positive control Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The repo self-update only pulls when the unit is already on REPO_BRANCH, so the stale 0.3.x value silently skipped updates on 0.4.x-deployed units. Bump to the current version branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold in recurring manual fixes surfaced by a Jira + sensor-log audit: - notifiers installed editable-as-pi so on-sensor git pulls deploy and brokkr survives reboot (HAM-158; same safe pattern as sindri HAM-163) - cleanup_legacy_services: drop retired pre-default units (autossh-hamma, brokkr-hamma) that compete with the -default units (sensor-log #43/#9) - normalize_pi_ownership: best-effort chown of pi paths + /etc/systemd/system mode, fixing the recurring root-owned-from-sudo breakage (sensor-log #78/#11/#33) - verify_deployment: pi-ownership checks + a --full hamma deploy-key check All new install steps are best-effort/non-fatal (regression-tested under set -e). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A dirty unmount leaves an empty /media/pi/DATA?? dir; on next boot udisks mounts the drive at a suffixed path (DATA071) that brokkr's DATA?? glob misses, so science data silently falls back to the SD card. Ship + enable a boot-time oneshot (ordered Before=udisks2.service) that rmdir's the empty orphans first. rmdir is empty-only, so it's a safe no-op once drives are mounted. Previously hand-installed per-sensor (sensor-log #52); this makes it fleet-standard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review-team findings (none blocking) addressed: - oneshot: make the load-bearing ordering explicit (Before=brokkr-hamma-default /multi-user.target) and fix the comment — Before=udisks2.service alone is not the real guarantee (udisks is D-Bus-activated; brokkr is the actual mounter) - verify: git ls-remote deploy-key check now uses BatchMode+ConnectTimeout so a stalled cellular link can't hang a --full verify - fetch_notification_key: chmod 600 the .googlechat webhook token - gpiozero probe: fall back to 99 so an empty result can't silently pin - tests: fix the set -e non-fatal harness so it actually exercises the guards (sudo-flag-stripping shim; LEGACY_SYSTEMD_DIR / PI_OWN_PATHS sandbox overrides) — mutation-verified the cleanup test now fails if a guard is dropped - tests: assert the oneshot's [Install]/WantedBy + DATA?? glob; refresh docstring Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Automates the recurring manual post-install steps that keep getting missed on sensor bring-up (Epic HAM-120), plus a set of install-hardening fixes surfaced by a Jira +
sensor-logaudit. Every new install step is best-effort / non-fatal — a failure logs a warning and never aborts the install.Base:
0.4.x. 7 commits, +671 lines. 106 unit/dry-run tests passing.What's in it
Post-install automation (new
Phase 7,--skip-postinstallto opt out):.googlechatwebhook key sostate_monitornotifications work (nowchmod 600)datasyncaccount (user/group/.ssh//mediaperms; the pull-side key is still installed separately)cleanup_legacy_services— remove retired pre--defaultunits (autossh-hamma.service,brokkr-hamma.service) that compete with the current units (sensor-log Add daily compression plugin for HAMMA trigger data #43/Mount udisks filename #9)normalize_pi_ownership— best-effortchown pi:pion/home/pi/{dev,.ssh,.config}+/etc/systemd/systemmode, fixing the recurring root-owned-from-sudobreakage (sensor-log fix(state_monitor): level-triggered, numeric-robust, observable auto-scrub #78/Python interpreter in sunsaver.py #11/PR Jira Tickets #33)brokkr install:
gpiozero<2.0on Python ≤3.7 (2.0 breaksrelay.pyon Buster); auto-heals on newer Pythonnotifierseditable-as-pi so on-sensorgit pulldeploys and brokkr survives reboot (same safe pattern as the sindri HAM-163 plan)hardware phase:
files/hamma-cleanup-stale-mountpoints.service) thatrmdirs empty/media/pi/DATA??orphans before the drive is mounted, so a dirty unmount can't push it to a suffixed path brokkr'sDATA??glob misses (sensor-log feat: on-demand sensor noise check script #52; previously hand-installed per unit).rmdiris empty-only → safe no-op once mounted.verify_deployment.sh:
check_post_install_config(timezone/HAM-71, gpiozero import,.googlechat, datasync, pi-ownership) and a--fullcheck_hamma_repo_accessdeploy-key check (bounded withBatchMode+ConnectTimeout).install.sh:
REPO_BRANCH0.3.x→0.4.xso the mjolnir-hamma self-update actually runs on 0.4.x units (it only pulls when already on that branch, so the stale value silently skipped updates).Jira sub-tickets
Closeable on merge: HAM-71, HAM-84, HAM-118, HAM-158.
Partially addressed (install hook only; per-sensor backfill remains): HAM-80.
Untracked audit items implemented here (will file under HAM-120): legacy-unit cleanup, ownership normalization, deploy-key verify, udisks oneshot.
Testing
pytest tests/unified→ 106 passing (newtests/unified/test_postinstall.py: dry-run manifest assertions + a sandboxedset -enon-fatal harness that is mutation-verified to fail if a guard is dropped).install.sh --dry-run→ exit 0; no new shellcheck findings in any changed line.verify_deploymentchecks pass live; the oneshot unit passessystemd-analyze verify(rc=0) and is a correct no-op on that healthy unit;rmdir-only safety proven (removes empty, keeps non-empty/mounted).Before=multi-user.target/brokkrordering explicit rather than relying on the non-load-bearingBefore=udisks2.service) is in.Known follow-ups (not blocking)
tests/integration/test-with-systemd.sh) not run here (no Docker on the dev host) — worth one real-Buster run before fleet rollout.🤖 Generated with Claude Code