Skip to content

Release v0.7.0#78

Merged
mrustl merged 27 commits into
masterfrom
dev
Jun 19, 2026
Merged

Release v0.7.0#78
mrustl merged 27 commits into
masterfrom
dev

Conversation

@mrustl

@mrustl mrustl commented Jun 18, 2026

Copy link
Copy Markdown
Member

wasserportal 0.7.0 2026-06-18

This release extends the ThingsBoard integration shipped in 0.6.0 so it also
works against self-hosted ThingsBoard instances and can push the entire
Wasserportal groundwater archive
, not just the five-station demo.

Self-hosted ThingsBoard authentication

  • New tb_login() plus username/password (JWT) authentication across all
    ThingsBoard tenant-API helpers (tb_setup_devices(), tb_get_device_id(),
    tb_list_device_telemetry_keys(), tb_delete_device_telemetry()).
    Self-hosted Community Edition has no account-level API keys (a ThingsBoard
    Cloud convenience), so it is reached via POST /api/auth/login → a
    short-lived JWT sent as X-Authorization: Bearer <token>. Set TB_USERNAME
    • TB_PASSWORD (these win over TB_API_KEY when both are present); the
      Cloud API-key path keeps working unchanged. The thingsboard-push.yaml
      workflow reads the two new credentials from the matching repository secrets.
  • tb_login() retries on {408, 429, 500, 502, 503, 504} with up to 4 tries,
    so tb_setup_devices() survives a cold-start 5xx from a self-hosted
    ThingsBoard sitting behind nginx / a load balancer.
  • tb_auth_header() warns when only one of TB_USERNAME / TB_PASSWORD
    is set and a leftover TB_API_KEY silently falls back to the Cloud path,
    so the common "one secret missing" misconfiguration is called out instead
    of failing obscurely downstream.
  • Non-2xx responses surface a server response-body excerpt (≤ ~800 chars) in
    R errors and retry messages. Stock ThingsBoard does not echo credentials,
    but operators whose reverse proxy echoes request fields should mask secrets
    in their CI logs.

Push every station, not just the demo five

  • TB_MAX_DEVICES=0 lifts the 5-device demo cap and pushes every selected
    station -- now wired into the workflow as a repository secret and a
    workflow_dispatch input.
  • TB_STATION_SCOPE chooses which groundwater stations the auto-pick
    considers: both (default -- level and quality, the proven demo set),
    any (level or quality), gwl / gwq (has that series, possibly both)
    or gwl-only / gwq-only (has only that series). TB_TELEMETRY_TYPES
    still decides which series get pushed per station.
  • Station scoring counts distinct quality parameters once via split() +
    vapply() (returning a plain named integer vector), so scoring the full
    several-hundred-station pool stays fast; the selection diagnostic now labels
    the strict intersect row clearly against the relaxed per-series counts.
  • The push script now validates its numeric TB_* variables up front
    (TB_MAX_DEVICES, TB_HISTORY_DAYS, TB_CHUNK_SIZE, TB_THROTTLE_SECONDS,
    TB_MAX_ACTIVE) and aborts with a clear message on a non-numeric value,
    instead of letting an NA crash mid-push after every device attribute set
    was already uploaded. The message flags the usual cause -- .Renviron does
    not support inline # comments.

For a full self-hosted push set TB_PLAN=ce, TB_MAX_DEVICES=0 and
TB_STATION_SCOPE=any. Mind the volume -- several hundred devices over the
full archive is millions of data points; start with a bounded
TB_HISTORY_DAYS to validate the run.

Documentation & packaging

  • Update the Kompetenzzentrum Wasser Berlin (KWB) author logo on the pkgdown
    site to the new brand asset.
  • The package now requires R ≥ 4.1.0 (Depends), reflecting the native
    |> pipe used in the ThingsBoard code; the unused LazyData field was
    removed.

This change is Reviewable

claude and others added 12 commits June 17, 2026 14:51
ThingsBoard Community Edition has no account-level API keys (a Cloud-only
convenience), so its tenant REST API can only be reached via
POST /api/auth/login (username + password), which returns a JWT sent as
X-Authorization: Bearer <token>.

- Add tb_login() (exported) and an internal tb_auth_header() resolver
- Route tb_setup_devices(), tb_get_device_id(),
  tb_list_device_telemetry_keys() and tb_delete_device_telemetry() through
  tb_auth_header(): username/password win over the API key when both are
  set; the existing ApiKey path (ThingsBoard Cloud) is unchanged
- Read TB_USERNAME / TB_PASSWORD from the environment; wire the two new
  secrets into thingsboard-push.yaml and the push-script preflight
- Document both auth routes in the demo vignette and NEWS; add tb_login.Rd
  and update the man pages / NAMESPACE for the changed signatures

The device-token telemetry push (/api/v1/{token}/telemetry) is identical
on every edition and stays untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
Wire two new knobs into inst/scripts/push_to_thingsboard.R and the
thingsboard-push.yaml workflow (repository secret + workflow_dispatch
input each), so non-demo / full pushes no longer need code edits:

- TB_MAX_DEVICES=0 lifts the 5-device demo cap and pushes every candidate
  station (previously TB_MAX_DEVICES was not exposed in the workflow at
  all, so CI was stuck at the default of 5).
- TB_STATION_SCOPE selects the auto-pick pool: "both" (default, level AND
  quality -- unchanged proven demo set), "any" (level OR quality), or
  "gwl" / "gwq" (only that series). TB_TELEMETRY_TYPES still decides which
  series get pushed per station.

Also count distinct gwq parameters once via tapply() instead of rescanning
the gwq table per station, so scoring the larger pool stays fast. Document
the knobs in the script header, the demo vignette and NEWS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
TB_STATION_SCOPE gained two values so the auto-pick can target stations
that have ONLY one series:

- gwl-only : has level data but NOT quality (setdiff)
- gwq-only : has quality data but NOT level

The existing gwl / gwq stay inclusive ("has that series, may also have the
other"); both / any are unchanged. The selection summary now logs the
"only gwl (no gwq)" and "only gwq (no gwl)" counts. Documented in the
script header, workflow input, demo vignette and NEWS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
Point the Kompetenzzentrum Wasser Berlin gGmbH (KWB) author logo at the
new brand asset (logos.kompetenz-wasser.io/KWB_Logo_M_Blau_RGB.svg).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
v0.6.0 was already tagged/released (PR #75) without the work on this
branch. Bump Version 0.6.0 -> 0.7.0 (minor: new exported tb_login() and
username/password auth are backward-compatible features, not just fixes)
and split NEWS.md: the post-0.6.0 changes (ThingsBoard username/password
JWT auth, configurable TB_MAX_DEVICES / TB_STATION_SCOPE incl. gwl-only /
gwq-only, KWB pkgdown logo) move from the already-released 0.6.0 section
into a new 0.7.0 section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Match the style of the existing 0.6.0 entries (which all end with a
period).

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
- Add tb_default_host() helper that treats TB_HOST="" (set-but-empty)
  as unset, falling back to the public-cloud default instead of silently
  hitting https://thingsboard.cloud. Routed through all nine callsites.
- Forward the already-resolved auth header from
  tb_delete_device_telemetry() to the nested
  tb_list_device_telemetry_keys() call, eliminating a redundant
  POST /api/auth/login round-trip on the username/password path.
- Replace stop_formatted() with bare stop() in tb_auth_header()'s
  "no credentials" branch (no sprintf args were being passed).

Regenerated man/*.Rd for the affected helpers.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
- Add Depends: R (>= 4.1.0): inst/scripts code and R/push_to_thingsboard.R
  use the native pipe |>, which R CMD check otherwise flags as an
  undeclared dependency.
- Remove LazyData: true (no data/ directory, R CMD build omitted it).
- NEWS: set the wasserportal 0.7.0 release date to 2026-06-18.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
Declare R (>= 4.1.0), drop LazyData, set 0.7.0 release date
Comment thread R/push_to_thingsboard.R Outdated
Comment thread inst/scripts/push_to_thingsboard.R
Comment thread inst/scripts/push_to_thingsboard.R Outdated
Comment thread R/push_to_thingsboard.R
Comment thread R/push_to_thingsboard.R
github-actions Bot and others added 2 commits June 18, 2026 11:02
If only one of TB_USERNAME / TB_PASSWORD is set (typical misconfiguration
when a workflow secret is missing) and TB_API_KEY happens to be present,
tb_auth_header() silently took the API-key path. Emit a warning so the
user sees the misconfiguration instead of only the 'auth: account API
key' log line.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
The "with gwl AND gwq" row used `master_intersect` (strict — present in
both gwl and gwq master files) while the other counts used `master_union`,
so the numbers did not add up the way a reader would expect when the two
master files don't perfectly overlap. Rename the label to
"in both master files AND has both series" so the asymmetry is visible,
and add an inline note explaining the intentional strict semantics.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
@mrustl

mrustl commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

@claude bitte am ende noch NEWS.md updaten

@claude

claude Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @mrustl's task in 1m 51s —— View job


Done

  • Read current NEWS.md to understand existing style
  • Check recent commits on dev for changes not yet reflected in NEWS
  • Update NEWS.md with the new entries
  • Commit and push to dev (25bfd9d)

Added two bullets to the 0.7.0 section of NEWS.md covering the PR-review follow-ups already landed on dev:

  • tb_auth_header() now warns when only one of TB_USERNAME / TB_PASSWORD is set and a leftover TB_API_KEY silently picks the API-key path (commit bd4640a).
  • The station-selection diagnostic in inst/scripts/push_to_thingsboard.R renames the strict-intersect row to in both master files AND has both series so the asymmetry against the per-series union rows is visible (commit 27a1a66).

The earlier 0.7.0 bullets (JWT auth, configurable station scope, KWB logo refresh) are untouched. The pending review threads about tapply()vapply(split(...)) and tb_login() retry widening are tracked in their own Claude jobs; once those land you may want a follow-up NEWS bullet for them too.
· Branch dev

github-actions Bot and others added 3 commits June 18, 2026 11:09
Add 0.7.0 entries for the two PR-review follow-ups landed in bd4640a
and 27a1a66:

* tb_auth_header() now warns when only one of TB_USERNAME / TB_PASSWORD
  is set and a leftover TB_API_KEY silently picks the API-key path
* The station-selection diagnostic in inst/scripts/push_to_thingsboard.R
  renames the strict (intersect of master_gwl and master_gwq) row to
  'in both master files AND has both series' so the asymmetry against
  the per-series union rows is visible

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
tapply() returns a 1-D array, not a named integer vector. Switching to
split() + vapply() keeps the same arithmetic but yields a plain named
integer vector and matches the idiom already used in
R/get_stations.R:72 and R/inspect_gh_pages_zips.R:114.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
Widen tb_login()'s retry predicate to {408, 429, 500, 502, 503, 504} and
bump max_tries to 4L, matching tb_push_station_telemetry(). The httr2
default only retries 429/503, so a self-hosted ThingsBoard behind nginx
/ a load balancer could briefly return 500/502/504 on cold starts and
abort tb_setup_devices() on first try. /api/auth/login is idempotent so
retrying is safe.

Add a @section "Credentials in error output" to the roxygen flagging
that tb_error_body() excerpts surface in R errors and req_retry()
stderr -- relevant only if a self-hosted instance echoes request fields
back in error responses.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
@mrustl

mrustl commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

@claude check if .Rd files are up to date and update NEWS.md

@claude

claude Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @mrustl's task in 4m 4s —— View job


Done

  • Checked .Rd files — all up to date

    • 9 exported tb_* helpers + inspect_gh_pages_zips each have a matching man/*.Rd; the 4 internal helpers (tb_default_host, tb_auth_header, tb_get_or_create_device, tb_get_device_access_token) are correctly @noRd / @keywords internal
    • NAMESPACE exports line up with all @export tags in R/
    • man/tb_login.Rd reflects the latest roxygen from c37d70f (retry-widening details + Credentials in error output section)
    • No R/ changes since the last man/*.Rd regeneration — no drift
  • Updated NEWS.md with two follow-up bullets for changes that landed on dev after the prior NEWS update — committed and pushed as d19f6b5:

    • Refreshed the station-scope bullet: via tapply()via split() + vapply() (returns a plain named integer vector, matches the idiom in R/get_stations.R / R/inspect_gh_pages_zips.R) — covers commit d96f576
    • Added a new bullet for tb_login() retry-widening: predicate now {408, 429, 500, 502, 503, 504}, max_tries = 4, matching tb_push_station_telemetry(), plus the documented trade-off that tb_error_body() excerpts surface in R errors / req_retry() stderr — covers commit c37d70f

The earlier 0.7.0 bullets (JWT auth, configurable station scope, KWB logo, partial-JWT warning, intersect-row label) are untouched.
· Branch dev

Adds two follow-up entries to the v0.7.0 NEWS section that landed on
dev after the prior NEWS update:

- tb_login() retry predicate widened to {408, 429, 500, 502, 503, 504}
  with max_tries bumped to 4, plus the "Credentials in error output"
  trade-off note (commit c37d70f).
- The station-scoring helper now uses split() + vapply() and returns
  a plain named integer vector instead of the 1-D array that the
  intermediate tapply() implementation produced (commit d96f576).

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
@mrustl

mrustl commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

@claude use Opus 4.8 as new default

@claude

claude Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @mrustl's task in 55s —— View job


Tasks

  • Locate where the Claude model default is set
  • Identify required change (claude-opus-4-7claude-opus-4-8)
  • Apply edits in .github/workflows/*.yamlblocked by permissions

Where the default lives

The default model is pinned in two workflow files:

  • .github/workflows/claude.yaml:38claude_args: '--model claude-opus-4-7'
  • .github/workflows/claude-code-review.yaml:40claude_args: '--model claude-opus-4-7 --allowed-tools "..."'

Both need claude-opus-4-7claude-opus-4-8.

Why I can't push this myself

I'm unable to modify files under .github/workflows/ — the GitHub App that runs me does not have the workflows permission, so any push that touches a workflow file is rejected by the API. You can find more context in the FAQ.

Patch to apply locally

--- a/.github/workflows/claude.yaml
+++ b/.github/workflows/claude.yaml
@@ -35,4 +35,4 @@ jobs:
         uses: anthropics/claude-code-action@v1
         with:
           claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
-          claude_args: '--model claude-opus-4-7'
+          claude_args: '--model claude-opus-4-8'
--- a/.github/workflows/claude-code-review.yaml
+++ b/.github/workflows/claude-code-review.yaml
@@ -37,4 +37,4 @@ jobs:
             - Test coverage
             - Documentation (roxygen2) completeness
             - Consistency with the existing codebase
-          claude_args: '--model claude-opus-4-7 --allowed-tools "mcp__github_inline_comment__create_inline_comment,Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"'
+          claude_args: '--model claude-opus-4-8 --allowed-tools "mcp__github_inline_comment__create_inline_comment,Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"'

One-liner to apply both edits:

sed -i 's/claude-opus-4-7/claude-opus-4-8/g' \
  .github/workflows/claude.yaml \
  .github/workflows/claude-code-review.yaml

Once you commit and push that (or merge a small PR with just these two changes), every subsequent @claude invocation on this repo will run on Opus 4.8.
· Branch dev

Validate TB_MAX_DEVICES / TB_HISTORY_DAYS / TB_CHUNK_SIZE /
TB_THROTTLE_SECONDS / TB_MAX_ACTIVE through a num_env() helper that aborts
with a clear message when the value is not a number, instead of letting an
NA from as.integer()/as.numeric() crash a downstream `if (x > 0)` only
after every device attribute set has already been pushed.

Common trigger: .Renviron does not support inline `# comments`, so
`TB_HISTORY_DAYS = 7  # foo` coerces to NA; the error message calls this
out explicitly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
claude and others added 2 commits June 18, 2026 13:20
The Depends bump and LazyData removal (commit d1e5833) changed DESCRIPTION
but had no changelog entry; add one so NEWS matches the GitHub release notes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
Fail-fast on bad TB_* env vars + document R (>= 4.1.0) in NEWS
Comment thread inst/scripts/push_to_thingsboard.R Outdated
Comment thread R/push_to_thingsboard.R
Comment thread R/push_to_thingsboard.R Outdated
Comment thread inst/scripts/push_to_thingsboard.R
github-actions Bot and others added 6 commits June 18, 2026 13:33
num_env() aborts on NA at parse time, so the is.na() branch in the
no_limit check could never fire. Simplify to the <= 0L test that
actually does work.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
Factor the body into an internal tb_list_device_telemetry_keys_impl()
that takes the pre-resolved X-Authorization header. The public function
no longer exposes auth=, so the exported signature stays consistent with
the other tb_*() helpers (credentials in, header resolved internally),
and there is no longer a silent precedence where auth= combined with
api_key/JWT credentials would have ignored the latter without warning.

tb_delete_device_telemetry() calls the internal worker directly, so the
one-round-trip saving for the chained delete -> list case is preserved.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
Two follow-ups to the station-selection block in
inst/scripts/push_to_thingsboard.R:

* Add a one-line message() when stations have gwl/gwq data but are
  missing from both master files. Every scope intersects its candidate
  set with master_union, so these orphans are silently dropped from the
  candidate pool; without the message a master/data drift was invisible
  in the diagnostic counts.

* Add an "in either master AND has both series" row to the diagnostic
  (master_union intersect ids_gwl intersect ids_gwq) so the row-sum
  identity with_gwl + with_gwq - both = only_gwl + only_gwq + both
  actually holds for readers scanning the message. Rename the existing
  strict row to "strict: in both masters AND both series" and append a
  "(strict: master_intersect)" annotation so the intentional asymmetry
  against master_union stays visible.

Co-authored-by: Michael Rustler <mrustl@users.noreply.github.com>
anthropics/claude-code-action@v1 blocks non-human actors by default, so a
PR opened or synchronized by claude[bot] failed with "Workflow initiated by
non-human actor: claude (type: Bot)". Add allowed_bots: "*" so bot-driven
PR events (e.g. the claude[bot] sync PRs on dev) run the review instead of
failing the check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5oA36vi7m6DSkah3tc5eo
Allow bot-initiated PRs in Claude Code Review workflow
@mrustl mrustl merged commit c8965aa into master Jun 19, 2026
2 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants