Fix InfluxDB 1.x mean/sum and grouped query double-aggregation by kizniche · Pull Request #1523 · kizniche/Mycodo

kizniche · 2026-03-30T01:38:16Z

Fixes InfluxDB 1.8.x crash bug when using MEAN or SUM in Flux queries. Previously, Mycodo worked around this by substituting MEDIAN for MEAN. Now, for InfluxDB 1.x, querying MEAN returns the actual mean and SUM returns the actual sum.

Changes Made

MEAN/SUM on InfluxDB 1.x: Removed the |> median() workaround for MEAN and the raw-point summing workaround for SUM. Raw points are now fetched and _manual_calculate_mean() / _manual_calculate_sum() compute the correct result client-side, avoiding the Flux crash (influxdata/influxdb#21649).
Grouped queries on InfluxDB 1.x: Uses Flux window()+reduce()+map() server-side to compute per-window means without calling aggregateWindow(fn: mean), keeping server-side processing (less data to the client) while avoiding the crash.
Callers simplified: output_sec_on, average_past_seconds, average_start_end_seconds, and sum_past_seconds no longer contain version-specific workaround logic — they all consume the single aggregated value that query_flux/query_string now always returns.
Dead code removed: _manual_aggregate_mean() function and its unreachable call site removed. All TODO: Change median to mean when issue is fixed comments removed.
Logging improved: logger.error for the SUM workaround demoted to logger.debug.
Trailing whitespace fixed in influxdb_get_first_point().

Testing

14 unit tests added in mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py covering _manual_calculate_mean and _manual_calculate_sum (empty input, None-skipping, single point, multiple points, correct scalar output — no live InfluxDB required).
All 14 tests pass.

📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Updates Mycodo’s InfluxDB Flux querying to work around InfluxDB 1.8.10 Flux panics for mean()/sum() and to re-enable mean-based downsampling/grouping behavior for 1.x.

Changes:

Replaces the prior InfluxDB 1.x aggregateWindow(..., fn: median) workaround with a manual mean/sum fallback path for 1.x results.
Adds helper structures/functions to compute mean/sum (and grouped mean) outside of Flux’s mean()/sum() for InfluxDB 1.x.
Simplifies several callers (output_sec_on, average_*, sum_past_seconds) to assume the query layer returns a single aggregated value.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/fc2c2777-4ee3-444f-a9dc-78e1e6eada6a Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

…d timestamps Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/34bde69d-5ac8-47d4-9798-bee3ae0942fb Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…plicit value Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/5cd03b98-919a-495c-944b-0f0ecf04d320 Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

kizniche · 2026-03-30T02:08:24Z

@copilot apply changes based on the comments in this thread

…anual_calculate_sum Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot · 2026-03-30T02:12:24Z

@copilot apply changes based on the comments in this thread

Done in commit 3bb9c4c. Added mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py with 24 unit tests for all three helpers — no live InfluxDB or Flask context needed. Covers: (1) group_sec with no value via _manual_aggregate_mean (epoch alignment, window-end timestamps, multi-window, boundary, sort, timezone), (2) value="MEAN" and value="SUM" via _manual_calculate_mean/_manual_calculate_sum, and (3) combination case confirming those return a single scalar (not per-window).

kizniche · 2026-03-30T02:13:26Z

@copilot Install the proper dependencies with the requirements.txt and requirements-testing.txt files in the install directory, remove unnecessary mocks in the tests you made, and rerun the tests.

…er deps now installed) Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/d45eb174-543a-4d8c-8bd4-df1c294b5a00 Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot · 2026-03-30T02:15:46Z

@copilot Install the proper dependencies with the requirements.txt and requirements-testing.txt files in the install directory, remove unnecessary mocks in the tests you made, and rerun the tests.

Done in commit f75d504. Installed dependencies from install/requirements.txt and install/requirements-testing.txt, removed all the sys.modules mocks from the test file, and confirmed all 24 tests still pass cleanly.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

mycodo_influxdb
- Triggering command: /usr/bin/python3 python3 -m pytest mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py -v (dns block)
none
- Triggering command: /usr/bin/python3 python3 -m pytest mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kizniche · 2026-03-30T02:26:04Z

@copilot apply changes based on the comments in this thread

…whitespace Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/b6c3ab53-7015-4636-9dba-2a0841a0589f Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot · 2026-03-30T02:30:23Z

@copilot apply changes based on the comments in this thread

Done in commit acaa9f4:

Removed the dead if group_sec and not value and not server_side_windowed: branch — server_side_windowed is always True in that code path, so it was unreachable.
Removed _manual_aggregate_mean() since nothing calls it anymore.
Fixed trailing whitespace in influxdb_get_first_point().
Updated tests accordingly (14 tests remain, all passing).

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

mycodo_influxdb
- Triggering command: /usr/bin/python3 python3 -m pytest mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py -v (dns block)
none
- Triggering command: /usr/bin/python3 python3 -m pytest mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

kizniche · 2026-03-30T02:33:34Z

@copilot Do one more code review to ensure this PR solves the main issue and conforms to the original PR summary:
InfluxDB 1.8.x has a crash bug if MEAN or SUM are used. Mycodo used to work around the issue by substituting with MEDIAN. For InfluxDB 1.8.x, querying the MEAN will now return the mean. There are minor code and logging improvements too, notably moving the manual calculation for SUM so that callers don't need to participate in the workaround.

Copilot · 2026-03-30T02:35:59Z

@copilot Do one more code review to ensure this PR solves the main issue and conforms to the original PR summary:
InfluxDB 1.8.x has a crash bug if MEAN or SUM are used. Mycodo used to work around th...

The PR correctly solves the stated issue. Here's a summary of how each requirement is met:

InfluxDB 1.8.x crash bug for MEAN/SUM: Fixed. The old workaround (|> median() for MEAN, and returning all raw points for SUM) is replaced:

value="MEAN" on 1.x: raw points are fetched, then _manual_calculate_mean() computes the true mean client-side.
value="SUM" on 1.x: raw points are fetched, then _manual_calculate_sum() computes the true sum client-side.
Neither |> mean() nor |> sum() are sent to InfluxDB 1.x, avoiding the crash.

Grouped queries (group_sec) on 1.x: Uses Flux window()+reduce()+map() server-side to compute per-window means without calling aggregateWindow(fn: mean). This keeps server-side processing (less data to the Pi) and avoids the crash.

Callers simplified: output_sec_on, average_past_seconds, average_start_end_seconds, and sum_past_seconds no longer need version-specific workaround logic — they all just consume the single aggregated value that query_flux/query_string now always returns.

Code quality: Dead code (_manual_aggregate_mean, its unreachable call site, all TODO: Change median to mean comments) removed. logger.error for SUM demoted to logger.debug. Trailing whitespace fixed.

Tests: 14 unit tests cover _manual_calculate_mean and _manual_calculate_sum (empty input, None-skipping, single point, multiple points, correct scalar output). All passing.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

mycodo_influxdb
- Triggering command: /usr/bin/python3 python3 -m pytest mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py -v (dns block)
none
- Triggering command: /usr/bin/python3 python3 -m pytest mycodo/tests/software_tests/test_influxdb/test_manual_aggregation.py -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Jade and others added 2 commits February 2, 2026 21:02

Fix mean and sum on influxdb 1.x

0470ee0

Update mycodo/utils/influx.py

263ac77

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

kizniche requested a review from Copilot March 30, 2026 01:38

Copilot started reviewing on behalf of kizniche March 30, 2026 01:38 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread mycodo/utils/influx.py

Comment thread mycodo/utils/influx.py Outdated

Comment thread mycodo/utils/influx.py Outdated

Comment thread mycodo/utils/influx.py

Comment thread mycodo/utils/influx.py Outdated

Comment thread mycodo/utils/influx.py

Copilot started work on behalf of kizniche March 30, 2026 01:47 View session

Fix double aggregation for InfluxDB 1.x grouped queries

0a30268

Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/fc2c2777-4ee3-444f-a9dc-78e1e6eada6a Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot AI changed the title ~~Fix influxdb 1x mean~~ Fix InfluxDB 1.x mean/sum and grouped query double-aggregation Mar 30, 2026

Copilot finished work on behalf of kizniche March 30, 2026 01:50

Copilot started work on behalf of kizniche March 30, 2026 01:56 View session

Fix _manual_aggregate_mean to use epoch-aligned windows and window-en…

c27c11b

…d timestamps Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/34bde69d-5ac8-47d4-9798-bee3ae0942fb Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot finished work on behalf of kizniche March 30, 2026 01:58

kizniche requested a review from Copilot March 30, 2026 01:59

Copilot started reviewing on behalf of kizniche March 30, 2026 02:00 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread mycodo/utils/influx.py

Comment thread mycodo/utils/influx.py

Copilot started work on behalf of kizniche March 30, 2026 02:05 View session

Fix _manual_aggregate_mean only applied when group_sec set without ex…

521980c

…plicit value Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/5cd03b98-919a-495c-944b-0f0ecf04d320 Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot finished work on behalf of kizniche March 30, 2026 02:07

Copilot started work on behalf of kizniche March 30, 2026 02:08 View session

Add unit tests for _manual_aggregate_mean, _manual_calculate_mean, _m…

3bb9c4c

…anual_calculate_sum Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot finished work on behalf of kizniche March 30, 2026 02:12

Copilot started work on behalf of kizniche March 30, 2026 02:13 View session

Remove manual sys.modules mocks from test_manual_aggregation.py (prop…

f75d504

…er deps now installed) Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/d45eb174-543a-4d8c-8bd4-df1c294b5a00 Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot finished work on behalf of kizniche March 30, 2026 02:16

kizniche requested a review from Copilot March 30, 2026 02:19

Copilot started reviewing on behalf of kizniche March 30, 2026 02:20 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread mycodo/utils/influx.py Outdated

Comment thread mycodo/utils/influx.py Outdated

Copilot started work on behalf of kizniche March 30, 2026 02:26 View session

Remove dead _manual_aggregate_mean branch and function; fix trailing …

acaa9f4

…whitespace Agent-Logs-Url: https://github.com/kizniche/Mycodo/sessions/b6c3ab53-7015-4636-9dba-2a0841a0589f Co-authored-by: kizniche <838427+kizniche@users.noreply.github.com>

Copilot finished work on behalf of kizniche March 30, 2026 02:30

Copilot started work on behalf of kizniche March 30, 2026 02:33 View session

Copilot finished work on behalf of kizniche March 30, 2026 02:36

Uh oh!

Conversation

kizniche commented Mar 30, 2026 • edited by Copilot AI Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes Made

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

kizniche commented Mar 30, 2026

Uh oh!

Copilot AI commented Mar 30, 2026

Uh oh!

kizniche commented Mar 30, 2026

Uh oh!

Copilot AI commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

kizniche commented Mar 30, 2026

Uh oh!

Copilot AI commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

kizniche commented Mar 30, 2026

Uh oh!

Copilot AI commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kizniche commented Mar 30, 2026 •

edited by Copilot AI

Loading

Copilot AI commented Mar 30, 2026 •

edited

Loading

Copilot AI commented Mar 30, 2026 •

edited

Loading

Copilot AI commented Mar 30, 2026 •

edited

Loading