Skip to content

Fix integration of stepped graphs with deduplicated values#516

Open
ArchdukeTim wants to merge 1 commit into
Mechanical-Advantage:mainfrom
ArchdukeTim:fix-stepped-integration-dedup
Open

Fix integration of stepped graphs with deduplicated values#516
ArchdukeTim wants to merge 1 commit into
Mechanical-Advantage:mainfrom
ArchdukeTim:fix-stepped-integration-dedup

Conversation

@ArchdukeTim
Copy link
Copy Markdown

Summary

The line graph Integrate filter currently uses the trapezoidal rule for all field types, but LogField.putData drops consecutive duplicate values when storing samples. For stepped fields this collapses constant runs — e.g. a setpoint held at 5 from t=0 to t=1 and then jumping to 10 is stored as [(0, 5), (1, 10)] rather than [(0, 5), (1, 5), (1, 10)].

Trapezoidal integration over the deduplicated data linearly interpolates between the two surviving samples and computes 1 × (5 + 10) / 2 = 7.5, when the true area under the stepped signal is 5 × 1 = 5.

#504 mitigated this for AdvantageKit logs by re-densifying samples at AKIT-loop timestamps before integration. That works because dense samples make trapezoidal converge to rectangle integration, but it does not help non-AdvantageKit logs (no AKIT timestamp field), which still produce incorrect integrals for stepped fields.

This change uses the integration formula appropriate to each graph's interpolation semantics:

  • Stepped (fieldItem.type === "stepped"): rectangle (left-Riemann) rule. Exact regardless of whether duplicates were stored or dropped.
  • Smooth / points: unchanged trapezoidal rule. Exact for linearly-interpolated samples.

Test plan

  • Open a non-AdvantageKit log (plain WPILOG / NetworkTables, no /Timestamp AKIT field) containing a stepped numeric field, e.g. a setpoint that holds at 5 for 1 s then jumps to 10 for 1 s. Add it to a line graph axis with the Integrate filter and confirm the integral reaches 5 at t=1 s and 15 at t=2 s (pre-fix it overshoots, e.g. ~7.5 at t=1 s).
  • Open an AdvantageKit log with a stepped integrated field and confirm no regression versus Fix integration behavior for stepped graphs #504's behavior.
  • Confirm a smooth numeric field still integrates via the trapezoidal rule unchanged.
  • Toggle the visible time range to verify startIndex slicing still works.

🤖 Generated with Claude Code

LogField drops consecutive duplicate values when storing samples, so a
constant-then-step signal like [(0, 5), (1, 5), (1, 10)] is stored as
just [(0, 5), (1, 10)]. The trapezoidal integration filter then linearly
interpolates between the two surviving points and overshoots — e.g.
1*(5+10)/2 = 7.5 instead of the correct 5*1 = 5.

For stepped graphs the value is held constant between samples, so use
the rectangle (left-Riemann) rule. This is exact regardless of whether
duplicate values were stored or dropped, and fixes integration for
non-AdvantageKit logs that don't benefit from the AKIT-timestamp
densification added in Mechanical-Advantage#504.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ArchdukeTim ArchdukeTim force-pushed the fix-stepped-integration-dedup branch from 82e2098 to db59148 Compare May 8, 2026 02:43
@blaze-developer
Copy link
Copy Markdown
Contributor

blaze-developer commented May 8, 2026

This is a promising idea, however we should consider what this means for the already-working case. It would mean that data previously calculated with trapezoidal integration, is now calculated with a Riemann sum. This would work better for log data that is de-duplicated as it intends to, but also changes behavior in duplicated AdvantageKit logs / WPILogs to use a left-riemann by default, which can arguably be considered less accurate for the kind of data we are measuring and the rate at which its being measured.

Also, this would mean that based on the visual choices you use for displaying your graph, the numerical values will be different. Which could be okay, but should be pointed out.

Another question, if this is the right solution, should #504 be reverted to save on the memory of the additional samples?

This could very well be the right solution, and definitely fixes up those edge cases. But as it arises a couple mathematical and design questions / concerns, I wanted to put my chip into the pile and see what others had to think.

Maybe a hybrid solution could be reached where AdvantageKit logs still use Trapezoidal Integration, but other de-duplicated data uses a Riemann sum?

// Trapezoidal integration
integral +=
(data.timestamps[i] - data.timestamps[prevIndex]) * (data.values[i] + data.values[prevIndex]) * 0.5;
if (fieldItem.type === "stepped") {
Copy link
Copy Markdown
Contributor

@blaze-developer blaze-developer May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (fieldItem.type === "stepped") {
if (fieldItem.type === "stepped" && akitTimestamps === undefined) {

I think this could be my proposed hybrid solution? This should probably be documented if it becomes the behavior. (And maybe a comment in the surrounding code?)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change would have to be documented either way as it would change the calculated results in some way, so users will understand why they might get slightly different numbers.

@blaze-developer
Copy link
Copy Markdown
Contributor

blaze-developer commented May 8, 2026

The other option is probably re-duplicating values in all logs, not just AdvantageKit ones, but I'm not sure about the technicalities of that, as AdvantageKit has a frame of reference already for its loop cycle times, and the reduplication would have to be a little arbitrary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants