Fix integration of stepped graphs with deduplicated values#516
Fix integration of stepped graphs with deduplicated values#516ArchdukeTim wants to merge 1 commit into
Conversation
LogField drops consecutive duplicate values when storing samples, so a constant-then-step signal like [(0, 5), (1, 5), (1, 10)] is stored as just [(0, 5), (1, 10)]. The trapezoidal integration filter then linearly interpolates between the two surviving points and overshoots — e.g. 1*(5+10)/2 = 7.5 instead of the correct 5*1 = 5. For stepped graphs the value is held constant between samples, so use the rectangle (left-Riemann) rule. This is exact regardless of whether duplicate values were stored or dropped, and fixes integration for non-AdvantageKit logs that don't benefit from the AKIT-timestamp densification added in Mechanical-Advantage#504. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
82e2098 to
db59148
Compare
|
This is a promising idea, however we should consider what this means for the already-working case. It would mean that data previously calculated with trapezoidal integration, is now calculated with a Riemann sum. This would work better for log data that is de-duplicated as it intends to, but also changes behavior in duplicated AdvantageKit logs / WPILogs to use a left-riemann by default, which can arguably be considered less accurate for the kind of data we are measuring and the rate at which its being measured. Also, this would mean that based on the visual choices you use for displaying your graph, the numerical values will be different. Which could be okay, but should be pointed out. Another question, if this is the right solution, should #504 be reverted to save on the memory of the additional samples? This could very well be the right solution, and definitely fixes up those edge cases. But as it arises a couple mathematical and design questions / concerns, I wanted to put my chip into the pile and see what others had to think. Maybe a hybrid solution could be reached where AdvantageKit logs still use Trapezoidal Integration, but other de-duplicated data uses a Riemann sum? |
| // Trapezoidal integration | ||
| integral += | ||
| (data.timestamps[i] - data.timestamps[prevIndex]) * (data.values[i] + data.values[prevIndex]) * 0.5; | ||
| if (fieldItem.type === "stepped") { |
There was a problem hiding this comment.
| if (fieldItem.type === "stepped") { | |
| if (fieldItem.type === "stepped" && akitTimestamps === undefined) { |
I think this could be my proposed hybrid solution? This should probably be documented if it becomes the behavior. (And maybe a comment in the surrounding code?)
There was a problem hiding this comment.
This change would have to be documented either way as it would change the calculated results in some way, so users will understand why they might get slightly different numbers.
|
The other option is probably re-duplicating values in all logs, not just AdvantageKit ones, but I'm not sure about the technicalities of that, as AdvantageKit has a frame of reference already for its loop cycle times, and the reduplication would have to be a little arbitrary. |
Summary
The line graph Integrate filter currently uses the trapezoidal rule for all field types, but
LogField.putDatadrops consecutive duplicate values when storing samples. For stepped fields this collapses constant runs — e.g. a setpoint held at5fromt=0tot=1and then jumping to10is stored as[(0, 5), (1, 10)]rather than[(0, 5), (1, 5), (1, 10)].Trapezoidal integration over the deduplicated data linearly interpolates between the two surviving samples and computes
1 × (5 + 10) / 2 = 7.5, when the true area under the stepped signal is5 × 1 = 5.#504 mitigated this for AdvantageKit logs by re-densifying samples at AKIT-loop timestamps before integration. That works because dense samples make trapezoidal converge to rectangle integration, but it does not help non-AdvantageKit logs (no AKIT timestamp field), which still produce incorrect integrals for stepped fields.
This change uses the integration formula appropriate to each graph's interpolation semantics:
fieldItem.type === "stepped"): rectangle (left-Riemann) rule. Exact regardless of whether duplicates were stored or dropped.Test plan
/TimestampAKIT field) containing a stepped numeric field, e.g. a setpoint that holds at 5 for 1 s then jumps to 10 for 1 s. Add it to a line graph axis with the Integrate filter and confirm the integral reaches 5 att=1 sand 15 att=2 s(pre-fix it overshoots, e.g. ~7.5 att=1 s).startIndexslicing still works.🤖 Generated with Claude Code