Commit 7f2f78d
authored
feat: fix AVG sliding windows wrong results with NULLs (#22139)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #22138 .
## Rationale for this change
`AVG` used as a window aggregate can return `NaN` (and, for `Decimal` /
`Duration`, panic on integer division by zero) when every value in the
window frame is NULL.
```sql
SELECT i,
AVG(v) OVER (ORDER BY i ROWS BETWEEN CURRENT ROW AND UNBOUNDED
FOLLOWING)
FROM (VALUES (1,1), (2,2), (3,NULL), (4,NULL)) t(i,v);
```
| i | current output | expected (DuckDB/PgSQL) |
|---|----------------|-------------------|
| 1 | 1.5 | 1.5 |
| 2 | 2.0 | 2.0 |
| 3 | **NaN** | **NULL** |
| 4 | **NaN** | **NULL** |
Root cause: sliding-window execution calls `Accumulator::retract_batch`
as rows leave the frame. Once every contributing value has been
retracted, `self.count` drops back to `0` but `self.sum` stays
`Some(0.0)` (or a tiny floating-point residual). `evaluate()` then
computes `sum / 0`, which yields `NaN` on `Float64`, and would panic
with integer division by zero on `DecimalAvgAccumulator` and
`DurationAvgAccumulator`.
The non-sliding aggregation path is unaffected because there `sum`
becomes `Some(_)` only after at least one non-NULL value has been added,
so `count == 0` implies `sum == None`.
## What changes are included in this PR?
`datafusion/functions-aggregate/src/average.rs` — guard all three
affected `evaluate()` implementations with an explicit `count == 0 →
None` short-circuit:
- `AvgAccumulator::evaluate` (Float64)
- `DecimalAvgAccumulator::evaluate` (Decimal32/64/128/256)
- `DurationAvgAccumulator::evaluate` (Duration*)
This matches the idiom already used by sibling retractable accumulators
(`variance.rs` uses an explicit `match self.count` before division;
`sum.rs` uses a `(self.count != 0).then_some(..)` guard).1 parent ccc67e9 commit 7f2f78d
2 files changed
Lines changed: 83 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
519 | 519 | | |
520 | 520 | | |
521 | 521 | | |
522 | | - | |
523 | | - | |
524 | | - | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
525 | 532 | | |
526 | 533 | | |
527 | 534 | | |
| |||
584 | 591 | | |
585 | 592 | | |
586 | 593 | | |
587 | | - | |
588 | | - | |
589 | | - | |
590 | | - | |
591 | | - | |
592 | | - | |
593 | | - | |
594 | | - | |
595 | | - | |
596 | | - | |
597 | | - | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
598 | 611 | | |
599 | 612 | | |
600 | 613 | | |
| |||
670 | 683 | | |
671 | 684 | | |
672 | 685 | | |
673 | | - | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
674 | 694 | | |
675 | 695 | | |
676 | 696 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6456 | 6456 | | |
6457 | 6457 | | |
6458 | 6458 | | |
| 6459 | + | |
| 6460 | + | |
| 6461 | + | |
| 6462 | + | |
| 6463 | + | |
| 6464 | + | |
| 6465 | + | |
| 6466 | + | |
| 6467 | + | |
| 6468 | + | |
| 6469 | + | |
| 6470 | + | |
| 6471 | + | |
| 6472 | + | |
| 6473 | + | |
| 6474 | + | |
| 6475 | + | |
| 6476 | + | |
| 6477 | + | |
| 6478 | + | |
| 6479 | + | |
| 6480 | + | |
| 6481 | + | |
| 6482 | + | |
| 6483 | + | |
| 6484 | + | |
| 6485 | + | |
| 6486 | + | |
| 6487 | + | |
| 6488 | + | |
| 6489 | + | |
| 6490 | + | |
| 6491 | + | |
| 6492 | + | |
| 6493 | + | |
| 6494 | + | |
| 6495 | + | |
| 6496 | + | |
| 6497 | + | |
| 6498 | + | |
| 6499 | + | |
| 6500 | + | |
| 6501 | + | |
| 6502 | + | |
| 6503 | + | |
| 6504 | + | |
| 6505 | + | |
| 6506 | + | |
6459 | 6507 | | |
6460 | 6508 | | |
6461 | 6509 | | |
| |||
0 commit comments