Skip to content

Return None for cardinality overflow#22309

Merged
kosiew merged 3 commits into
apache:mainfrom
jx2lee:jx2lee/cardinality-overflow
May 28, 2026
Merged

Return None for cardinality overflow#22309
kosiew merged 3 commits into
apache:mainfrom
jx2lee:jx2lee/cardinality-overflow

Conversation

@jx2lee

@jx2lee jx2lee commented May 17, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #22232
(cc @Dandandan )

Rationale for this change

Prevent Interval::cardinality() from overflowing on the full i64 range.

What changes are included in this PR?

  • use checked_add(1) instead of + 1
  • add a regression test for the full i64 range

Are these changes tested?

  • cargo fmt --all --check
  • cargo test -p datafusion-expr-common test_cardinality_full_i64_range_does_not_overflow --lib
  • cargo clippy --all-targets --all-features -- -D warnings could not run locally because cmake is not installed

Are there any user-facing changes?

  • No

@github-actions github-actions Bot added the logical-expr Logical plan and expressions label May 17, 2026
@kumarUjjawal

Copy link
Copy Markdown
Contributor

Thank you @jx2lee for working on this. Can you add the original test from the issue to make sure the change is working.

  query I
  SELECT * FROM (VALUES (1)) AS t(x)
  WHERE x BETWEEN -9223372036854775808 AND 9223372036854775807
  ----
  1

@github-actions github-actions Bot added the sqllogictest SQL Logic Tests (.slt) label May 19, 2026
@jx2lee jx2lee force-pushed the jx2lee/cardinality-overflow branch from 4711b0b to 98ce8fc Compare May 19, 2026 21:38
01)ProjectionExec: expr=[c1@0 >= 2 AND c1@0 <= 3 as select_between_data.c1 BETWEEN Int64(2) AND Int64(3)]
02)--DataSourceExec: partitions=1, partition_sizes=[1]

# regression test: full i64 BETWEEN bounds should not overflow

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jx2lee
Thanks for the fix here. I verified the core invariant around Interval::cardinality() and the new checked_add at the final inclusive + 1 layer correctly returns None instead of panicking or wrapping when the inclusive cardinality exceeds u64::MAX. This covers the full i64 range while preserving existing max-but-representable ranges.

I just had one small non-blocking suggestion.

}

#[test]
fn test_cardinality_full_i64_range_does_not_overflow() -> Result<()> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch on the overflow edge case. One additional regression test that could be helpful would be a sibling case for ScalarValue::UInt64(Some(0))..=ScalarValue::UInt64(Some(u64::MAX)) returning None.

The fix applies more generally to any integer or temporal range where distance == u64::MAX, so adding the unsigned boundary case would help document that broader invariant and guard against future regressions there as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kosiew Thanks for reviewing ! I added testcase for u64

@jx2lee jx2lee force-pushed the jx2lee/cardinality-overflow branch from dbd8bbc to 615665c Compare May 27, 2026 14:52
@kosiew kosiew added this pull request to the merge queue May 28, 2026
Merged via the queue into apache:main with commit c48e993 May 28, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

panic: interval analysis Interval::cardinality adds past u64::MAX for BETWEEN spanning full i64 range

3 participants