Skip to content

feat(unparser): support DISTINCT FROM operators in the MySQL dialect#22999

Open
zyuiop wants to merge 4 commits into
apache:mainfrom
zyuiop:feat/unparser-spaceship
Open

feat(unparser): support DISTINCT FROM operators in the MySQL dialect#22999
zyuiop wants to merge 4 commits into
apache:mainfrom
zyuiop:feat/unparser-spaceship

Conversation

@zyuiop

@zyuiop zyuiop commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #22997

Rationale for this change

MySQL does not support the IS NOT DISTINCT FROM syntax, but has an alternative syntax which does the same, the spaceship operator.

What changes are included in this PR?

Adds a DistinctFromStyle attribute to dialects that dictates how IS DISTINCT FROM is unparsed.

Are these changes tested?

Yes

Are there any user-facing changes?

No, we provide a default implementation for DistinctFromStyle

@zyuiop zyuiop changed the title Feat/unparser spaceship feat: support DISTINCT FROM operators in the MySQL dialect Jun 17, 2026
@zyuiop zyuiop changed the title feat: support DISTINCT FROM operators in the MySQL dialect feat(unparser): support DISTINCT FROM operators in the MySQL dialect Jun 18, 2026

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zyuiop
Thanks for the fix here. I think there are a couple of behavior and correctness issues that should be addressed before this lands.

}

/// The style to use when unparsing DISTINCT FROM style expressions
fn distinct_from_style(&self) -> DistinctFromStyle {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this new default changes existing behavior for downstream and custom dialects. Before this PR, a Dialect that did not override this area could still unparse IS DISTINCT FROM and IS NOT DISTINCT FROM using the standard full-text syntax.

With the default now set to Unsupported, those same dialects can start returning not_impl_err! for expressions that previously worked. The MySQL fix only needs a MySQL-specific override, so I think the trait default should preserve the old behavior. For example, this could default to DistinctFromStyle::FullText and only dialects that need a different spelling would override it.

@zyuiop zyuiop Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I have therefore removed the Unsupported enum variant

ast::Expr::IsDistinctFrom(Box::new(l), Box::new(r)),
))),
DistinctFromStyle::DiamondOperators => {
Ok(ast::Expr::Nested(Box::new(ast::Expr::UnaryOp {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the MySQL spelling for IS DISTINCT FROM needs to force the <=> comparison to be parsed as the operand of NOT.

As written, NOT <=> can be ambiguous with MySQL HIGH_NOT_PRECEDENCE, where NOT a <=> b may be parsed as (NOT a) <=> b. That is not equivalent to a IS DISTINCT FROM b.

Could this build the unary expression over a nested spaceship comparison instead? For example: NOT (c1 <=> true), or with outer parentheses as (NOT (c1 <=> true)).

@zyuiop zyuiop Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I have added Expr::Nested inside the UnaryOp

Comment thread datafusion/sql/src/unparser/dialect.rs Outdated
/// DBMS supports `IS (NOT) DISTINCT FROM`
FullText,
/// DMBS supports equivalent operations via `<=>` and `<>`
DiamondOperators,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small naming/comment suggestion: the DiamondOperators comment says this style uses <=> and <>, but the implementation uses <=> and NOT (<=>), which seems right.

Since <> is not null-safe and would not be equivalent to IS DISTINCT FROM, it might be clearer to rename this or update the comment to describe it as a spaceship or null-safe-equality style.

@zyuiop zyuiop Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, I initially intended to use <> and after checking the docs more carefully I noticed it was not equivalent.

I changed the comment and renamed the enum variant to Spaceship.

@zyuiop

zyuiop commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Regarding the PR feedback, should I squash all my changes, or will it be done either way when merging?

@github-actions github-actions Bot added the sql SQL Planner label Jun 18, 2026
@kosiew

kosiew commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

should I squash all my changes, or will it be done either way when merging?

it's be squashed during merge into 1 commit.

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zyuiop
Thanks for the updates. I re-reviewed the latest changes and confirmed the previous blocking comments have been addressed:

  1. Dialect::distinct_from_style now defaults to DistinctFromStyle::FullText, preserving behavior for existing downstream and custom dialects.
  2. MySQL IS DISTINCT FROM now unparses as NOT applied to a nested <=> comparison, which avoids HIGH_NOT_PRECEDENCE ambiguity.
  3. The enum variant has been renamed from DiamondOperators to Spaceship, and the comment no longer implies <> is equivalent.

No blocking findings from this follow-up review.

I left one optional documentation nit below.

Comment thread datafusion/sql/src/unparser/dialect.rs Outdated
@kosiew

kosiew commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Merging with main to get #23053

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unparser: support IsDistinctFrom/IsNotDistinctFrom operator

2 participants