Skip to content

fix(extensions)!: correct return type for subtract:date_iday operation#1029

Merged
nielspardon merged 1 commit into
substrait-io:mainfrom
nielspardon:par-datetime-substract
May 12, 2026
Merged

fix(extensions)!: correct return type for subtract:date_iday operation#1029
nielspardon merged 1 commit into
substrait-io:mainfrom
nielspardon:par-datetime-substract

Conversation

@nielspardon
Copy link
Copy Markdown
Member

@nielspardon nielspardon commented Apr 1, 2026

BREAKING CHANGE: changes the return type of the subtract:date_iday function from date to precision_timestamp

Fix the return type of the subtract function when subtracting an interval_day from a date. Since interval_day is an interval of days to subseconds the operation should return a precision_timestamp type, not a date type.

This corrects the function signature in functions_datetime.yaml where date + interval_day was incorrectly specified to return date instead of precision_timestamp.

This is similar to #1007 and was originally reported in #576.

This also fixes the subtract:ts_iday test cases to use a precision type parameter for iday.


This change is Reviewable

@nielspardon nielspardon marked this pull request as draft April 1, 2026 17:38
@nielspardon nielspardon force-pushed the par-datetime-substract branch 3 times, most recently from fa7469c to b42694a Compare April 1, 2026 17:52
@nielspardon nielspardon marked this pull request as ready for review April 1, 2026 17:52
- name: y
value: interval_day<P>
return: date
return: precision_timestamp<P>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait what? The existing change looks correct to my eyes.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you subtract 1 day, 2 hours, 3 seconds, 4 microseconds from a date what do you expect to be returned? Certainly not a full date anymore but a timestamp and given that interval_day has a precision for subseconds it would be a precision timestamp of the same precision as interval_day.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing that out but that opens even more questions. If that is the case, we should update all other things? For instance, timestamp and timestamp_tz are at microsecond precision. So if you do math with interval_day<P>, it should be the higher precision of max(6,P)?

or,

if (P <= 6) then (timestamp) else (precision_timestamp<P>) 

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I already have this PR open for removing the old timestamp, timestamp_tz and time types which should simplify such scenarios somewhat. We can check if the remaining precision_timestamp, precision_timestamp_tz, precision_time functions handle the precision correctly.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I merged #994 and rebased this PR on latest main.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do indeed have the situation in function signatures where datetime return type precision is derived from directly from the argument precision where e.g. for add:pts_iday both arguments need to have the same precision P:

- args:
- name: x
value: precision_timestamp<P>
- name: y
value: interval_day<P>
return: precision_timestamp<P>

We could (in a follow-up) change this for more flexibility to:

      - args:
          - name: x
            value: precision_timestamp<P1>
          - name: y
            value: interval_day<P2>
        return: |
          P = max(P1, P2)
          precision_timestamp<P>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

created a new issue for looking into the return type precision: #1033

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thank you for creating the follow up issue!

Copy link
Copy Markdown
Member Author

@nielspardon nielspardon Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: found another issue today raising the exact same issue #926

Looks like @vbarua is of the opinion that one should explicitly cast the argument precision to the same precision before the function call in the older issue.

Copy link
Copy Markdown
Contributor

@yongchul yongchul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 but I have no idea whether this will cause any problem to anyone or not...

Signed-off-by: Niels Pardon <par@zurich.ibm.com>
@nielspardon nielspardon force-pushed the par-datetime-substract branch from 134a5d6 to 53c4421 Compare May 11, 2026 14:09
@nielspardon nielspardon requested a review from benbellick as a code owner May 11, 2026 14:09
Copy link
Copy Markdown
Member

@benbellick benbellick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nielspardon nielspardon merged commit 26fbd04 into substrait-io:main May 12, 2026
12 checks passed
@nielspardon nielspardon deleted the par-datetime-substract branch May 12, 2026 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants