fix(extensions)!: correct return type for subtract:date_iday operation#1029
Conversation
fa7469c to
b42694a
Compare
| - name: y | ||
| value: interval_day<P> | ||
| return: date | ||
| return: precision_timestamp<P> |
There was a problem hiding this comment.
Wait what? The existing change looks correct to my eyes.
There was a problem hiding this comment.
If you subtract 1 day, 2 hours, 3 seconds, 4 microseconds from a date what do you expect to be returned? Certainly not a full date anymore but a timestamp and given that interval_day has a precision for subseconds it would be a precision timestamp of the same precision as interval_day.
There was a problem hiding this comment.
Thank you for pointing that out but that opens even more questions. If that is the case, we should update all other things? For instance, timestamp and timestamp_tz are at microsecond precision. So if you do math with interval_day<P>, it should be the higher precision of max(6,P)?
or,
if (P <= 6) then (timestamp) else (precision_timestamp<P>)
There was a problem hiding this comment.
So I already have this PR open for removing the old timestamp, timestamp_tz and time types which should simplify such scenarios somewhat. We can check if the remaining precision_timestamp, precision_timestamp_tz, precision_time functions handle the precision correctly.
There was a problem hiding this comment.
I merged #994 and rebased this PR on latest main.
There was a problem hiding this comment.
We do indeed have the situation in function signatures where datetime return type precision is derived from directly from the argument precision where e.g. for add:pts_iday both arguments need to have the same precision P:
substrait/extensions/functions_datetime.yaml
Lines 206 to 211 in 87d73b6
We could (in a follow-up) change this for more flexibility to:
- args:
- name: x
value: precision_timestamp<P1>
- name: y
value: interval_day<P2>
return: |
P = max(P1, P2)
precision_timestamp<P>There was a problem hiding this comment.
created a new issue for looking into the return type precision: #1033
There was a problem hiding this comment.
Sounds good. Thank you for creating the follow up issue!
b42694a to
134a5d6
Compare
yongchul
left a comment
There was a problem hiding this comment.
+1 but I have no idea whether this will cause any problem to anyone or not...
Signed-off-by: Niels Pardon <par@zurich.ibm.com>
134a5d6 to
53c4421
Compare
BREAKING CHANGE: changes the return type of the
subtract:date_idayfunction fromdatetoprecision_timestampFix the return type of the
subtractfunction when subtracting aninterval_dayfrom adate. Sinceinterval_dayis an interval of days to subseconds the operation should return aprecision_timestamptype, not adatetype.This corrects the function signature in
functions_datetime.yamlwheredate+interval_daywas incorrectly specified to returndateinstead ofprecision_timestamp.This is similar to #1007 and was originally reported in #576.
This also fixes the
subtract:ts_idaytest cases to use a precision type parameter foriday.This change is