fix(mysql): transpile UNIX_TIMESTAMP to other dialects (#7754)#7755
fix(mysql): transpile UNIX_TIMESTAMP to other dialects (#7754)#7755danmar2000 wants to merge 1 commit into
Conversation
geooo109
left a comment
There was a problem hiding this comment.
Nice work! left a comment in order the implementation more robust.
| self.validate_all( | ||
| "SELECT UNIX_TIMESTAMP(ts)", | ||
| write={ | ||
| "mysql": "SELECT UNIX_TIMESTAMP(ts)", | ||
| "spark": "SELECT UNIX_TIMESTAMP(ts)", | ||
| "hive": "SELECT UNIX_TIMESTAMP(ts)", | ||
| "duckdb": "SELECT EPOCH(ts)", | ||
| "postgres": "SELECT DATE_PART('epoch', ts)", | ||
| "redshift": "SELECT DATE_PART('epoch', ts)", | ||
| "snowflake": "SELECT EXTRACT(epoch_second FROM ts)", | ||
| "presto": "SELECT TO_UNIXTIME(ts)", | ||
| "trino": "SELECT TO_UNIXTIME(ts)", | ||
| }, | ||
| ) | ||
| self.validate_all( | ||
| "SELECT UNIX_TIMESTAMP()", | ||
| write={ | ||
| "mysql": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())", | ||
| "spark": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())", | ||
| "hive": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())", | ||
| "duckdb": "SELECT EPOCH(CURRENT_TIMESTAMP)", | ||
| "postgres": "SELECT DATE_PART('epoch', CURRENT_TIMESTAMP)", | ||
| "redshift": "SELECT DATE_PART('epoch', GETDATE())", | ||
| "snowflake": "SELECT EXTRACT(epoch_second FROM CURRENT_TIMESTAMP())", | ||
| "presto": "SELECT TO_UNIXTIME(CURRENT_TIMESTAMP)", | ||
| "trino": "SELECT TO_UNIXTIME(CURRENT_TIMESTAMP)", | ||
| }, | ||
| ) |
There was a problem hiding this comment.
did you try these against the engines ?
The transpilation should preserve the output type of MySQL.
mysql> SELECT UNIX_TIMESTAMP('2021-01-01 00:00:00.2');
Field 1: `UNIX_TIMESTAMP('2021-01-01 00:00:00.2')`
Catalog: `def`
Database: ``
Table: ``
Org_table: ``
Type: NEWDECIMAL
Collation: binary (63)
Length: 14
Max_length: 12
Decimals: 1
Flags: NOT_NULL BINARY NUM
+-----------------------------------------+
| UNIX_TIMESTAMP('2021-01-01 00:00:00.2') |
+-----------------------------------------+
| 1609452000.2 |
+-----------------------------------------+
1 row in set (0.000 sec)
===================================
mysql> SELECT UNIX_TIMESTAMP('2021-01-01 00:00:00');
Field 1: `UNIX_TIMESTAMP('2021-01-01 00:00:00')`
Catalog: `def`
Database: ``
Table: ``
Org_table: ``
Type: LONGLONG
Collation: binary (63)
Length: 21
Max_length: 10
Decimals: 0
Flags: NOT_NULL BINARY NUM
+---------------------------------------+
| UNIX_TIMESTAMP('2021-01-01 00:00:00') |
+---------------------------------------+
| 1609452000 |
+---------------------------------------+
1 row in set (0.000 sec)
My tests ^ show that based on the input the output type changes for MySQL.
My suggestion is to focus on the transpilation from mysql to duckdb for now.
-
We should take care of the input argument. MySQL can take as input a raw string.
We should wrap the argument of MySQL with aexp.TsOrDsToTimestampin order to cast it at the output (duckdb doesn't support string args). -
Let's apply 1. and then validate that my mysql queries above, transpile correctly to duckdb.
cc: @georgesittas (for visibility)
672ef82 to
3ca02b6
Compare
|
@danmar2000 I have tested various cases for this: We can't cover a big portion of these ^ inputs without a very complex solution. If the input is a string/integer, for example, it has a lot of formats that we can't transpile. Keep in mind that the input argument can be a column and hide the actual value until run time (argument not a literal). So we'd have to generate logic on top of the final transpiled query. Moreover, even if we assume we don't cover those cases, we'd still have to generate, on top of the query, a We could reduce the scope of this PR by annotating the parsed AST and transpiling only when the input is a date. But that isn't trivial either, because of cases like: Here ^ MySQL's Finally, I'm not sure we can parse this function as So, I'm closing this for now. |
|
@geooo109 I'll add that there's also a timezone issue. MySQL's |
Map MySQL
UNIX_TIMESTAMPtoexp.TimeToUnixso it transpiles to the correct equivalent in other dialects instead of being passed through unchanged.Closes #7754