Skip to content

fix(mysql): transpile UNIX_TIMESTAMP to other dialects (#7754)#7755

Closed
danmar2000 wants to merge 1 commit into
tobymao:mainfrom
danmar2000:fix-mysql-unix-timestamp
Closed

fix(mysql): transpile UNIX_TIMESTAMP to other dialects (#7754)#7755
danmar2000 wants to merge 1 commit into
tobymao:mainfrom
danmar2000:fix-mysql-unix-timestamp

Conversation

@danmar2000

Copy link
Copy Markdown

Map MySQL UNIX_TIMESTAMP to exp.TimeToUnix so it transpiles to the correct equivalent in other dialects instead of being passed through unchanged.

Closes #7754

@geooo109 geooo109 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! left a comment in order the implementation more robust.

Comment on lines +491 to +518
self.validate_all(
"SELECT UNIX_TIMESTAMP(ts)",
write={
"mysql": "SELECT UNIX_TIMESTAMP(ts)",
"spark": "SELECT UNIX_TIMESTAMP(ts)",
"hive": "SELECT UNIX_TIMESTAMP(ts)",
"duckdb": "SELECT EPOCH(ts)",
"postgres": "SELECT DATE_PART('epoch', ts)",
"redshift": "SELECT DATE_PART('epoch', ts)",
"snowflake": "SELECT EXTRACT(epoch_second FROM ts)",
"presto": "SELECT TO_UNIXTIME(ts)",
"trino": "SELECT TO_UNIXTIME(ts)",
},
)
self.validate_all(
"SELECT UNIX_TIMESTAMP()",
write={
"mysql": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())",
"spark": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())",
"hive": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())",
"duckdb": "SELECT EPOCH(CURRENT_TIMESTAMP)",
"postgres": "SELECT DATE_PART('epoch', CURRENT_TIMESTAMP)",
"redshift": "SELECT DATE_PART('epoch', GETDATE())",
"snowflake": "SELECT EXTRACT(epoch_second FROM CURRENT_TIMESTAMP())",
"presto": "SELECT TO_UNIXTIME(CURRENT_TIMESTAMP)",
"trino": "SELECT TO_UNIXTIME(CURRENT_TIMESTAMP)",
},
)

@geooo109 geooo109 Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you try these against the engines ?

The transpilation should preserve the output type of MySQL.

mysql> SELECT UNIX_TIMESTAMP('2021-01-01 00:00:00.2');
Field   1:  `UNIX_TIMESTAMP('2021-01-01 00:00:00.2')`
Catalog:    `def`
Database:   ``
Table:      ``
Org_table:  ``
Type:       NEWDECIMAL
Collation:  binary (63)
Length:     14
Max_length: 12
Decimals:   1
Flags:      NOT_NULL BINARY NUM 

+-----------------------------------------+
| UNIX_TIMESTAMP('2021-01-01 00:00:00.2') |
+-----------------------------------------+
|                            1609452000.2 |
+-----------------------------------------+
1 row in set (0.000 sec)
===================================
mysql> SELECT UNIX_TIMESTAMP('2021-01-01 00:00:00');
Field   1:  `UNIX_TIMESTAMP('2021-01-01 00:00:00')`
Catalog:    `def`
Database:   ``
Table:      ``
Org_table:  ``
Type:       LONGLONG
Collation:  binary (63)
Length:     21
Max_length: 10
Decimals:   0
Flags:      NOT_NULL BINARY NUM 

+---------------------------------------+
| UNIX_TIMESTAMP('2021-01-01 00:00:00') |
+---------------------------------------+
|                            1609452000 |
+---------------------------------------+
1 row in set (0.000 sec)

My tests ^ show that based on the input the output type changes for MySQL.

My suggestion is to focus on the transpilation from mysql to duckdb for now.

  1. We should take care of the input argument. MySQL can take as input a raw string.
    We should wrap the argument of MySQL with a exp.TsOrDsToTimestamp in order to cast it at the output (duckdb doesn't support string args).

  2. Let's apply 1. and then validate that my mysql queries above, transpile correctly to duckdb.

cc: @georgesittas (for visibility)

@danmar2000 danmar2000 force-pushed the fix-mysql-unix-timestamp branch from 672ef82 to 3ca02b6 Compare June 16, 2026 13:31
@geooo109

geooo109 commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

@danmar2000 I have tested various cases for this:

mysql> SELECT
    ->   UNIX_TIMESTAMP('2015-11-13 10:20:19')      AS plain,
    ->   UNIX_TIMESTAMP('2015-11-13 10:20:19.012')  AS fractional,
    ->   UNIX_TIMESTAMP('2015-11-13')               AS date_only,
    ->   UNIX_TIMESTAMP('2015/11/13 10:20:19')      AS slash,
    ->   UNIX_TIMESTAMP('2015.11.13 10@20@19')      AS mixed_delim,
    ->   UNIX_TIMESTAMP('10:11:12')                 AS looks_like_time,
    ->   UNIX_TIMESTAMP('10:45:15')                 AS invalid_month,
    ->   UNIX_TIMESTAMP(20151113102019)             AS num_full,
    ->   UNIX_TIMESTAMP(151113)                     AS num_yymmdd,
    ->   UNIX_TIMESTAMP('1960-01-01 00:00:00')      AS before_epoch,
    ->   UNIX_TIMESTAMP('2004-04-31 10:00:00')      AS invalid_date,
    ->   UNIX_TIMESTAMP(NULL)                       AS null_arg;
+------------+----------------+------------+------------+-------------+-----------------+---------------+------------+------------+--------------+--------------+----------+
| plain      | fractional     | date_only  | slash      | mixed_delim | looks_like_time | invalid_month | num_full   | num_yymmdd | before_epoch | invalid_date | null_arg |
+------------+----------------+------------+------------+-------------+-----------------+---------------+------------+------------+--------------+--------------+----------+
| 1447402819 | 1447402819.012 | 1447365600 | 1447402819 |  1447402819 |      1289512800 |      0.000000 | 1447402819 | 1447365600 |            0 |     0.000000 |     NULL |
+------------+----------------+------------+------------+-------------+-----------------+---------------+------------+------------+--------------+--------------+----------+
1 row in set, 5 warnings (0.001 sec)

We can't cover a big portion of these ^ inputs without a very complex solution. If the input is a string/integer, for example, it has a lot of formats that we can't transpile. Keep in mind that the input argument can be a column and hide the actual value until run time (argument not a literal). So we'd have to generate logic on top of the final transpiled query.

Moreover, even if we assume we don't cover those cases, we'd still have to generate, on top of the query, a NULL check for when the input argument is NULL, plus a check for the valid MySQL date range
For both of these cases ^, EPOCH doesn't match the semantics of the MySQL input.

We could reduce the scope of this PR by annotating the parsed AST and transpiling only when the input is a date. But that isn't trivial either, because of cases like:

"mysql": "SELECT UNIX_TIMESTAMP(CURRENT_TIMESTAMP())",
"duckdb": "SELECT EPOCH(CURRENT_TIMESTAMP)",

Here ^ MySQL's CURRENT_TIMESTAMP returns a DATETIME (without fractional seconds), while DuckDB's CURRENT_TIMESTAMP returns fractional seconds, so we'd have to handle that mismatch too.

Finally, I'm not sure we can parse this function as TimeToUnix, since we also have StrToUnix. UNIX_TIMESTAMP accepts various input types, which makes the mapping confusing.

So, I'm closing this for now.

@geooo109 geooo109 closed this Jun 16, 2026
@danmar2000

Copy link
Copy Markdown
Author

@geooo109
Thanks a lot for the review!
I really appreciate the time you spent on this.

I'll add that there's also a timezone issue. MySQL's UNIX_TIMESTAMP applies the session timezone, while EPOCH(CAST(...)) treats the value as naive, so they can silently produce different results even on simple inputs. I think closing this is the right call. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UNIX_TIMESTAMP is not transpiled from MySQL to other dialects

2 participants