[Fix] Adapt Arrow timestamp conversion for DATETIME and TIMESTAMPTZ - #366
Merged
Merged
Conversation
CalvinKirs
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Background:
PR apache/doris#38215 added timezone support to
datetime. Versions prior to this PR hardcoded the use ofTimeStampMicroVector, whereas subsequent versions returnTimeStampTZVector. Consequently, to ensure compatibility, the connector inferred the time unit based on the timestamp itself.Changes:
Since
datetimeis inherently timezone-agnostic, PR apache/doris#65823 removed the timezone fromdatetime, causing it to return a native timestamp;timestamptzcontinues to returnTimeStampTZVector.Connector Adaptation:
No conversion is performed if there is no timezone; the existing default conversion logic is applied if a timezone is present.
Impact:
Reading
datetimedata may result in errors on versions prior to PR apache/doris#38215.Checklist(Required)
Further comments
None.