Skip to content

[Fix] Adapt Arrow timestamp conversion for DATETIME and TIMESTAMPTZ - #366

Merged
JNSimba merged 1 commit into
apache:masterfrom
JNSimba:fix/timestamp-vector-conversion
Jul 27, 2026
Merged

[Fix] Adapt Arrow timestamp conversion for DATETIME and TIMESTAMPTZ#366
JNSimba merged 1 commit into
apache:masterfrom
JNSimba:fix/timestamp-vector-conversion

Conversation

@JNSimba

@JNSimba JNSimba commented Jul 21, 2026

Copy link
Copy Markdown
Member

Proposed changes

Background:

PR apache/doris#38215 added timezone support to datetime. Versions prior to this PR hardcoded the use of TimeStampMicroVector, whereas subsequent versions return TimeStampTZVector. Consequently, to ensure compatibility, the connector inferred the time unit based on the timestamp itself.

Changes:

Since datetime is inherently timezone-agnostic, PR apache/doris#65823 removed the timezone from datetime, causing it to return a native timestamp; timestamptz continues to return TimeStampTZVector.

Connector Adaptation:

No conversion is performed if there is no timezone; the existing default conversion logic is applied if a timezone is present.

Impact:

Reading datetime data may result in errors on versions prior to PR apache/doris#38215.

Checklist(Required)

  1. Does it affect the original behavior: Yes
  2. Has unit tests been added: Yes
  3. Has document been added or modified: No Need
  4. Does it need to update dependencies: No
  5. Are there any changes that cannot be rolled back: No

Further comments

None.

@JNSimba JNSimba changed the title [Fix] Support TIMESTAMPTZ Arrow conversion [Fix] Adapt Arrow timestamp conversion for DATETIME and TIMESTAMPTZ Jul 21, 2026
@JNSimba
JNSimba merged commit 278f1ff into apache:master Jul 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants