Skip to content

python(fix): infer parquet time column for int64/uint64 timestamps#582

Merged
wei-qlu merged 4 commits into
mainfrom
python/parquet-time-column
May 22, 2026
Merged

python(fix): infer parquet time column for int64/uint64 timestamps#582
wei-qlu merged 4 commits into
mainfrom
python/parquet-time-column

Conversation

@wei-qlu
Copy link
Copy Markdown
Contributor

@wei-qlu wei-qlu commented May 21, 2026

Summary

  • This adds a client-side fallback that picks the first INT64 or UINT64 column whose name (case-insensitive) is ts, timestamp, or time and removes it from data_columns.
  • Applies to both PARQUET_FLATDATASET and PARQUET_SINGLE_CHANNEL_PER_ROW layouts.

Test plan

  • Unit tests: pytest python/lib/sift_client/_tests/resources/test_data_imports.py
  • Import a Parquet file with a bare int64 timestamp column via client.data_imports.import_from_path(...) and confirm the returned config promotes it to time_column and excludes it from data_columns.
  • Import a Parquet file whose timestamp uses Arrow timestamp(ns) and confirm server-side detection still wins (no regression).

@wei-qlu wei-qlu marked this pull request as ready for review May 21, 2026 23:50
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 21, 2026

Python docs preview: https://sift-stack.github.io/sift/python/pr-582/

Deployed from c916aca. The link may take up to a minute to become live as GitHub Pages propagates.

Comment thread python/lib/sift_client/resources/data_imports.py Outdated
Comment thread python/lib/sift_client/resources/data_imports.py Outdated
@wei-qlu wei-qlu requested a review from marc-sift May 22, 2026 00:09
@wei-qlu wei-qlu merged commit 329ec11 into main May 22, 2026
22 checks passed
@wei-qlu wei-qlu deleted the python/parquet-time-column branch May 22, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants