Skip to content

Accept +00:00 and other zero-offset timezones as UTC#320

Merged
chmp merged 7 commits into
chmp:mainfrom
ryzhyk:recognize_utc_timezone
Jul 19, 2026
Merged

Accept +00:00 and other zero-offset timezones as UTC#320
chmp merged 7 commits into
chmp:mainfrom
ryzhyk:recognize_utc_timezone

Conversation

@ryzhyk

@ryzhyk ryzhyk commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Arrow encodes every timezone-aware timestamp as a UTC instant; the timezone string is only a display hint. Previously, timestamp deserializer only recognized the "UTC" string as UTC timezone specifier. This commit adds support for other zero-offset designators (+00:00, -00:00, +0000, -0000, Z) alongside "UTC" through a shared is_utc_timezone helper used by both the deserialization and serialization paths.

ryzhyk added a commit to mihaibudiu/dbsp that referenced this pull request Jul 15, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ryzhyk added a commit to mihaibudiu/dbsp that referenced this pull request Jul 15, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ryzhyk added a commit to mihaibudiu/dbsp that referenced this pull request Jul 15, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ryzhyk added a commit to mihaibudiu/dbsp that referenced this pull request Jul 15, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ryzhyk added a commit to mihaibudiu/dbsp that referenced this pull request Jul 15, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ryzhyk added a commit to mihaibudiu/dbsp that referenced this pull request Jul 15, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
lstwn pushed a commit to lstwn/feldera that referenced this pull request Jul 16, 2026
Use our fork of the crate until chmp/serde_arrow#320
is merged.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
@chmp

chmp commented Jul 18, 2026

Copy link
Copy Markdown
Owner

@ryzhyk Thanks a lot. looks good at first glance. Running tests just now.

A question regarding release: I changed the public API, slightly, and removed support for outdated arrow versions in parallel. Before doing a proper major release, I would like to cut a pre-release first. Do you have any expectation for timelines?h

@chmp

chmp commented Jul 18, 2026

Copy link
Copy Markdown
Owner

@ryzhyk Could you fix some minor things before merging? Thanks a lot.

(full disclosure these are findings by running an LLM over your change):

  1. MINOR: Public schema docs are now stale. serde_arrow/src/internal/schema/mod.rs:109 still says timestamp timezone is either None or Some("Utc"). This PR expands accepted UTC designators to Z, +00:00, +0000, etc., so the documented schema contract no longer matches behavior. Fix: update this line to describe accepted UTC zero-offset designators, or say “a supported UTC timezone designator”.
  2. MINOR: Serialization path changed but is not directly covered. serde_arrow/src/internal/serialization/timestamp_builder.rs:89 now accepts the same new designators for writing, but the new tests in serde_arrow/src/test_with_arrow/impls/arrow_timestamp.rs:282 only exercise deserialization and non-UTC rejection. Fix: add a looped serialization test using Timestamp(Microsecond, Some(tz)) for the same UTC_DESIGNATORS, serialize "2025-01-20T19:30:42Z", and assert the produced timestamp value.

My findings:

  • Could you update the changelog to document your changes?
My ask for clarification of 2
Point 2 refers to this branch:

  serde_arrow/src/internal/serialization/timestamp_builder.rs:89

  fn is_utc_tz(tz: Option<&str>) -> Result<bool> {
      match tz {
          None => Ok(false),
          Some(tz) if is_utc_timezone(tz) => Ok(true),
          Some(tz) => fail!("timezone {tz} is not supported"),
      }
  }

  That affects serialization before the value type does much useful work. If a user supplies a schema like:

  Timestamp(Microsecond, Some("+00:00"))

  then ArrayBuilder::from_marrow(...) constructs a TimestampBuilder, calls is_utc_tz, and sets utc: true. Later, for string-like timestamp input, this
  controls parsing:

  if self.utc {
      s.parse::<DateTime<Utc>>()?
  } else {
      s.parse::<NaiveDateTime>()?.and_utc()
  }

  So the PR changed write-side behavior from “reject Some("+00:00") schema” to “accept it and parse UTC timestamp strings”. The existing new test proves
  the read side accepts arrays tagged +00:00; it does not prove the write side accepts a schema tagged +00:00.

  If all relevant production serialization uses integer timestamp types or chrono types with serde timestamp helpers, then the serialized numeric value is
  indeed controlled by the type. My point is narrower: the schema timezone acceptance and UTC-vs-naive string parsing branch changed, so a direct
  regression test for builder construction/serialization with Some("+00:00") would lock in the behavior the commit claims.

@ryzhyk

ryzhyk commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

@ryzhyk Thanks a lot. looks good at first glance. Running tests just now.

A question regarding release: I changed the public API, slightly, and removed support for outdated arrow versions in parallel. Before doing a proper major release, I would like to cut a pre-release first. Do you have any expectation for timelines?h

@chmp , we are currently using my fork of serde-arrow, but this means that we cannot release the fixed version of our crate that depends on it on crates.io until this PR makes it to the next release. So earlier would definitely be better, but I don't want to interfere with your release process.

ryzhyk added 2 commits July 18, 2026 13:20
Arrow encodes every timezone-aware timestamp as a UTC instant; the
timezone string is only a display hint. Previously, timestamp
deserializer only recognized the "UTC" string as UTC timezone
specifier. This commit adds support for other zero-offset designators
(+00:00, -00:00, +0000, -0000, Z) alongside "UTC" through a shared
is_utc_timezone helper used by both the deserialization and serialization
paths.
- Update stale docs.
- Test the serialization path.
- Update changelog.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
@ryzhyk
ryzhyk force-pushed the recognize_utc_timezone branch from 9555b34 to 4e6587e Compare July 18, 2026 20:52
@ryzhyk

ryzhyk commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

@chmp , I rebased and addressed your comments. Thanks for the review!

@chmp

chmp commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Thanks a lot. I took the liberty of performing some minor changes myself:

  • reword the changelog item
  • add your GH handle and this MR to the change
  • add a test that strings with the new offsets are correctly parsed
  • support zero offsets in schema tracing
  • update the tests for the new tracing behavior

@chmp

chmp commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Re. Release: I will release another pre-release (0.15.0-rc.2), before a stable release I would like to test first for a bit in my own consumers whether anything breaks. The plan would be to cut a proper release next week.

@chmp
chmp merged commit 806fd33 into chmp:main Jul 19, 2026
2 checks passed
@chmp

chmp commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Released your changes as 0.15.0-rc.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants