Skip to content

feat(api) add timestamp function to node api#1789

Closed
oceanusxiv wants to merge 1 commit into
dora-rs:mainfrom
oceanusxiv:efang2/node_timestamp
Closed

feat(api) add timestamp function to node api#1789
oceanusxiv wants to merge 1 commit into
dora-rs:mainfrom
oceanusxiv:efang2/node_timestamp

Conversation

@oceanusxiv
Copy link
Copy Markdown
Contributor

Summary

  • Add timestamp() method to the Rust DoraNode that returns the current HLC timestamp from the node's internal clock
  • Expose timestamp() on the Python Node class, returning a timezone-aware UTC datetime.datetime
  • Make the cached datetime_module() helper public in dora-operator-api-python so the node crate can reuse it instead of calling PyModule::import on every invocation

Motivation

DoraNode already maintains an internal uhlc::HLC clock used to timestamp every outgoing message, but there was no way for user code to read the current clock value directly. This is useful for:

  • Measuring processing latency by comparing the current clock against an input event's timestamp
  • Correlating log entries or external events with the same logical clock used by dora's data plane
  • Profiling node processing time without introducing a separate clock source

Changes

File Change
apis/rust/node/src/node/mod.rs Add pub fn timestamp(&self) -> uhlc::Timestamp
apis/python/node/src/lib.rs Add timestamp() PyO3 method returning datetime.datetime (UTC)
apis/python/operator/src/lib.rs Make datetime_module() pub for cross-crate reuse
apis/python/node/dora/__init__.pyi Add type stub for timestamp()

Usage

Rust:

let node = DoraNode::init_from_env()?;
let ts = node.timestamp();
println!("Current HLC time: {ts}");

Python:

from dora import Node

node = Node()
print(f"Current HLC time: {node.timestamp()}")

for event in node:
    if event["type"] == "INPUT":
        latency = node.timestamp() - event["metadata"]["timestamp"]
        print(f"Processing latency: {latency}")

@trunk-io
Copy link
Copy Markdown
Contributor

trunk-io Bot commented May 1, 2026

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@oceanusxiv oceanusxiv changed the title add timestamp function to node api feat(api) add timestamp function to node api May 1, 2026
@heyong4725
Copy link
Copy Markdown
Collaborator

Closing in favor of #1877, which rescues this against current main. Cherry-picks cleanly, plus expanded docstrings (the HLC-vs-SystemTime guidance from your motivation is now in the rustdoc) and a regression test pinning the node-clock-not-fresh-clock contract. Co-authored-by credit preserved. Thanks for the original work!

@heyong4725 heyong4725 closed this May 20, 2026
heyong4725 added a commit that referenced this pull request May 20, 2026
… of #1789)

Rescues @oceanusxiv's #1789 against current main. The PR sat for 18 days
with no reviews and trivially-clean cherry-pick against `main`, so this
is a near-verbatim re-application plus expanded docstrings and a
regression test.

Why
===

`DoraNode` already maintains an internal `uhlc::HLC` clock that stamps
every outgoing message, but user code had no way to read that clock
directly. Without this method, a node that wants to measure per-event
processing latency has to fall back to `std::time::SystemTime::now()` —
a different physical clock from the HLC, so subtracting that from
`event.metadata.timestamp` is meaningless across daemons.

Three concrete use cases motivated #1789:

  1. `latency = node.timestamp() - event.metadata.timestamp` measured
     against the same clock dora's data plane uses.
  2. Correlating log lines / external observations with the HLC.
  3. Profiling node processing time without introducing a second
     clock source.

What's added
============

* `apis/rust/node/src/node/mod.rs` — `DoraNode::timestamp() ->
  uhlc::Timestamp`. Single-line body: `self.clock.new_timestamp()`.
  Docstring spells out the "use this against `event.metadata.timestamp`,
  NOT against `SystemTime::now()`" rule.

* `apis/python/node/src/lib.rs` — `Node.timestamp() -> datetime.datetime`.
  Converts HLC physical time to a UTC-aware `datetime`. The Python
  docstring explicitly notes the resolution caveats (microsecond
  precision, HLC logical counter dropped) and points strict-ordering
  callers at the Rust API.

* `apis/python/operator/src/lib.rs` — promote `datetime_module()` from
  private to `pub` so the node crate can reuse the cached
  `PyModule::import("datetime")` instead of re-importing on every
  call. Same cache pattern dora already uses for its own outgoing
  metadata conversions in the operator crate.

* `apis/python/node/dora/__init__.pyi` — type stub for `timestamp()`.

What was added on top of #1789
==============================

* Expanded both docstrings to call out the load-bearing "use the HLC,
  not SystemTime" guidance and the Python resolution caveats.
* New regression test
  `node::tests::timestamp_uses_node_clock_and_is_monotonic` that
  asserts two `timestamp()` calls share an HLC ID (i.e., they read
  the *node's* clock, not a freshly-created one) AND are strictly
  monotonic. Without this test, a refactor that gives `timestamp()`
  its own clock would silently break the latency-measurement use case
  in the docstring.

Verification
============

  cargo fmt --all -- --check
  cargo clippy --all --exclude dora-{node-api,operator-api,ros2-bridge}-python -- -D warnings
  cargo test -p dora-node-api --lib
    (83/83 pass; the new test runs in 0.11s)
  cargo check --examples

Co-authored-by: oceanusxiv <oceanusxiv@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
heyong4725 added a commit that referenced this pull request May 20, 2026
…#1789) (#1877)

feat(api): add `node.timestamp()` accessor for the node's HLC (rescue of #1789)

Rescues @oceanusxiv's #1789 against current main. The PR sat for 18 days
with no reviews and trivially-clean cherry-pick against `main`, so this
is a near-verbatim re-application plus expanded docstrings and a
regression test.

Why
===

`DoraNode` already maintains an internal `uhlc::HLC` clock that stamps
every outgoing message, but user code had no way to read that clock
directly. Without this method, a node that wants to measure per-event
processing latency has to fall back to `std::time::SystemTime::now()` —
a different physical clock from the HLC, so subtracting that from
`event.metadata.timestamp` is meaningless across daemons.

Three concrete use cases motivated #1789:

  1. `latency = node.timestamp() - event.metadata.timestamp` measured
     against the same clock dora's data plane uses.
  2. Correlating log lines / external observations with the HLC.
  3. Profiling node processing time without introducing a second
     clock source.

What's added
============

* `apis/rust/node/src/node/mod.rs` — `DoraNode::timestamp() ->
  uhlc::Timestamp`. Single-line body: `self.clock.new_timestamp()`.
  Docstring spells out the "use this against `event.metadata.timestamp`,
  NOT against `SystemTime::now()`" rule.

* `apis/python/node/src/lib.rs` — `Node.timestamp() -> datetime.datetime`.
  Converts HLC physical time to a UTC-aware `datetime`. The Python
  docstring explicitly notes the resolution caveats (microsecond
  precision, HLC logical counter dropped) and points strict-ordering
  callers at the Rust API.

* `apis/python/operator/src/lib.rs` — promote `datetime_module()` from
  private to `pub` so the node crate can reuse the cached
  `PyModule::import("datetime")` instead of re-importing on every
  call. Same cache pattern dora already uses for its own outgoing
  metadata conversions in the operator crate.

* `apis/python/node/dora/__init__.pyi` — type stub for `timestamp()`.

What was added on top of #1789
==============================

* Expanded both docstrings to call out the load-bearing "use the HLC,
  not SystemTime" guidance and the Python resolution caveats.
* New regression test
  `node::tests::timestamp_uses_node_clock_and_is_monotonic` that
  asserts two `timestamp()` calls share an HLC ID (i.e., they read
  the *node's* clock, not a freshly-created one) AND are strictly
  monotonic. Without this test, a refactor that gives `timestamp()`
  its own clock would silently break the latency-measurement use case
  in the docstring.

Verification
============

  cargo fmt --all -- --check
  cargo clippy --all --exclude dora-{node-api,operator-api,ros2-bridge}-python -- -D warnings
  cargo test -p dora-node-api --lib
    (83/83 pass; the new test runs in 0.11s)
  cargo check --examples

Co-authored-by: oceanusxiv <oceanusxiv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants