Skip to content

feat(cli): add dora record and dora play commands#1494

Closed
adityabelgaonkar05 wants to merge 2 commits into
dora-rs:mainfrom
adityabelgaonkar05:main
Closed

feat(cli): add dora record and dora play commands#1494
adityabelgaonkar05 wants to merge 2 commits into
dora-rs:mainfrom
adityabelgaonkar05:main

Conversation

@adityabelgaonkar05
Copy link
Copy Markdown

Closes #1488

Summary

  • Adds dora record to capture dataflow messages from Zenoh into Arrow IPC (.arrows) files
    • Supports --all or specific topics, configurable duration (-t), Ctrl+C to stop
    • Batched writing (100 msgs/batch), atomic file writes via temp file rename
  • Adds dora play to replay recorded messages back into a running dataflow
    • Rate scaling (--rate 2.0 for 2x speed), loop support (--loop)
    • Schema validation, microsecond-precision timing preservation
  • Reuses existing TopicSelector/DataflowSelector from dora topic commands

Test plan

  • dora record --help / dora play --help show correct usage
  • dora record --all -t 5 captures messages from a dataflow with publish_all_messages_to_zenoh: true
  • dora play recording.arrows replays messages with correct timing
  • dora play recording.arrows --rate 2.0 plays at double speed
  • dora play recording.arrows --loop loops continuously
  • Recording a non-existent dataflow gives a clear error
  • Playing an invalid file gives a clear error

Adds native recording and playback of dataflow messages using Arrow IPC format.
 subscribes to Zenoh-published outputs and writes them to .arrows files
with batched writing and atomic file operations.  reads recordings and
republishes messages with configurable rate scaling and loop support.

Requires  in the dataflow descriptor.
@heyong4725
Copy link
Copy Markdown
Collaborator

Hi @adityabelgaonkar05 — first, thanks for the substantive work here. 687 lines of careful Zenoh-sniffing + Arrow IPC plumbing is real effort, and I'm sorry the PR sat without feedback for 2 months. That's not on you.

Closing this PR rather than asking for a rebase, because the ground moved under it in a way that's not a rescue-friendly situation. Walking through why so the call doesn't feel arbitrary.

What happened on main

Four weeks after you opened this PR, the 1.0 consolidation commit 145ccce0 (merged 2026-04-17) landed a dora record and dora replay implementation directly — addressing the same issue #1488 but via a different architectural approach. It's been iterated on since (#1673, #1691, #1714, #1715), so the maintainer investment in that design is firm.

The two approaches differ at the foundational level:

Aspect This PR Current main
Capture mechanism Zenoh subscriber (external observer; sniffs published topics) Inject a record node into the dataflow graph (modify the YAML at record time)
File format .arrows (Apache Arrow IPC) .drec (internal dora_recording format)
Replay mechanism Publish back to Zenoh Replace recorder nodes with replay nodes in the dataflow
Closest analog rosbag (which is what issue #1488 explicitly asked for) A dora-native node-graph transformation

Both are legitimate. The Zenoh-sniffing pattern is closer to what #1488's wording requested ("first-class equivalent to rosbag"). The dataflow-modification pattern is more idiomatic to dora's zero-copy / Arrow-native architecture and works without Zenoh being on.

Why this isn't a rebase-and-merge rescue

  1. Command-namespace collision. dora record exists on main as a different command. Two record subcommands can't coexist; the rescue would need a rename (e.g. dora zenoh-record, dora zenoh-play).
  2. Architectural commitment. The 1.0 consolidation made a deliberate choice for the dataflow-modification design. Adding a parallel Zenoh-sniffing surface is a "do we want two record modes?" maintainer-design question, not a mechanical merge.
  3. No prior discussion. Issue Native Data Recording and Playback System (dora record / dora play) #1488 is open but neither approach was negotiated on it before either implementation was written. Picking up the Zenoh-sniffing approach now would skip the design step.

Where the Zenoh-sniffing idea could still go

If you (or anyone reading this) still want the Zenoh-sniffing pattern alongside the existing .drec recorder — and there are real reasons to want it; the external-observer model is genuinely useful for cross-system debugging — the right path is:

  1. Open a fresh issue (or comment on Native Data Recording and Playback System (dora record / dora play) #1488) proposing it explicitly as a second mode alongside the existing dora record. Frame: "Add dora record --mode zenoh for sniffing-style capture across a running dataflow without modifying its graph; complements the existing dataflow-modification mode."
  2. Wait for a maintainer thumbs-up on the design before re-implementing. A 2-month silence on a PR with no design pre-approval is a stronger signal to discuss the architecture first than to push code.
  3. Reuse your existing code if/when the design lands — the Zenoh subscriber loop and Arrow IPC framing you wrote here are the substantive parts; a future PR could pull them in directly.

The original code in this branch isn't going anywhere — it stays on the PR branch and in fork history. If the design discussion lands favorably you have a head start.

One concrete note on the implementation

I read through binaries/cli/src/command/record.rs and play.rs. The Zenoh subscriber loop is clean and the batched-write + atomic-rename pattern (write to .tmp then rename) is the right shape for crash safety. The schema-validation step on play is also a nice touch that the existing .drec recorder may not do. If the design path opens up, this code is worth keeping.

Thanks again — the substance was there, the timing wasn't.

@heyong4725 heyong4725 closed this May 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native Data Recording and Playback System (dora record / dora play)

2 participants