Wait Free Realtime Publisher

[bjin2364]

Background

RealtimePublisher class uses locking mechanisms to publish data in a separate thread, but given it follows a single-producer single-consumer paradigm, we can create a wait-free implementation using the LockFreeQueue data structure in this repo. This PR is an attempt to introduce this implementation.

Changes

• Adds a WaitFreeRealtimePublisher class, which compared to RealtimePublisher:
  • similarities:
    • spawns a separate thread for publishing messages
  • differences:
    • wait-free
    • background thread has configurable priority and CPU affinity (Linux only)
    • does not spawn publishing thread on construction
    • slightly different public API
• Introduces a benchmark script to compare WaitFreeRealtimePublisher to RealtimePublisher
  • consequently slightly changes RealtimePublisher to support benchmarking, and also adds publisher interface abstraction for benchmarks and test

Testing

• unit tests
• benchmark

Benchmark Results

Settings:

• linux machine with isolated cores 0,1. Publishing thread is pinned to core 1 while main thread is core 0
• disabled cpu scaling
• compiled with release mode

Example Results: wait-free publishing side is 2 orders of magnitude faster than RealtimePublisher on the producer-side. I'm still a bit confused by the publishing thread side: the other day I was also seeing an order of magnitude of messages that were published, so maybe this benchmark needs some tuning.

Running ./build/realtime_tools/realtime_publishers_benchmark
Run on (12 X 400 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x6)
  L1 Instruction 32 KiB (x6)
  L2 Unified 1280 KiB (x6)
  L3 Unified 12288 KiB (x1)
Load Average: 0.62, 0.55, 0.60
----------------------------------------------------------------------------------------------
Benchmark                                    Time             CPU   Iterations UserCounters...
----------------------------------------------------------------------------------------------
BM_RealtimePublisher                      26.9 ns         17.0 ns     41174747 num_publishes=484.333k
BM_WaitFreeRealtimePublisher<1>/1        0.398 ns        0.398 ns   1000000000 num_publishes=7.508k
BM_WaitFreeRealtimePublisher<1>/5        0.397 ns        0.397 ns   1000000000 num_publishes=6.984k
BM_WaitFreeRealtimePublisher<1>/10       0.398 ns        0.398 ns   1000000000 num_publishes=6.399k
BM_WaitFreeRealtimePublisher<1>/50       0.396 ns        0.396 ns   1000000000 num_publishes=3.866k
BM_WaitFreeRealtimePublisher<2>/1        0.340 ns        0.340 ns   1000000000 num_publishes=12.859k
BM_WaitFreeRealtimePublisher<2>/5        0.340 ns        0.340 ns   1000000000 num_publishes=11.842k
BM_WaitFreeRealtimePublisher<2>/10       0.339 ns        0.339 ns   1000000000 num_publishes=10.871k
BM_WaitFreeRealtimePublisher<2>/50       0.337 ns        0.337 ns   1000000000 num_publishes=6.621k
BM_WaitFreeRealtimePublisher<5>/1        0.314 ns        0.314 ns   1000000000 num_publishes=37.859k
BM_WaitFreeRealtimePublisher<5>/5        0.313 ns        0.313 ns   1000000000 num_publishes=36.552k
BM_WaitFreeRealtimePublisher<5>/10       0.313 ns        0.313 ns   1000000000 num_publishes=33.087k
BM_WaitFreeRealtimePublisher<5>/50       0.312 ns        0.312 ns   1000000000 num_publishes=19.569k
BM_DefaultWaitFreeRealtimePublisher      0.340 ns        0.340 ns   1000000000 num_publishes=12.854k

Notes

• I'm not super familiar with copyright side, so just copy and pasted the same licensing as some other files in this repo.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 60.98901% with 71 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.89%. Comparing base (69bd9a3) to head (1e630a9).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
..._tools/benchmark/realtime_publishers_benchmark.cpp	0.00%	40 Missing ⚠️
...de/realtime_tools/wait_free_realtime_publisher.hpp	54.54%	18 Missing and 7 partials ⚠️
realtime_tools/test/wait_free_publisher_test.cpp	92.77%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #461      +/-   ##
==========================================
- Coverage   85.25%   81.89%   -3.36%     
==========================================
  Files          17       21       +4     
  Lines        1397     1552     +155     
  Branches      132      152      +20     
==========================================
+ Hits         1191     1271      +80     
- Misses        120      186      +66     
- Partials       86       95       +9     

Flag	Coverage Δ
unittests	81.89% <60.98%> (-3.36%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ools/include/realtime_tools/realtime_publisher.hpp	78.26% <100.00%> (-11.99%)	⬇️
...clude/realtime_tools/utils/publisher_interface.hpp	100.00% <100.00%> (ø)
realtime_tools/test/wait_free_publisher_test.cpp	92.77% <92.77%> (ø)
...de/realtime_tools/wait_free_realtime_publisher.hpp	54.54% <54.54%> (ø)
..._tools/benchmark/realtime_publishers_benchmark.cpp	0.00% <0.00%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[christophfroehlich]

Is this ready for review? We can first discuss if the approach is fine, and then try to fix the backwards compatibility and windows build.

[bjin2364]

@christophfroehlich sorry yeah I would like to have a review on it. I put it in draft with that intention, but I'm not totally familiar with the review process here.

There seemed to be a weird inconsistency with the benchmark results that I wanted to circle back on (which is part of the reason why I kept this in draft), but I haven't found the time to do so. I suppose regardless, the current state of the PR should still be a net positive?

I'll just take it out of draft.

[bjin2364]

2025-12-07T20:34:24+00:00
Running ./build/realtime_tools/realtime_publishers_benchmark
Run on (12 X 2699.97 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x6)
  L1 Instruction 32 KiB (x6)
  L2 Unified 1280 KiB (x6)
  L3 Unified 12288 KiB (x1)
Load Average: 8.10, 2.49, 1.07
----------------------------------------------------------------------------------------------
Benchmark                                    Time             CPU   Iterations UserCounters...
----------------------------------------------------------------------------------------------
BM_RealtimePublisher                       120 ns          119 ns      9822526 num_publishes=407.265k push_failure_percent=95.8538
BM_WaitFreeRealtimePublisher<1>/0         85.4 ns         85.4 ns      8370687 num_publishes=826.754k push_failure_percent=90.1232
BM_WaitFreeRealtimePublisher<1>/1         1.52 ns         1.52 ns    459542692 num_publishes=13.291k push_failure_percent=99.9971
BM_WaitFreeRealtimePublisher<1>/5         1.55 ns         1.52 ns    459953193 num_publishes=12.294k push_failure_percent=99.9973
BM_WaitFreeRealtimePublisher<1>/10        1.54 ns         1.52 ns    460225867 num_publishes=11.197k push_failure_percent=99.9976
BM_WaitFreeRealtimePublisher<2>/0         16.1 ns         16.0 ns     41522408 num_publishes=1.32714M push_failure_percent=96.8038
BM_WaitFreeRealtimePublisher<2>/1         1.90 ns         1.89 ns    354443608 num_publishes=25.39k push_failure_percent=99.9928
BM_WaitFreeRealtimePublisher<2>/5         1.89 ns         1.87 ns    373377742 num_publishes=24.546k push_failure_percent=99.9934
BM_WaitFreeRealtimePublisher<2>/10        1.92 ns         1.91 ns    373437382 num_publishes=22.885k push_failure_percent=99.9939
BM_WaitFreeRealtimePublisher<5>/0         11.6 ns         11.3 ns     60855587 num_publishes=2.52833M push_failure_percent=95.8454
BM_WaitFreeRealtimePublisher<5>/1         1.63 ns         1.62 ns    419569340 num_publishes=65.388k push_failure_percent=99.9844
BM_WaitFreeRealtimePublisher<5>/5         1.61 ns         1.59 ns    440364466 num_publishes=62.016k push_failure_percent=99.9859
BM_WaitFreeRealtimePublisher<5>/10        1.64 ns         1.64 ns    440803775 num_publishes=58.247k push_failure_percent=99.9868
BM_DefaultWaitFreeRealtimePublisher       1.69 ns         1.67 ns    401184995 num_publishes=25.432k push_failure_percent=99.9937

Ran benchmarks again with some updated settings, also with stress running on all cores to see how publishers would handle things under worst case loads.

• Maximizing the push calls and the publisher thread calls can happen if publisher thread is busy waiting, which makes sense
• in all other cases where we put a "sleep" in the publisher thread, there are fewer publishes, but I think this difference makes sense because WaitFreeRealtimePublisher doesn't notify the publisher thread like the RealtimePublisher does.
• this benchmark is a bit extreme in that we're calling publish repeatedly in a while loop rather than the practical use-case, which is calling in a fixed period

[github-actions]

This PR is stale because it has been open for 45 days with no activity. Please tag a maintainer for help on completing this PR, or close it if you think it has become obsolete.