Skip to content

Commit 50cc828

Browse files
gonzalocasasclaude
andcommitted
fix: stop leaking a twisted log observer per Ros instance
TwistedEventLoopManager.__init__ instantiated PythonLoggingObserver and started it; the observer was only stopped in manager.terminate(), which in turn was only invoked by Ros.terminate() (which stops the reactor permanently). Downstream code that follows the documented RosClient.__exit__ -> Ros.close() pattern never called terminate(), so every Ros instance added a permanent observer to twisted's globalLogPublisher. In long-running pytest sessions that create many short-lived Ros instances (e.g. compas_fab's integration suite), this accumulated tens of observers and slowed every twisted log emission linearly. It also contributed to occasional RosTimeoutError flakes on the next Ros.run() after many cycles. Make the observer a process-wide singleton: TwistedEventLoopManager lazily starts a single module-level PythonLoggingObserver via a thread- safe initializer, and terminate() now stops it once and clears the sentinel. Verified by a new tests/ros1/test_lifecycle.py case that constructs 10 Ros instances and asserts at most one observer is added. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent ee7f35a commit 50cc828

3 files changed

Lines changed: 104 additions & 3 deletions

File tree

CHANGELOG.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,14 @@ Unreleased
1616

1717
**Fixed**
1818

19+
* ``TwistedEventLoopManager`` no longer leaks one ``PythonLoggingObserver`` per
20+
``Ros`` instance. The observer is now a process-wide singleton — there's only
21+
ever one twisted-log → stdlib-logging bridge regardless of how many ``Ros``
22+
instances come and go. Long-running pytest sessions that create many
23+
short-lived ``Ros`` instances were accumulating one observer per cycle,
24+
slowing every log emission linearly and contributing to the occasional
25+
``RosTimeoutError`` flakes that motivated this lifecycle pass.
26+
1927
**Deprecated**
2028

2129
**Removed**

src/roslibpy/comm/comm_autobahn.py

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,37 @@ def set_max_retries(cls, max_retries):
156156
cls.maxRetries = max_retries
157157

158158

159+
# The twisted reactor is a process-wide singleton, so the log observer that
160+
# fans twisted's log events into the stdlib `logging` module is one too. Any
161+
# Python process only ever needs one of these registered — registering more
162+
# means every log event walks an O(n) observer list and ``globalLogPublisher``
163+
# keeps strong references to every observer for the life of the process.
164+
#
165+
# Prior to 2.1, ``TwistedEventLoopManager.__init__`` started a fresh
166+
# ``PythonLoggingObserver`` per instance, and downstream wrappers that call
167+
# ``Ros.close()`` (rather than ``Ros.terminate()``, which stops the reactor
168+
# permanently) never invoked ``manager.terminate()`` and so never stopped the
169+
# observer. Long-running pytest sessions that create many ``Ros`` instances
170+
# would accumulate one observer per ``Ros``, slowing every log emission
171+
# linearly and contributing to occasional ``RosTimeoutError`` flakes on the
172+
# next ``Ros.run()``.
173+
_LOG_OBSERVER = None
174+
_LOG_OBSERVER_LOCK = threading.Lock()
175+
176+
177+
def _ensure_log_observer_started():
178+
"""Lazily start the single process-wide twisted log → stdlib logging bridge."""
179+
global _LOG_OBSERVER
180+
if _LOG_OBSERVER is not None:
181+
return
182+
with _LOG_OBSERVER_LOCK:
183+
if _LOG_OBSERVER is not None:
184+
return
185+
observer = log.PythonLoggingObserver()
186+
observer.start()
187+
_LOG_OBSERVER = observer
188+
189+
159190
class TwistedEventLoopManager(object):
160191
"""Manage the main event loop using Twisted reactor.
161192
@@ -166,8 +197,11 @@ class TwistedEventLoopManager(object):
166197
"""
167198

168199
def __init__(self):
169-
self._log_observer = log.PythonLoggingObserver()
170-
self._log_observer.start()
200+
# Twisted's reactor is a process singleton — so is the observer that
201+
# bridges twisted's log events into stdlib `logging`. See the module-
202+
# level comment for the lifecycle details.
203+
_ensure_log_observer_started()
204+
self._log_observer = _LOG_OBSERVER
171205

172206
def run(self):
173207
"""Kick-starts a non-blocking event loop.
@@ -273,4 +307,13 @@ def terminate(self):
273307
if self._thread:
274308
self._thread.join()
275309

276-
self._log_observer.stop()
310+
# The observer is the process-wide singleton (see module top). Stop
311+
# it once on terminate — the reactor cannot be restarted after stop,
312+
# so no future TwistedEventLoopManager will need the bridge in this
313+
# process. Subsequent terminates no-op.
314+
global _LOG_OBSERVER
315+
with _LOG_OBSERVER_LOCK:
316+
if _LOG_OBSERVER is not None:
317+
_LOG_OBSERVER.stop()
318+
_LOG_OBSERVER = None
319+
self._log_observer = None

tests/ros1/test_lifecycle.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
"""Lifecycle regression tests for the 2.1 fixes.
2+
3+
Targets bugs that emerged from downstream pytest sessions (``compas_fab``)
4+
where many sequential ``Ros`` instances are constructed, run, and closed
5+
in the same Python process.
6+
7+
If these tests flake, the corresponding fix is regressing — diagnose,
8+
don't retry-loop.
9+
"""
10+
11+
from __future__ import print_function
12+
13+
from roslibpy import Ros
14+
15+
HOST = "127.0.0.1"
16+
PORT = 9090
17+
URL = "ws://%s:%d" % (HOST, PORT)
18+
19+
20+
def test_log_observer_does_not_leak():
21+
"""Constructing many ``Ros`` instances must not leak twisted log observers.
22+
23+
Before A1 every ``TwistedEventLoopManager.__init__`` (created lazily
24+
one-per-``Ros``) called ``PythonLoggingObserver().start()`` and never
25+
cleaned up unless ``manager.terminate()`` was explicitly called.
26+
Downstream wrappers (compas_fab's ``RosClient.__exit__``) call
27+
``close()``, not ``terminate()`` — by design, since ``terminate()``
28+
stops the reactor permanently. So observers piled up forever.
29+
30+
After A1 there's exactly one process-wide observer regardless of how
31+
many ``Ros()`` instances come and go.
32+
"""
33+
from twisted.logger import globalLogPublisher
34+
35+
initial = len(list(globalLogPublisher._observers))
36+
37+
for _ in range(10):
38+
ros = Ros(URL)
39+
ros.run()
40+
ros.close()
41+
42+
final = len(list(globalLogPublisher._observers))
43+
# Allow a single-observer drift in case the very first cycle is the one
44+
# that registers the singleton. The leak case grows by len(cycles).
45+
assert final - initial <= 1, (
46+
"Leaked %d twisted log observers across 10 lifecycle cycles "
47+
"(expected at most 1)" % (final - initial)
48+
)
49+
50+

0 commit comments

Comments
 (0)