Describe the bug
While a raw callback (Connection.on_receive_raw, Connection.on_send_raw, or the Server equivalents) is registered, c104 leaks two Python objects — a memoryview and a bytes — for every frame that passes the hook. The leak is independent of what the Python callback does; an empty callback body leaks at the same rate.
For a long-running client that keeps on_receive_raw registered while receiving continuous telemetry, this grows RSS without bound (in our case ~50 MB/h at a few dozen frames/s, OOM-killing a 1 GiB container every ~18 h).
I believe this is also the real root cause of #49: the memory growth there stopped when the reporter removed their raw debug callbacks, and Point.transmit() itself was never the leak — which is why Valgrind/ASan on the transmit path found nothing.
Root cause
src/remote/Connection.cpp (v2.2.1, unchanged on main), Connection::onReceiveRaw:
Module::ScopedGilAcquire const scoped("Connection.on_receive_raw");
PyObject *pymemview =
PyMemoryView_FromMemory(cp.get(), msgSize, PyBUF_READ); // new reference, never released
PyObject *pybytes = PyBytes_FromObject(pymemview); // new reference
self->py_onReceiveRaw.call(self, py::handle(pybytes)); // py::handle is non-owning
Both new references are never Py_DECREF'd: the memoryview leaks outright, and the bytes is wrapped in a non-owning py::handle, so pybind11 never takes ownership either. The same pattern exists in all four raw-callback dispatch sites:
Connection::onReceiveRaw — src/remote/Connection.cpp
Connection::onSendRaw — src/remote/Connection.cpp
Server::onReceiveRaw — src/Server.cpp
Server::onSendRaw — src/Server.cpp
Suggested fix (I will attempt myself shortly)
PyObject *pymemview = PyMemoryView_FromMemory(cp.get(), msgSize, PyBUF_READ);
PyObject *pybytes = PyBytes_FromObject(pymemview);
Py_DECREF(pymemview);
self->py_onReceiveRaw.call(self, py::reinterpret_steal<py::object>(pybytes));
(py::reinterpret_steal hands the bytes reference to pybind11, which releases it
after the call returns.)
To Reproduce
Self-contained script — local server transmitting ~150 msg/s to a client in the same process; 60 s with an empty on_receive_raw registered, then 60 s unregistered:
import gc
import threading
import time
import tracemalloc
import c104
def rss_kb() -> int:
with open("/proc/self/status") as f:
for line in f:
if line.startswith("VmRSS:"):
return int(line.split()[1])
def on_raw(connection: c104.Connection, data: bytes) -> None:
pass # completely empty: the leak does not depend on the callback body
server = c104.Server(ip="127.0.0.1", port=2404)
station = server.add_station(common_address=47)
srv_points = [station.add_point(io_address=100 + i, type=c104.Type.M_ME_NC_1) for i in range(20)]
server.start()
client = c104.Client()
connection = client.add_connection(ip="127.0.0.1", port=2404, init=c104.Init.NONE)
cl_station = connection.add_station(common_address=47)
for i in range(20):
cl_station.add_point(io_address=100 + i, type=c104.Type.M_ME_NC_1)
client.start()
while not connection.is_connected:
time.sleep(0.1)
stop = threading.Event()
sent = 0
def spam() -> None:
global sent
while not stop.is_set():
pt = srv_points[sent % 20]
pt.value = float(sent % 100)
pt.transmit(cause=c104.Cot.SPONTANEOUS)
sent += 1
time.sleep(1 / 150)
threading.Thread(target=spam, daemon=True).start()
time.sleep(5) # settle
tracemalloc.start()
for phase, registered in (("registered", True), ("unregistered", False)):
connection.on_receive_raw(on_raw if registered else None)
gc.collect()
rss0, traced0, sent0 = rss_kb(), tracemalloc.get_traced_memory()[0], sent
time.sleep(60)
gc.collect()
rss1, traced1, sent1 = rss_kb(), tracemalloc.get_traced_memory()[0], sent
n = sent1 - sent0
print(f"on_receive_raw {phase}: {n} msgs in 60s -> "
f"RSS {rss1 - rss0:+d} kB, tracemalloc {(traced1 - traced0) // 1024:+d} kB "
f"({(traced1 - traced0) // max(n, 1)} B/msg)")
stop.set()
client.stop()
server.stop()
Observed output
on_receive_raw registered: 8609 msgs in 60s -> RSS +6292 kB, tracemalloc +3071 kB (365 B/msg)
on_receive_raw unregistered: 8610 msgs in 60s -> RSS +0 kB, tracemalloc -1 kB (-1 B/msg)
tracemalloc attributes the growth to <unknown>:0 (allocations made from the C extension with no Python traceback), consistent with objects created via the C API and never released. Per-point on_receive callbacks at the same message rate show no growth — only the raw-bytes callbacks are affected.
Expected behavior
Memory usage stays flat while a raw callback is registered.
System
- c104 2.2.1 (PyPI wheel), also inspected on current main — dispatch code unchanged
- CPython 3.13.5, Linux x86_64 (also observed on aarch64 in production)
Describe the bug
While a raw callback (
Connection.on_receive_raw,Connection.on_send_raw, or theServerequivalents) is registered, c104 leaks two Python objects — amemoryviewand abytes— for every frame that passes the hook. The leak is independent of what the Python callback does; an empty callback body leaks at the same rate.For a long-running client that keeps
on_receive_rawregistered while receiving continuous telemetry, this grows RSS without bound (in our case ~50 MB/h at a few dozen frames/s, OOM-killing a 1 GiB container every ~18 h).I believe this is also the real root cause of #49: the memory growth there stopped when the reporter removed their raw debug callbacks, and
Point.transmit()itself was never the leak — which is why Valgrind/ASan on the transmit path found nothing.Root cause
src/remote/Connection.cpp(v2.2.1, unchanged on main),Connection::onReceiveRaw:Both new references are never
Py_DECREF'd: thememoryviewleaks outright, and thebytesis wrapped in a non-owningpy::handle, so pybind11 never takes ownership either. The same pattern exists in all four raw-callback dispatch sites:Connection::onReceiveRaw— src/remote/Connection.cppConnection::onSendRaw— src/remote/Connection.cppServer::onReceiveRaw— src/Server.cppServer::onSendRaw— src/Server.cppSuggested fix (I will attempt myself shortly)
PyObject *pymemview = PyMemoryView_FromMemory(cp.get(), msgSize, PyBUF_READ); PyObject *pybytes = PyBytes_FromObject(pymemview); Py_DECREF(pymemview); self->py_onReceiveRaw.call(self, py::reinterpret_steal<py::object>(pybytes));(
py::reinterpret_stealhands thebytesreference to pybind11, which releases itafter the call returns.)
To Reproduce
Self-contained script — local server transmitting ~150 msg/s to a client in the same process; 60 s with an empty
on_receive_rawregistered, then 60 s unregistered:Observed output
tracemallocattributes the growth to<unknown>:0(allocations made from the C extension with no Python traceback), consistent with objects created via the C API and never released. Per-pointon_receivecallbacks at the same message rate show no growth — only the raw-bytes callbacks are affected.Expected behavior
Memory usage stays flat while a raw callback is registered.
System