Skip to content

fix(client_reqrep): clear _traces in close/release to break session reference cycle#12669

Open
prshant70 wants to merge 4 commits into
aio-libs:masterfrom
prshant70:fix/client-response-circular-ref-cleanup
Open

fix(client_reqrep): clear _traces in close/release to break session reference cycle#12669
prshant70 wants to merge 4 commits into
aio-libs:masterfrom
prshant70:fix/client-response-circular-ref-cleanup

Conversation

@prshant70
Copy link
Copy Markdown

@prshant70 prshant70 commented May 20, 2026

What

Clears self._traces = [] in ClientResponse.close() and ClientResponse.release() to deterministically break a circular reference that causes slow memory growth (issue #11043).

A CHANGES/11043.bugfix towncrier fragment is also included.

Closes #11043

Why

Issue #11043 reports slow, steady RSS growth when making repeated HTTP requests with tracing enabled. The root cause is a reference cycle that prevents CPython's reference-counting from reclaiming ClientResponse objects promptly:

ClientResponse._traces  ->  [Trace, ...]
Trace._session          ->  ClientSession
ClientSession           ->  (keeps other things alive)

_cleanup_writer() already clears self._session = None, but self._traces was never cleared, keeping every Trace (and therefore the session) alive until the cycle GC ran — which is delayed and non-deterministic under load.

How

self._traces = [] is added to close() and release()not to _cleanup_writer().

The reason _cleanup_writer() is excluded: it is also called from _response_eof(), which fires synchronously as a payload EOF callback during await self.content.read(). At that point read() has not yet iterated _traces to fire on_response_chunk_received hooks. Clearing _traces there silently dropped all response chunk trace callbacks (confirmed by test_request_tracing and test_request_tracing_url_params regressions).

close() and release() are always called after the user finishes consuming the response (via context manager __aexit__), so the cycle is still broken deterministically — just at the right time.

Key files:

  • aiohttp/client_reqrep.pyself._traces = [] added to close() and release()
  • CHANGES/11043.bugfix — towncrier changelog fragment
  • tests/test_client_response.py — four regression tests (including one that asserts _response_eof() intentionally does NOT clear _traces)

Testing

pytest tests/test_client_response.py tests/test_client_session.py -q

New tests:

  • test_close_clears_traces_and_session_traces == [] and _session is None after close()
  • test_release_clears_traces_and_session — same after release()
  • test_response_eof_clears_session_but_not_traces_session is None but _traces preserved after _response_eof(), so trace callbacks can still fire
  • test_no_circular_ref_after_close — CPython-only weakref test confirming the object is collected after close()

Checklist

  • Tests pass (pytest tests/test_client_response.py tests/test_client_session.py -q)
  • Changelog entry added (CHANGES/11043.bugfix)
  • No new abstractions or unnecessary changes

@prshant70 prshant70 requested a review from asvetlov as a code owner May 20, 2026 17:54
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label May 20, 2026
…r ref

- add `self._traces = []` in `_cleanup_writer()` after `self._session = None`
- this deterministically breaks the ClientResponse->_traces->Trace->_session
  cycle on every close/release/eof path, avoiding slow RSS growth (aio-libs#11043)
- add tests verifying _traces and _session are cleared via close(), release(),
  and _response_eof(); add weakref test confirming the cycle is broken
@prshant70 prshant70 force-pushed the fix/client-response-circular-ref-cleanup branch from 94fe253 to 9f1b471 Compare May 20, 2026 18:14
@codecov
Copy link
Copy Markdown

codecov Bot commented May 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.95%. Comparing base (1f005f1) to head (07ec948).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #12669   +/-   ##
=======================================
  Coverage   98.95%   98.95%           
=======================================
  Files         131      131           
  Lines       46688    46731   +43     
  Branches     2421     2421           
=======================================
+ Hits        46200    46243   +43     
  Misses        366      366           
  Partials      122      122           
Flag Coverage Δ
Autobahn 22.41% <16.27%> (-0.01%) ⬇️
CI-GHA 98.92% <100.00%> (+<0.01%) ⬆️
OS-Linux 98.67% <100.00%> (-0.01%) ⬇️
OS-Windows 97.04% <100.00%> (+<0.01%) ⬆️
OS-macOS 97.93% <100.00%> (-0.01%) ⬇️
Py-3.10 98.16% <100.00%> (+<0.01%) ⬆️
Py-3.11 98.41% <100.00%> (-0.01%) ⬇️
Py-3.12 98.50% <100.00%> (-0.01%) ⬇️
Py-3.13 98.48% <100.00%> (-0.01%) ⬇️
Py-3.14 98.49% <100.00%> (-0.01%) ⬇️
Py-3.14t 97.55% <100.00%> (+<0.01%) ⬆️
Py-pypy-3.11 97.40% <76.74%> (-0.04%) ⬇️
VM-macos 97.93% <100.00%> (-0.01%) ⬇️
VM-ubuntu 98.67% <100.00%> (-0.01%) ⬇️
VM-windows 97.04% <100.00%> (+<0.01%) ⬆️
cython-coverage 37.91% <4.65%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 20, 2026

Merging this PR will not alter performance

✅ 72 untouched benchmarks
⏩ 72 skipped benchmarks1


Comparing prshant70:fix/client-response-circular-ref-cleanup (07ec948) with master (1f005f1)

Open in CodSpeed

Footnotes

  1. 72 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@prshant70 prshant70 marked this pull request as draft May 20, 2026 18:26
_response_eof() fires synchronously during payload EOF, which happens
inside read() before read() iterates _traces to send callbacks. Clearing
_traces there silently dropped on_response_chunk_received hooks, breaking
test_request_tracing and test_request_tracing_url_params.

Move self._traces = [] to close() and release() instead — both are called
after the user has finished reading, so the cycle is still broken
deterministically without interfering with tracing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@prshant70 prshant70 marked this pull request as ready for review May 20, 2026 18:35
@prshant70 prshant70 marked this pull request as draft May 20, 2026 18:36
@prshant70 prshant70 changed the title fix(client_reqrep): clear _traces in _cleanup_writer to break session reference cycle fix(client_reqrep): clear _traces in close/release to break session reference cycle May 20, 2026
@prshant70 prshant70 marked this pull request as ready for review May 20, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aiohttp post memory leak?

1 participant