fix(client_reqrep): clear _traces in close/release to break session reference cycle#12669
Open
prshant70 wants to merge 4 commits into
Open
fix(client_reqrep): clear _traces in close/release to break session reference cycle#12669prshant70 wants to merge 4 commits into
prshant70 wants to merge 4 commits into
Conversation
…r ref - add `self._traces = []` in `_cleanup_writer()` after `self._session = None` - this deterministically breaks the ClientResponse->_traces->Trace->_session cycle on every close/release/eof path, avoiding slow RSS growth (aio-libs#11043) - add tests verifying _traces and _session are cleared via close(), release(), and _response_eof(); add weakref test confirming the cycle is broken
94fe253 to
9f1b471
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #12669 +/- ##
=======================================
Coverage 98.95% 98.95%
=======================================
Files 131 131
Lines 46688 46731 +43
Branches 2421 2421
=======================================
+ Hits 46200 46243 +43
Misses 366 366
Partials 122 122
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Merging this PR will not alter performance
Comparing Footnotes
|
_response_eof() fires synchronously during payload EOF, which happens inside read() before read() iterates _traces to send callbacks. Clearing _traces there silently dropped on_response_chunk_received hooks, breaking test_request_tracing and test_request_tracing_url_params. Move self._traces = [] to close() and release() instead — both are called after the user has finished reading, so the cycle is still broken deterministically without interfering with tracing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Clears
self._traces = []inClientResponse.close()andClientResponse.release()to deterministically break a circular reference that causes slow memory growth (issue #11043).A
CHANGES/11043.bugfixtowncrier fragment is also included.Closes #11043
Why
Issue #11043 reports slow, steady RSS growth when making repeated HTTP requests with tracing enabled. The root cause is a reference cycle that prevents CPython's reference-counting from reclaiming
ClientResponseobjects promptly:_cleanup_writer()already clearsself._session = None, butself._traceswas never cleared, keeping everyTrace(and therefore the session) alive until the cycle GC ran — which is delayed and non-deterministic under load.How
self._traces = []is added toclose()andrelease()— not to_cleanup_writer().The reason
_cleanup_writer()is excluded: it is also called from_response_eof(), which fires synchronously as a payload EOF callback duringawait self.content.read(). At that pointread()has not yet iterated_tracesto fireon_response_chunk_receivedhooks. Clearing_tracesthere silently dropped all response chunk trace callbacks (confirmed bytest_request_tracingandtest_request_tracing_url_paramsregressions).close()andrelease()are always called after the user finishes consuming the response (via context manager__aexit__), so the cycle is still broken deterministically — just at the right time.Key files:
aiohttp/client_reqrep.py—self._traces = []added toclose()andrelease()CHANGES/11043.bugfix— towncrier changelog fragmenttests/test_client_response.py— four regression tests (including one that asserts_response_eof()intentionally does NOT clear_traces)Testing
New tests:
test_close_clears_traces_and_session—_traces == []and_session is Noneafterclose()test_release_clears_traces_and_session— same afterrelease()test_response_eof_clears_session_but_not_traces—_session is Nonebut_tracespreserved after_response_eof(), so trace callbacks can still firetest_no_circular_ref_after_close— CPython-only weakref test confirming the object is collected afterclose()Checklist
pytest tests/test_client_response.py tests/test_client_session.py -q)CHANGES/11043.bugfix)