Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information#136790
Merged
encukou merged 3 commits intopython:mainfrom Jul 19, 2025
Merged
Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information#136790encukou merged 3 commits intopython:mainfrom
encukou merged 3 commits intopython:mainfrom
Conversation
…tion This commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. This code can be seen here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110 While the annotation macro is defined here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598 And the corresponding attribute is documented in: * for gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute * for clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis
encukou
reviewed
Jul 19, 2025
encukou
approved these changes
Jul 19, 2025
Member
|
Thank you the PR, and for taking the time to make it this short! |
|
Thanks @disconnect3d for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14. |
|
Thanks @disconnect3d for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Jul 19, 2025
…tion (pythonGH-136790) * Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information This commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. This code can be seen here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110 While the annotation macro is defined here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598 And the corresponding attribute is documented in: * for gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute * for clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis * Update Doc/c-api/memory.rst * Improve --with-address-sanitizer and pymalloc docs --------- (cherry picked from commit d19bb44) Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Jul 19, 2025
…tion (pythonGH-136790) * Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information This commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. This code can be seen here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110 While the annotation macro is defined here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598 And the corresponding attribute is documented in: * for gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute * for clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis * Update Doc/c-api/memory.rst * Improve --with-address-sanitizer and pymalloc docs --------- (cherry picked from commit d19bb44) Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
|
GH-136798 is a backport of this pull request to the 3.14 branch. |
|
GH-136799 is a backport of this pull request to the 3.13 branch. |
encukou
added a commit
that referenced
this pull request
Jul 20, 2025
…information (GH-136790) (GH-136798) Extend the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. (cherry picked from commit d19bb44) Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
encukou
added a commit
that referenced
this pull request
Jul 20, 2025
…information (GH-136790) (#136799) Extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). (cherry picked from commit d19bb44) Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
taegyunkim
pushed a commit
to taegyunkim/cpython
that referenced
this pull request
Aug 4, 2025
…tion (pythonGH-136790) * Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information This commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. This code can be seen here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110 While the annotation macro is defined here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598 And the corresponding attribute is documented in: * for gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute * for clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis * Update Doc/c-api/memory.rst * Improve --with-address-sanitizer and pymalloc docs --------- Co-authored-by: Petr Viktorin <encukou@gmail.com>
Agent-Hellboy
pushed a commit
to Agent-Hellboy/cpython
that referenced
this pull request
Aug 19, 2025
…tion (pythonGH-136790) * Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information This commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. This code can be seen here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110 While the annotation macro is defined here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598 And the corresponding attribute is documented in: * for gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute * for clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis * Update Doc/c-api/memory.rst * Improve --with-address-sanitizer and pymalloc docs --------- Co-authored-by: Petr Viktorin <encukou@gmail.com>
kumaraditya303
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Sep 9, 2025
…information (pythonGH-136790) (pythonGH-136798) Extend the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. (cherry picked from commit d19bb44) Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit extends the documentation for disabling pymalloc with the
--without-pymallocflag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the
python -m testsuite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the
--without-pymallocflag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as theaddress_in_rangefunction used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:
cpython/Objects/obmalloc.c
Lines 2096 to 2110 in acefb97
While the annotation macro is defined here:
cpython/Include/pyport.h
Lines 582 to 598 in acefb97
And the corresponding attribute is documented in:
📚 Documentation preview 📚: https://cpython-previews--136790.org.readthedocs.build/