gh-142889: Restructure PyDictKeysObject memory layout for simpler access#145097
Open
clintonsteiner wants to merge 12 commits intopython:mainfrom
Open
gh-142889: Restructure PyDictKeysObject memory layout for simpler access#145097clintonsteiner wants to merge 12 commits intopython:mainfrom
clintonsteiner wants to merge 12 commits intopython:mainfrom
Conversation
|
All commit authors signed the Contributor License Agreement. |
…er entry access Restructure dict keys allocation to store dk_indices before the PyDictKeysObject header and keep dk_entries after the header. Update dict index access and related allocation/free/clone paths, adjust gdb dict entry location logic, and add layout coverage tests. Local dict microbenchmarks showed about a 1.4% overall improvement, with most operations around 1-2% faster.
clintonsteiner
force-pushed
the
gh-142889
branch
from
2074c8a to
9eec989
Compare
brijkapadia
reviewed
Feb 22, 2026
markshannon
reviewed
Mar 11, 2026
Member
markshannon
left a comment
markshannon
left a comment
There was a problem hiding this comment.
This looks promising.
I've a few suggestions, but nothing major.
…ucture replace char dk_indices[] with explicit union named dk_entries because union has non-zero size, update allocation and size reporting to use offsetof in place of sizeof, keeping actual memory footprint unchanged remove _DK_INDICES_END(), inline uses in macros adds PyDictKeysObject pointer note to layout diagram for dictobject.c
…ointer arithmetic The previous macros used negative indexing from the PyDictKeysObject pointer (e.g. ((int8_t*)keys)[-1-idx]) to access the indices stored before the struct. This is undefined behavior: the compiler sees keys as typed PyDictKeysObject* and negative indices are formally out-of-bounds, allowing MSVC's release optimizer to generate incorrect hash table lookups. Fix by using _DK_INDICES_BASE(keys) (the actual malloc base) with positive forward indexing, which is well-defined pointer arithmetic within the allocation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restructure dict keys allocation to store dk_indices before the PyDictKeysObject header and keep dk_entries after the header.
Update dict index access and related allocation/free/clone paths, adjust gdb dict entry location logic, and add layout coverage tests.
Local dict microbenchmarks showed about a 1.4% overall improvement, with most operations around 1-2% faster.