[mypyc] Backport cached-group Extension.depends fix from upstream#11
Merged
Conversation
The cross-group header-deps work that shipped in 2.1.0.post1 was upstreamed and merged as ab8e4bf, but the improvements added during review never flowed back to release-2.1: - mypyc_build: when a fully-cached group returns its cfile name with empty contents, re-read the on-disk .c before calling get_header_deps. The existing fallback only covered groups that return no cfile entries at all, so Extension.depends stayed empty for the shape that actually occurs and setuptools never recompiled stale consumer .o files when a dep's export-table struct layout shifted. - get_header_deps: assert non-empty contents to keep this from regressing silently. - fudge_dir_mtimes: stop shifting linker outputs back; combined with write_file's +1s bump this made every .c permanently newer than its .so, forcing unconditional rebuilds that masked depends bugs in tests. - Add the testIncrementalCrossGroupExportTableOffsets regression test. This is the bug behind the sqlglot CI segfault: a cached sqlglotc wheel shipped a stale qualify.o whose quote_identifiers slot dispatched into qualify_outputs after a new function was inserted mid-struct in qualify_columns' export table.
VaggelisD
approved these changes
Jun 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The cross-group header-deps work that shipped in 2.1.0.post1 was upstreamed and merged (squash commit
ab8e4bf1305416372ea70076193a766800f26410), but the improvements added during review never flowed back torelease-2.1. The release branch only carries the original pre-review commits.The missing piece matters in practice: when a group is fully cached, mypyc returns its cfile name with empty contents. The per-cfile loop then calls
get_header_depson the empty string, soExtension.dependsends up empty for every cached group. The existing fallback (749ff715c) only covers groups that return no cfile entries at all, which is not the shape that occurs under pip's multi-invocation build.With empty depends, setuptools'
newer_groupcheck never sees changed cross-group export-table headers, and stale consumer.ofiles get linked against rebuilt dependencies whose struct layout shifted. This is the bug behind the sqlglot CI segfault: a cached sqlglotc wheel shipped a stalequalify.owhosequote_identifiersexport-table slot dispatched intoqualify_outputsafter_add_ilike_columnswas inserted mid-struct inqualify_columns' export table.Fix (verbatim backport of the review-added hunks)
mypyc_build: whenctextis empty for a cached group, re-read the on-disk.cbefore computing header deps.get_header_deps: assert non-empty contents so this can't regress silently.fudge_dir_mtimes: stop shifting linker outputs back. Combined withwrite_file's +1s bump, the old behavior made every.cpermanently newer than its.so, forcing unconditional rebuilds that masked depends bugs in the test harness.testIncrementalCrossGroupExportTableOffsetsregression test.Verification
fudge_dir_mtimesskipbuild.pyfixAll 3 test variants pass,
test_misc.py8/8, and the incremental/cross-module/separate run-test slice passes 97/97.