Skip to content

[mypyc] Backport cached-group Extension.depends fix from upstream#11

Merged
georgesittas merged 1 commit into
release-2.1from
fix/cached-group-header-deps
Jun 14, 2026
Merged

[mypyc] Backport cached-group Extension.depends fix from upstream#11
georgesittas merged 1 commit into
release-2.1from
fix/cached-group-header-deps

Conversation

@georgesittas

Copy link
Copy Markdown
Collaborator

Problem

The cross-group header-deps work that shipped in 2.1.0.post1 was upstreamed and merged (squash commit ab8e4bf1305416372ea70076193a766800f26410), but the improvements added during review never flowed back to release-2.1. The release branch only carries the original pre-review commits.

The missing piece matters in practice: when a group is fully cached, mypyc returns its cfile name with empty contents. The per-cfile loop then calls get_header_deps on the empty string, so Extension.depends ends up empty for every cached group. The existing fallback (749ff715c) only covers groups that return no cfile entries at all, which is not the shape that occurs under pip's multi-invocation build.

With empty depends, setuptools' newer_group check never sees changed cross-group export-table headers, and stale consumer .o files get linked against rebuilt dependencies whose struct layout shifted. This is the bug behind the sqlglot CI segfault: a cached sqlglotc wheel shipped a stale qualify.o whose quote_identifiers export-table slot dispatched into qualify_outputs after _add_ilike_columns was inserted mid-struct in qualify_columns' export table.

Fix (verbatim backport of the review-added hunks)

  • mypyc_build: when ctext is empty for a cached group, re-read the on-disk .c before computing header deps.
  • get_header_deps: assert non-empty contents so this can't regress silently.
  • fudge_dir_mtimes: stop shifting linker outputs back. Combined with write_file's +1s bump, the old behavior made every .c permanently newer than its .so, forcing unconditional rebuilds that masked depends bugs in the test harness.
  • Add the testIncrementalCrossGroupExportTableOffsets regression test.

Verification

fudge_dir_mtimes skip build.py fix result
no no passes (bug masked)
yes no fails (wrong class returned from cross-group call)
yes yes passes

All 3 test variants pass, test_misc.py 8/8, and the incremental/cross-module/separate run-test slice passes 97/97.

The cross-group header-deps work that shipped in 2.1.0.post1 was
upstreamed and merged as ab8e4bf, but
the improvements added during review never flowed back to release-2.1:

- mypyc_build: when a fully-cached group returns its cfile name with
  empty contents, re-read the on-disk .c before calling get_header_deps.
  The existing fallback only covered groups that return no cfile entries
  at all, so Extension.depends stayed empty for the shape that actually
  occurs and setuptools never recompiled stale consumer .o files when a
  dep's export-table struct layout shifted.
- get_header_deps: assert non-empty contents to keep this from
  regressing silently.
- fudge_dir_mtimes: stop shifting linker outputs back; combined with
  write_file's +1s bump this made every .c permanently newer than its
  .so, forcing unconditional rebuilds that masked depends bugs in tests.
- Add the testIncrementalCrossGroupExportTableOffsets regression test.

This is the bug behind the sqlglot CI segfault: a cached sqlglotc wheel
shipped a stale qualify.o whose quote_identifiers slot dispatched into
qualify_outputs after a new function was inserted mid-struct in
qualify_columns' export table.
@georgesittas
georgesittas merged commit 4732c1c into release-2.1 Jun 14, 2026
16 checks passed
@georgesittas
georgesittas deleted the fix/cached-group-header-deps branch June 14, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants