Skip to content

Commit 6163a98

Browse files
committed
[mypyc] Fix non-deterministic class struct layout under separate=True
detect_undefined_bitmap() was extending cl.bitmap_attrs in place. Under separate=True each SCC's analyze_always_defined_attrs is invoked once per group, and detect_undefined_bitmap recurses through cl.base_mro from the subclass into its base classes. The seen set passed in dedupes within one call but is fresh per call, so every subclass-group call re-extends the shared base class's bitmap_attrs with another copy of the contributions. The base class's emitted ObjectStruct then grows by one bitmap field per ~32 subclasses processed in the same build. The exact final length is a function of how many SCCs went through compile_scc_to_ir this run: - clean build: every SCC fresh -> base bitmap_attrs accumulates fully - incremental build affecting N subclasses: base accumulates a fraction - second incremental: yet another count Subclasses not rebuilt this round still see their base's old, larger struct layout. Any attribute access on the base segfaults with a mismatched bitmap-field offset. Pre-existing in mypyc; only manifested once the prior over-conservative 44-file always-rebuild was lifted (1.20.0.post5), because that wasteful behavior kept rebuild sets self-consistent. Fix: compute a fresh local list and assign at the end. The function becomes naturally idempotent across repeated calls — same input, same output, regardless of how many groups have visited the class. No new fields, no serialization changes. Verified against sqlglot[c] (separate=True, ~100 modules): Edit: add a method to MySQLParser (a class with 7 dialect subclasses) Before: parser.h struct layout differs between clean and incremental builds; make unitc segfaults at first parser-using test. After: parser.h identical between clean and incremental; make unitc passes (1163 tests, 0 segfaults).
1 parent 25996f1 commit 6163a98

1 file changed

Lines changed: 16 additions & 4 deletions

File tree

mypyc/analysis/attrdefined.py

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -424,14 +424,26 @@ def detect_undefined_bitmap(cl: ClassIR, seen: set[ClassIR]) -> None:
424424
for base in cl.base_mro[1:]:
425425
detect_undefined_bitmap(base, seen)
426426

427+
# Compute a fresh list rather than appending to cl.bitmap_attrs in place.
428+
# Under separate=True each SCC's analyze_always_defined_attrs recurses
429+
# through shared base classes (the `seen` set above only dedupes within
430+
# one call), so every subclass group would otherwise re-extend the base's
431+
# bitmap_attrs with another copy of the contributions. The base's emitted
432+
# struct size would then grow with each call, and later incremental builds
433+
# would observe a different (smaller) count and emit a different layout
434+
# for the same class — leaving any not-rebuilt subclass with a stale view
435+
# of the base's struct, which segfaults on attribute access. Recomputing
436+
# fresh makes the function naturally idempotent.
437+
new_attrs: list[str] = []
427438
if len(cl.base_mro) > 1:
428-
cl.bitmap_attrs.extend(cl.base_mro[1].bitmap_attrs)
439+
new_attrs.extend(cl.base_mro[1].bitmap_attrs)
429440
for n, t in cl.attributes.items():
430441
if t.error_overlap and not cl.is_always_defined(n):
431-
cl.bitmap_attrs.append(n)
442+
new_attrs.append(n)
432443

433444
for base in cl.mro[1:]:
434445
if base.is_trait:
435446
for n, t in base.attributes.items():
436-
if t.error_overlap and not cl.is_always_defined(n) and n not in cl.bitmap_attrs:
437-
cl.bitmap_attrs.append(n)
447+
if t.error_overlap and not cl.is_always_defined(n) and n not in new_attrs:
448+
new_attrs.append(n)
449+
cl.bitmap_attrs = new_attrs

0 commit comments

Comments
 (0)