Skip to content

[mypyc] Record MRO ancestor modules as deps of subclass-defining modules#12

Merged
geooo109 merged 2 commits into
release-2.1from
class-ancestor-deps
Jul 17, 2026
Merged

[mypyc] Record MRO ancestor modules as deps of subclass-defining modules#12
geooo109 merged 2 commits into
release-2.1from
class-ancestor-deps

Conversation

@geooo109

@geooo109 geooo109 commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

(Context: surfaced by this sqlglot CI failure)

Mypy's incremental system only recompiles a module when a recorded dependency changes, and a dependency edge to an ancestor's module is created in cases such as an explicit import or a checked expression whose type references that ancestor. A bare subclass definition produces neither, so the transitive ancestors never enter the module's dependency graph:

# a.py
class A:
    def foo1(self) -> int: ...
    def foo2(self) -> int ... <- we remove this method

# b.py
from a import A
class B(A):
    def other(self) -> str: ...

# c.py
from b import B
class C(B): pass <- no dependecy on A, only on B

For type checking ^ is fine: a change in a distant ancestor doesn't affect the subclass's interface. But mypyc's generated C embeds the full layout of every ancestor into the subclass, including a vtable slot for each inherited method and struct offsets for inherited attributes. Thus, the generated C depends on ancestors that the dependency graph doesn't capture.

The goal of this PR is to make the dependency graph reflect what the generated C actually depends on. When compiling with mypyc, a module that defines a class now records the modules defining every ancestor in that class's MRO as indirect dependencies, so an ancestor's interface change reaches the subclass-defining module directly.

Under separate=True incremental builds, a module whose only content is a
subclass definition (e.g. `class DuneParser(TrinoParser): pass`) recorded
no dependency on the modules defining its transitive bases: it does not
import them, and it produces no expression types, so the existing
indirect-dependency machinery (TypeIndirectionVisitor, which walks the MRO
of every checked expression's type) never fires for it. The same holds for
subclass bodies containing only constants, annotations, or methods that
never use self.

mypyc's generated C embeds every ancestor's layout (vtable arrays reference
each inherited method by name through the ancestor's export table), so when
a base class several hops up changed its method set, interface-hash
propagation stopped at intermediate modules whose own interfaces were
unchanged, the defining module stayed fresh, and its stale generated C was
compiled against the ancestor's regenerated header:

  error: 'struct export_table_sqlglot___parser' has no member named
  'CPyDef_sqlglot___parser___Parser____parse_max_min_by'

Surfaced by the tobymao/sqlglot#7875 CI failure:
https://github.com/tobymao/sqlglot/actions/runs/29492925074/job/87603095733

Removing a base method fails at C compile time; adding one risks
inconsistent vtable layouts; the staleness can also surface as a KeyError
deserializing the stale IR cache's vtable.

Fix: in State.finish_passes, for mypyc-compiled modules only, record the
modules defining each MRO ancestor of every class defined in the module as
indirect dependencies (State.compiled_class_ancestor_refs). The names flow
through the existing patch_indirect_dependencies sink (deduped, PRI_INDIRECT,
hash-snapshotted), so no new mechanism is introduced; this completes the
existing design symmetrically with use sites. Plain mypy behavior is
unchanged.

Tests: testIncrementalCrossGroupInheritedMethodRemoved and
testIncrementalCrossGroupInheritedMethodRemovedNonEmptyLeaf, both failing
before and passing after the change; full TestRunSeparate suite green.

This comment was marked as resolved.

@geooo109
geooo109 merged commit 7a2ced7 into release-2.1 Jul 17, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants