Value-Constrained TypeVar incorrectly narrowed down by isinstance checks

[wpeale]

Bug Report

When using a TypeVar with a value restriction to multiple classes, isinstance checks seem to incorrectly narrow down the TypeVar. Possibly connected to #10287

To Reproduce

from typing import TypeVar


class A:
    pass


class B:
    pass


OperableT = TypeVar(
    "OperableT",
    A,
    B,
)


class Operator:
    def process(self, obj: OperableT) -> OperableT:
        if isinstance(obj, A):
            return A()
        elif isinstance(obj, B):
            return B()

Expected Behavior

Mypy should be able to resolve the type of the OperableT TypeVar and correctly validate the return value in the process() method.

Actual Behavior

Mypy instead throws a return error for the above:
error: Incompatible return value type (got "A", expected "B") [return-value]

For some reason, in the first isinstance check, mypy seems to resolve the OperableT TypeVar to "B" instead of "A."

• Mypy version used: 0.770 (tested also with 0.812)
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.8.2
• Operating system and version: Ubuntu 20.04

[erictraut]

I think mypy's behavior is correct in this case, although the error message is a bit confusing.

The signature for process indicates that the return type of this function exactly matches the type of obj. The isinstance check guarantees that obj is a subtype of A, but it doesn't guarantee that it's of type A. It could be a subclass of A, for example, so a type checker must indicate a type error if you return A in this case.

Here's an alternative solution that might meet your needs:

class Operator:
    @overload
    def process(self, obj: A) -> A: ...
    @overload
    def process(self, obj: B) -> B: ...
    def process(self, obj: Union[A, B]) -> Union[A, B]:
        if isinstance(obj, A):
            return A()
        elif isinstance(obj, B):
            return B()

[noah-built]

@erictraut Thank you for the help! That makes sense, and we can switch to using @overload if needed. But this further example seems to resolve the subclass issue, and we still get a seemingly incorrect mypy error?

from typing import TypeVar


class A:
    pass


class B:
    pass


OperableT = TypeVar("OperableT", A, B)


class Operator:
    def process(self, obj: OperableT) -> OperableT:
        if type(obj) == A:
            return A()
        elif type(obj) == B:
            return B()
        else:
            raise TypeError

Results:

mypy_test.py:18: error: Incompatible return value type (got "A", expected "B")  [return-value]
                return A()
                       ^
mypy_test.py:20: error: Incompatible return value type (got "B", expected "A")  [return-value]
                return B()
                       ^

[hauntsaninja]

Yeah, see #7260 and #4445 for that one

[noah-built]

Got it -- the fact that the errors appear does make sense then, it just seems like the error messages are pretty divorced from the underlying issues.

Thanks for the help!

[DetachHead]

i think this issue can be closed as the behavior is correct. i raised #11536 for the bad error message

alternatively we can rename this issue and close mine?

[greenatatlassian]

The two linked issues regarding type narrowing using type(A) is A and type(A) == A are resolved, yet the code above still does not pass type checking. I believe there is still an underlying bug here.

Consider this even simpler, though ultimately very similar, code:

from typing import TypeVar, Type

class A:
    pass

class B:
    pass

AB_Type = TypeVar('AB_Type', A, B)

def select(a_or_b: AB_Type) -> Type[AB_Type]:
    if isinstance(a_or_b, A):
        return A
    elif isinstance(a_or_b, B):
        return B
    else:
        raise TypeError()

For this code, mypy complains of a type error on only one (!?) branch of the select:

main.py:13: error: Incompatible return value type (got "type[A]", expected "type[B]")  [return-value]
Found 1 error in 1 file (checked 1 source file)

Note that in line 13, the type is already correctly narrowed to A by the is instance check, so how can the generic also have the value B? (The behaviour is identical if the type narrowing is done with type(a_or_b) is A.)

I do not believe the original argument that:

The signature for process indicates that the return type of this function exactly matches the type of obj.

is correct. In all of the examples, a constrained type is used. Constrained types "can only ever be solved as being exactly one of the constraints given". So the signature (in my example) must solve to either:
def select(a_or_b: A) -> Type[A] or def select(a_or_b: B) -> Type[B], but not def select(a_or_b: A) -> Type[B]. Simply replacing the AB_Type with A and B (as should be the valid solutions for AB_Type) both pass type checking with no issue, so this can't be a subclassing problem.

If I change this code to be intentionally wrong, then I get an error that hints at what the bug might be. If, in my code above, I change line 13 to be return A(), and the same for line 15, then I get this error:

main.py:13: error: Incompatible return value type (got "A", expected "type[A]")  [return-value]
main.py:13: error: Incompatible return value type (got "A", expected "type[B]")  [return-value]
main.py:15: error: Incompatible return value type (got "B", expected "type[B]")  [return-value]

The first and last are as expected, but the middle one is unexpected—it looks like mypy is trying to check the case where the solutions for AB_Type don't match, i.e. the signature def select(a_or_b: A) -> Type[B]. The code given would indeed fail checking with that signature, but that signature should not be allowed for a constrained type (i.e. AB_Type must be either A or B eveywhere in the signature.)

[hauntsaninja]

#20703 fixes noah-built's case here: #10302 (comment)

greenatatlassian's case still points out an issue

[ilevkivskyi]

I think the original example as well as example of @greenatatlassian are unsafe. Consider this:

from typing import TypeVar, Type

class A:
    pass
class B:
    x = 1

AB_Type = TypeVar('AB_Type', A, B)

def select(a_or_b: AB_Type) -> Type[AB_Type]:
    if isinstance(a_or_b, A):
        return A
    elif isinstance(a_or_b, B):
        return B
    else:
        raise TypeError()

def test(x: B) -> None:
    select(x).x

class Surprise(A, B): pass
test(Surprise())  # AttributeError

Although mypy behavior is confusing, it is correct: the problem arises only in the first if-branch, since this is the branch that will be taken by a class that subclasses both A and B.

I know that mypy intentionally turns a blind eye on similar examples with overloads, etc. But existing cases where we intentionally ignore multiple inheritance are motivated by very common patterns. This doesn't look like a common pattern, so IMO doesn't justify a hole in the type system. And since @hauntsaninja already fixed the other (safe) example, I would propose to close this issue.

[hauntsaninja]

the problem arises only in the first if-branch

Ah, I didn't think about it much and just assumed the single branch reporting was weird, but this makes sense.