Issue 45665: Problems caused by isinstance(list[int], type) returning True - Python tracker

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
process
Status: open Resolution:
Dependencies: 40296 45438 45662 45663 45664 46032 Superseder:
Assigned To: Nosy List: AlexWaygood, gvanrossum, joperez, kj, martinitus, serhiy.storchaka
Priority: normal Keywords:

Created on 2021-10-29 08:27 by serhiy.storchaka, last changed 2022-04-11 14:59 by admin.

Messages (17)
msg405290 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-29 08:27
This is a meta-issue for problems caused by isinstance(list[int]) returning True.

See also discussion in issue45438.
msg405291 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-29 08:41
isinstance(x, type) returns True for instances of types.GenericAlias (like list[int]). While it may help in some cases related to typing, in many unrelated cases it causes problems if the value which is not a type passes checks for types.

Also, isinstance(x, type) been not equal to issubclass(type(x), type) can cause other problems. No matter what the result should be, it should be consistent.

There are many occurrences of isinstance(..., type) in the code.

$ find Lib -name '*.py' \! -path '*/test*' -exec egrep 'isinstance.*,
type\)' '{}' + | wc -l
55

And all of them can potentially be broken if pass a types.GenericAlias instance. Many of them are safe, but every case should be analyzed.
msg405427 - (view) Author: Martin Rueckl (martinitus) * Date: 2021-11-01 10:36
One thing that probably should be considered in this context:

isinstance(arg, type) == issubclass(type(arg), type)

Holds True for arg in (Optional[X], Union[X, Y]). Both sides evaluate to False. (v3.10.0)

While I still think both sides evaluating to True would be more intuitive, this supports the proposed change.

Small test snippet:

```
from typing import Dict, List, Set, Tuple, Optional, Union

import pytest


@pytest.mark.parametrize('arg', [
    list, List[int], list[int],
    dict, Dict[str, int], dict[str, int],
    set, Set[int], set[int],
    tuple, Tuple[str, int], tuple[str, int],
    Optional[int],
    Union[int, str]
])
def test_invariant(arg):
    same = isinstance(arg, type) == issubclass(type(arg), type)
    result = "Check" if same else "Failed"
    print(f"\n{result}: Testing: {arg=} with {type(arg)=}: {isinstance(arg, type)=} <> {issubclass(type(arg), type)=}")
    assert same

```

Any other commonly used annotations that could be added to the checklist?
msg405429 - (view) Author: Martin Rueckl (martinitus) * Date: 2021-11-01 10:42
Sorry for the noise: 
- Literal['a', 'b'],
- TypeVar('T')
Behave like Optional/Union
msg408207 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2021-12-10 13:05
#46032 is related to this issue.
msg409167 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-12-25 04:55
See https://github.com/python/mypy/issues/9773#issuecomment-1000975000. I may have talked myself into agreeing with Serhiy there! It does seem inconsistent that Any is not considered a type but list[int] is:

>>> isinstance(list[int], type)
True
>>> import typing
>>> isinstance(typing.Any, type)
msg409233 - (view) Author: Joseph Perez (joperez) * Date: 2021-12-27 16:42
There is also https://bugs.python.org/issue44293 about inspect.isclass
msg409679 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2022-01-04 14:08
> It does seem inconsistent that Any is not considered a type but list[int] is

Yeah, almost none of the typing "types" are types ever since PEP 560. Issue45755 was a side effect too.
msg409691 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-01-04 17:01
So is it too late to change this? This went out with 3.10, Serhiy has argued it's a bugfix.
msg409694 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2022-01-04 18:00
`isinstance(list[int], type)` returns `True` in 3.9 as well, so the behaviour has been around for a while. (Not a comment on whether the change is worth making, just a note.)
msg409697 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-01-04 18:09
Okay, so it is probably here to stay. We could justify it (after the fact :-) by saying that you can instantiate list[int], but not Any.

Are there exceptions to that? (I.e. are there other annotation types that have isinstance(X, type) return False but can be instantiated, or the other way around?)
msg409701 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2022-01-04 19:44
My plan was to fix as much bugs in the stdlib and backport workaround to 3.9 and 3.10, then propose to revert this "feature" in 3.11.

There is a risk of introducing some regressions by this change, but we can handle it. I think it is better to do it now and fix potential bugs in third-party code (we cannot add workarounds in all third-party code) than keep this weird special case forever.
msg409702 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2022-01-04 19:45
Yes, there are a few exceptions to that :(

```
>>> from typing import Annotated
>>> x = Annotated[int, "idk some int"]
>>> x()
0
>>> isinstance(x, type)
False
>>>
>>> from re import Pattern
>>> y = Pattern[str]
>>> y()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create 're.Pattern' instances
>>> isinstance(y, type)
True
```
msg409703 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2022-01-04 19:47
I agree with Serhiy.
msg409711 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-01-04 21:23
I now agree with Serhiy's plan. We should execute ASAP so people get a chance to try this in the next alpha release.

We will still allow instantiating e.g. list[int], right?
msg409792 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2022-01-05 17:48
Gentle ping, as the next alpha will be release soon
msg409881 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2022-01-06 16:58
> We will still allow instantiating e.g. list[int], right?

I certainly hope so! That would be a much more breaking change if we were to change that, and I can't personally see any benefit to doing so.