list is not equivalent to typing.List under combination of generics and futures

[mark-todd]

Initial Checks

• I have searched GitHub for a duplicate issue and I'm sure this is something new
• I have searched Google & StackOverflow for a solution and couldn't find anything
• I have read and followed the docs and still think this is a bug
• I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description

This is a strange one - under the below conditions, exchanging typing.List for the more modern list produces the below error. I can't reproduce in a simpler example than this - it looks to me like a bug in pydantic:

Traceback (most recent call last):
  File "pydantic/validators.py", line 751, in pydantic.validators.find_validators
TypeError: issubclass() arg 1 must be a class

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "{path_to_test}/test.py", line 11, in <module>
    outer = Outer["Inner"]
  File "{path_to_pydantic}/pydantic/generics.py", line 163, in __class_getitem__
    _prepare_model_fields(created_model, fields, instance_type_hints, typevars_map)
  File "{path_to_pydantic}/pydantic/generics.py", line 389, in _prepare_model_fields
    field.prepare()
  File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 758, in pydantic.fields.ModelField._type_analysis
  File "pydantic/fields.py", line 808, in pydantic.fields.ModelField._create_sub_type
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 557, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 831, in pydantic.fields.ModelField.populate_validators
  File "pydantic/validators.py", line 760, in find_validators
RuntimeError: error checking inheritance of 'Inner' (type: str)

I've also displayed the minimal example below - thanks!

Example Code

from typing import Generic, List, TypeVar

from pydantic import BaseModel
from pydantic.generics import GenericModel

ChangingType = TypeVar("ChangingType")

class Outer(GenericModel, Generic[ChangingType]):
    fields_: list[ChangingType] # Swapping list -> List on this line fixes the bug

outer = Outer["Inner"]
class Inner(BaseModel):
    new: list[outer]

outer.update_forward_refs()
Inner.update_forward_refs()

out = Inner.parse_obj({"new": [{"fields_": []}]})

print(out)

Python, Pydantic & OS Version

pydantic version: 1.10.9
pydantic compiled: True
install path: /{package location}/.venv/lib64/python3.10/site-packages/pydantic
python version: 3.10.10 (main, Feb  8 2023, 00:00:00) [GCC 12.2.1 20221121 (Red Hat 12.2.1-4)]
platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
optional deps. installed: ['typing-extensions']

Affected Components

• Compatibility between releases
• Data validation/parsing
• Data serialization - .dict() and .json()
• JSON Schema
• Dataclasses
• Model Config
• Field Types - adding or changing a particular data type
• Function validation decorator
• Generic Models
• Other Model behaviour - construct(), pickling, private attributes, ORM mode
• Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

[mark-todd]

I've found a couple more illustrations - one is a simplified version of the original minimal, and second illustrates the same bug with Dict -> dict

List version

from typing import Generic, List, TypeVar

from pydantic import BaseModel
from pydantic.generics import GenericModel

ChangingType = TypeVar("ChangingType")

class Outer(GenericModel, Generic[ChangingType]):
    test: list[ChangingType]

OuterNew = Outer["Inner"]
class Inner(BaseModel):
    new: int

OuterNew.update_forward_refs()
out = OuterNew.parse_obj({"test": [{"new": 1}]})

print(out)

Dict version

from typing import Dict, Generic, List, TypeVar

from pydantic import BaseModel
from pydantic.generics import GenericModel

ChangingType = TypeVar("ChangingType")

class Outer(GenericModel, Generic[ChangingType]):
    test: dict[str, ChangingType]

OuterNew = Outer["Inner"]
class Inner(BaseModel):
    new: int

OuterNew.update_forward_refs()
out = OuterNew.parse_obj({"test": {"t": {"new": 1}}})

print(out)

[mark-todd]

https://github.com/pydantic/pydantic/blob/1.10.X-fixes/pydantic/typing.py#L218

Think this is relevent - just testing

Update: Tested this function - seems to be working ok

[mark-todd]

Ok think I might have it: https://github.com/pydantic/pydantic/blob/1.10.X-fixes/pydantic/generics.py#L293

In this line the origins are correctly found as List or list, but the difference is that...

List['test'] -> List[ForwardRef('test')]
While
list['test'] -> list['test'], when it should be list[ForwardRef('test')] - which as it turns out are not equivalent. There needs to be something in this replace_args function that converts strings to ForwardRefs

[mark-todd]

This fork fixes the bug - it unfortunately also breaks one of the tests for discriminated union - not sure how to get round this yet:
https://github.com/mark-todd/pydantic/tree/generics-forward-ref-fix

[lig]

This works in Pydantic v2.3.0.

from typing import Generic, List, TypeVar

from pydantic import BaseModel

ChangingType = TypeVar("ChangingType")

class Outer(BaseModel, Generic[ChangingType]):
    fields_: list[ChangingType] # Swapping list -> List on this line fixes the bug

outer = Outer["Inner"]
class Inner(BaseModel):
    new: list[outer]

out = Inner.model_validate({"new": [{"fields_": []}]})

print(out)
# new=[Outer[Inner](fields_=[])]

There is even no need to use BaseModel.model_rebuild() method (former update_forward_refs).