Recursive type not working

[spacether]

Bug Report

I am using a recursive type to constrain int/str/float/None/bool/Tuple[recursive_type, ...],dict[str, recursive_type]
but mypy says that a return type that I am defining is incompatible even though it is a subset of the recursive type union.

To Reproduce

Why does mypy say that my return type is incompatible?
Here is my code:

from __future__ import annotations
import typing
import dataclasses

import immutabledict

class Unset:
    pass

unset: Unset = Unset()

OUTPUT_BASE_TYPES = typing.Union[
    immutabledict.immutabledict[str, 'OUTPUT_BASE_TYPES'],
    str,
    int,
    float,
    bool,
    None,
    typing.Tuple['OUTPUT_BASE_TYPES', ...],
]

@dataclasses.dataclass
class ApiResponse:
    body: typing.Union[OUTPUT_BASE_TYPES, Unset] = unset


class MyDict(immutabledict.immutabledict[str, typing.Union[None, str]]):
    pass

@dataclasses.dataclass
class CustomApiResponse(ApiResponse):
    body: MyDict


inst = ApiResponse(body=1)
other_inst = CustomApiResponse(body=MyDict({}))

But the definition of the values for immutabledict typing.Union[None, str] are a subset of the OUTPUT_BASE_TYPES union of types
so why is this failing?

How do i get it to work? What do I need to change?

Actual Behavior

mypy reports:

mypy union.py union.py:36: error: Incompatible types in assignment (expression has type "MyDict", base class "ApiResponse" defined the type as "Union[OUTPUT_BASE_TYPES, Unset]") [assignment]

Your Environment
pip install immutabledict

• Mypy version used: mypy 1.4.1 (compiled: yes)
• Mypy command-line flags: none, mypy union.py
• Mypy configuration options from mypy.ini N/A
• Python version used: Python 3.7.12 (default, Sep 20 2022, 17:18:51)
  [Clang 10.0.1 (clang-1001.0.46.4)] on darwin

[ikonst]

Update: I'm wrong. OP has an immutabledict that derives from typing.Mapping.

The problem is that they're not compatible:

d1 = MyDict()
d2: OUTPUT_BASE_TYPES = d1

assert isinstance(d2, immutabledict.immutabledict)
reveal_type(d2)  # N: revealed type is "immutabledict.immutabledict[str, 'OUTPUT_BASE_TYPES']"

# this should be illegal since MyDict values are "str | None"
d2['foobar'] = 42
# and mypy still doesn't known:
reveal_type(d1['foobar'])  # N: revealed type is "str | None"
reveal_type(d2['foobar'])  # N: revealed type is "OUTPUT_BASE_TYPES" = "str | int | ..."

# and of course
assert d1['foobar'] == 42

[spacether]

Thank you for your answer.
But str|None conforms to the constraints of OUTPUT_BASE_TYPES.

What are solutions here?

• create all the possible type combinations and use them as tuple values and immutabledict.immutabledict values in my OUTPUT_BASE_TYPES?
• are there simpler ways to do this that still provide the type constraints?

The simplest solution that I see is:

T = typing.TypeVar(
    'T',
    immutabledict.immutabledict,
    str,
    int,
    float,
    bool,
    None,
    typing.Tuple,
)


@dataclasses.dataclass
class ApiResponse(typing.Generic[T]):
    body: typing.Union[T, Unset] = unset

And that loses the recursive type definition.

[ikonst]

Actually I might be wrong here. For plain dict it's certainly correct: if you extend dict[K, V] as mydict[K2, V2], you cannot make V2 broader nor narrower because overrides must remain compatible (per Liskov Substitution Principle).

Assuming immutabledict is correctly typed (looks like it is) then it should be possible:
https://mypy-play.net/?mypy=latest&python=3.11&gist=13a5cd05ea54c0d1bb04f2724a0cb21f

Gotta dig deeper here.

[ilevkivskyi]

You know, just by calling something immutable doesn't make it automatically covariant, LOL. Here is the definition from that library:

_K = TypeVar("_K")
_V = TypeVar("_V")


class immutabledict(Mapping[_K, _V]):
    ...

(juts as a reminder type variables are invariant by default)

[spacether]

@ilevkivskyi thank you for the hint. Other immutable collections (Sequence/Tuple) are covariant per mypy docs.
Once I changed the _K TypeVar to covariant=True then my original code worked.

[ikonst]

An issue should be filled against immutabledict

[spacether]

Yup I did that here: corenting/immutabledict#243
And I filed a PR on it: corenting/immutabledict#244