Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for PEP 705 #284

Merged
merged 9 commits into from
Nov 29, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Release 4.9.0 (???)

- Add support for PEP 705, adding `typing_extensions.ReadOnly`. Patch
by Jelle Zijlstra.
- All parameters on `NewType.__call__` are now positional-only. This means that
the signature of `typing_extensions.NewType.__call__` now exactly matches the
signature of `typing.NewType.__call__`. Patch by Alex Waygood.
Expand Down
29 changes: 28 additions & 1 deletion doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,12 @@ Special typing primitives
present in a protocol class's :py:term:`method resolution order`. See
:issue:`245` for some examples.

.. data:: ReadOnly

See :pep:`705`. Indicates that a :class:`TypedDict` item may not be modified.

.. versionadded:: 4.9.0

.. data:: Required

See :py:data:`typing.Required` and :pep:`655`. In ``typing`` since 3.11.
Expand All @@ -344,7 +350,7 @@ Special typing primitives

See :py:data:`typing.TypeGuard` and :pep:`647`. In ``typing`` since 3.10.

.. class:: TypedDict
.. class:: TypedDict(dict, total=True)

See :py:class:`typing.TypedDict` and :pep:`589`. In ``typing`` since 3.8.

Expand All @@ -366,6 +372,23 @@ Special typing primitives
raises a :py:exc:`DeprecationWarning` when this syntax is used in Python 3.12
or lower and fails with a :py:exc:`TypeError` in Python 3.13 and higher.

``typing_extensions`` supports the experimental :data:`ReadOnly` qualifier
proposed by :pep:`705`. It is reflected in the following attributes::

.. attribute:: __readonly_keys__

A :py:class:`frozenset` containing the names of all read-only keys. Keys
are read-only if they carry the :data:`ReadOnly` qualifier.

.. versionadded:: 4.9.0

.. attribute:: __mutable_keys__

A :py:class:`frozenset` containing the names of all mutable keys. Keys
are mutable if they do not carry the :data:`ReadOnly` qualifier.

.. versionadded:: 4.9.0

.. versionchanged:: 4.3.0

Added support for generic ``TypedDict``\ s.
Expand Down Expand Up @@ -394,6 +417,10 @@ Special typing primitives
disallowed in Python 3.15. To create a TypedDict class with 0 fields,
use ``class TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``.

.. versionchanged:: 4.9.0

Support for the :data:`ReadOnly` qualifier was added.

.. class:: TypeVar(name, *constraints, bound=None, covariant=False,
contravariant=False, infer_variance=False, default=...)

Expand Down
57 changes: 54 additions & 3 deletions src/test_typing_extensions.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
import typing_extensions
from typing_extensions import NoReturn, Any, ClassVar, Final, IntVar, Literal, Type, NewType, TypedDict, Self
from typing_extensions import TypeAlias, ParamSpec, Concatenate, ParamSpecArgs, ParamSpecKwargs, TypeGuard
from typing_extensions import Awaitable, AsyncIterator, AsyncContextManager, Required, NotRequired
from typing_extensions import Awaitable, AsyncIterator, AsyncContextManager, Required, NotRequired, ReadOnly
from typing_extensions import Protocol, runtime, runtime_checkable, Annotated, final, is_typeddict
from typing_extensions import TypeVarTuple, Unpack, dataclass_transform, reveal_type, Never, assert_never, LiteralString
from typing_extensions import assert_type, get_type_hints, get_origin, get_args, get_original_bases
Expand Down Expand Up @@ -3520,7 +3520,7 @@ def test_typeddict_create_errors(self):

def test_typeddict_errors(self):
Emp = TypedDict('Emp', {'name': str, 'id': int})
if sys.version_info >= (3, 13):
if hasattr(typing, "ReadOnly"):
self.assertEqual(TypedDict.__module__, 'typing')
else:
self.assertEqual(TypedDict.__module__, 'typing_extensions')
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -4047,6 +4047,55 @@ class T4(TypedDict, Generic[S]): pass
self.assertEqual(klass.__optional_keys__, set())
self.assertIsInstance(klass(), dict)

def test_readonly_inheritance(self):
class Base1(TypedDict):
a: ReadOnly[int]

class Child1(Base1):
b: str

self.assertEqual(Child1.__readonly_keys__, frozenset({'a'}))
self.assertEqual(Child1.__mutable_keys__, frozenset({'b'}))

class Base2(TypedDict):
a: ReadOnly[int]

class Child2(Base2):
b: str

self.assertEqual(Child1.__readonly_keys__, frozenset({'a'}))
self.assertEqual(Child1.__mutable_keys__, frozenset({'b'}))

def test_cannot_make_mutable_key_readonly(self):
class Base(TypedDict):
a: int

with self.assertRaises(TypeError):
class Child(Base):
a: ReadOnly[int]

def test_can_make_readonly_key_mutable(self):
class Base(TypedDict):
a: ReadOnly[int]

class Child(Base):
a: int

self.assertEqual(Child.__readonly_keys__, frozenset())
self.assertEqual(Child.__mutable_keys__, frozenset({'a'}))

def test_combine_qualifiers(self):
class AllTheThings(TypedDict):
a: Annotated[Required[ReadOnly[int]], "why not"]
b: Required[Annotated[ReadOnly[int], "why not"]]
c: ReadOnly[NotRequired[Annotated[int, "why not"]]]
d: NotRequired[Annotated[int, "why not"]]

self.assertEqual(AllTheThings.__required_keys__, frozenset({'a', 'b'}))
self.assertEqual(AllTheThings.__optional_keys__, frozenset({'c', 'd'}))
self.assertEqual(AllTheThings.__readonly_keys__, frozenset({'a', 'b', 'c'}))
self.assertEqual(AllTheThings.__mutable_keys__, frozenset({'d'}))


class AnnotatedTests(BaseTestCase):

Expand Down Expand Up @@ -5187,7 +5236,9 @@ def test_typing_extensions_defers_when_possible(self):
'SupportsRound', 'Unpack',
}
if sys.version_info < (3, 13):
exclude |= {'NamedTuple', 'Protocol', 'TypedDict', 'is_typeddict'}
exclude |= {'NamedTuple', 'Protocol'}
if not hasattr(typing, 'ReadOnly'):
exclude |= {'TypedDict', 'is_typeddict'}
for item in typing_extensions.__all__:
if item not in exclude and hasattr(typing, item):
self.assertIs(
Expand Down
103 changes: 92 additions & 11 deletions src/typing_extensions.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@
'TYPE_CHECKING',
'Never',
'NoReturn',
'ReadOnly',
'Required',
'NotRequired',

Expand Down Expand Up @@ -768,7 +769,7 @@ def inner(func):
return inner


if sys.version_info >= (3, 13):
if hasattr(typing, "ReadOnly"):
# The standard library TypedDict in Python 3.8 does not store runtime information
# about which (if any) keys are optional. See https://bugs.python.org/issue38834
# The standard library TypedDict in Python 3.9.0/1 does not honour the "total"
Expand All @@ -779,15 +780,37 @@ def inner(func):
# Aaaand on 3.12 we add __orig_bases__ to TypedDict
# to enable better runtime introspection.
# On 3.13 we deprecate some odd ways of creating TypedDicts.
# PEP 705 proposes adding the ReadOnly[] qualifier.
TypedDict = typing.TypedDict
_TypedDictMeta = typing._TypedDictMeta
is_typeddict = typing.is_typeddict
else:
# 3.10.0 and later
_TAKES_MODULE = "module" in inspect.signature(typing._type_check).parameters

def _get_typeddict_qualifiers(annotation_type):
while True:
annotation_origin = get_origin(annotation_type)
if annotation_origin is Annotated:
annotation_args = get_args(annotation_type)
if annotation_args:
annotation_type = annotation_args[0]
else:
break
elif annotation_origin is Required:
yield Required
annotation_type, = get_args(annotation_type)
elif annotation_origin is NotRequired:
yield NotRequired
annotation_type, = get_args(annotation_type)
elif annotation_origin is ReadOnly:
yield ReadOnly
annotation_type, = get_args(annotation_type)
else:
break

class _TypedDictMeta(type):
def __new__(cls, name, bases, ns, total=True):
def __new__(cls, name, bases, ns, *, total=True):
"""Create new typed dict class object.

This method is called when TypedDict is subclassed,
Expand Down Expand Up @@ -830,33 +853,44 @@ def __new__(cls, name, bases, ns, total=True):
}
required_keys = set()
optional_keys = set()
readonly_keys = set()
mutable_keys = set()

for base in bases:
annotations.update(base.__dict__.get('__annotations__', {}))
required_keys.update(base.__dict__.get('__required_keys__', ()))
optional_keys.update(base.__dict__.get('__optional_keys__', ()))
readonly_keys.update(base.__dict__.get('__readonly_keys__', ()))
mutable_keys.update(base.__dict__.get('__mutable_keys__', ()))
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved

annotations.update(own_annotations)
for annotation_key, annotation_type in own_annotations.items():
annotation_origin = get_origin(annotation_type)
if annotation_origin is Annotated:
annotation_args = get_args(annotation_type)
if annotation_args:
annotation_type = annotation_args[0]
annotation_origin = get_origin(annotation_type)

if annotation_origin is Required:
qualifiers = set(_get_typeddict_qualifiers(annotation_type))

if Required in qualifiers:
required_keys.add(annotation_key)
elif annotation_origin is NotRequired:
elif NotRequired in qualifiers:
optional_keys.add(annotation_key)
elif total:
required_keys.add(annotation_key)
else:
optional_keys.add(annotation_key)
if ReadOnly in qualifiers:
if annotation_key in mutable_keys:
raise TypeError(
f"Cannot override mutable key {annotation_key!r}"
" with read-only key"
)
readonly_keys.add(annotation_key)
else:
mutable_keys.add(annotation_key)
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
readonly_keys.discard(annotation_key)

tp_dict.__annotations__ = annotations
tp_dict.__required_keys__ = frozenset(required_keys)
tp_dict.__optional_keys__ = frozenset(optional_keys)
tp_dict.__readonly_keys__ = frozenset(readonly_keys)
tp_dict.__mutable_keys__ = frozenset(mutable_keys)
if not hasattr(tp_dict, '__total__'):
tp_dict.__total__ = total
return tp_dict
Expand Down Expand Up @@ -937,6 +971,8 @@ class Point2D(TypedDict):
raise TypeError("TypedDict takes either a dict or keyword arguments,"
" but not both")
if kwargs:
if sys.version_info >= (3, 13):
raise TypeError("TypedDict takes no keyword arguments")
warnings.warn(
"The kwargs-based syntax for TypedDict definitions is deprecated "
"in Python 3.11, will be removed in Python 3.13, and may not be "
Expand Down Expand Up @@ -1925,6 +1961,51 @@ class Movie(TypedDict):
""")


if hasattr(typing, 'ReadOnly'):
ReadOnly = typing.ReadOnly
elif sys.version_info[:2] >= (3, 9): # 3.9-3.12
@_ExtensionsSpecialForm
def ReadOnly(self, parameters):
"""A special typing construct to mark a key of a TypedDict as read-only.
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
For example:
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved

class Movie(TypedDict):
title: ReadOnly[str]
year: int

def mutate_movie(m: Movie) -> None:
m["year"] = 1992 # allowed
m["title"] = "The Matrix" # typechecker error

There is no runtime checking for this propery.
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
"""
item = typing._type_check(parameters, f'{self._name} accepts only a single type.')
return typing._GenericAlias(self, (item,))

else: # 3.8
class _ReadOnlyForm(_ExtensionsSpecialForm, _root=True):
def __getitem__(self, parameters):
item = typing._type_check(parameters,
f'{self._name} accepts only a single type.')
return typing._GenericAlias(self, (item,))

ReadOnly = _ReadOnlyForm(
'ReadOnly',
doc="""A special typing construct to mark a key of a TypedDict as read-only.
For example:
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved

class Movie(TypedDict):
title: ReadOnly[str]
year: int

def mutate_movie(m: Movie) -> None:
m["year"] = 1992 # allowed
m["title"] = "The Matrix" # typechecker error

There is no runtime checking for this propery.
""")


_UNPACK_DOC = """\
Type unpack operator.

Expand Down