gh-112319: Allow special protocol members #112340

randolf-scholz · 2023-11-23T15:39:49Z

Issue: @runtime_checkable Protocol doesn't check if __class_getitem__ exists. #112319

randolf-scholz · 2023-11-23T15:58:10Z

Lib/typing.py

+            cls.__protocol_attrs__ = (
+                _get_local_protocol_members(namespace) | _get_parent_members(cls)
+            )
+            # local_attrs =  _get_local_protocol_members(namespace)
+            # cls.__protocol_attrs__ = local_attrs.union(*map(_get_cls_members, bases))


Not sure which variant is better...

The only difference between _get_cls_members and _get_parent_members is that the former checks cls.__mro__[:-1] and the latter only cls.__mro__[1:-1].

I would expect the _get_parent_members to be better for deeply inherited classes, and the _get_cls_members to be better for classes with many disjoint parents.

Lib/typing.py

AlexWaygood

haven't looked deeply yet, just a few comments from an initial skim:

AlexWaygood · 2023-11-23T16:21:13Z

Lib/typing.py

-    '__init__', '__module__', '__new__', '__slots__',
-    '__subclasshook__', '__weakref__', '__class_getitem__',
-    '__match_args__',
+    '__module__', '__slots__', '__match_args__', '__qualname__',
+})
+_SPECIAL_CALLABLE_NAMES = frozenset({
+    '__init__', '__new__', '__subclasshook__','__class_getitem__', '__weakref__',
 })

 # These special attributes will be not collected as protocol members.
 EXCLUDED_ATTRIBUTES = _TYPING_INTERNALS | _SPECIAL_NAMES | {'_MutableMapping__marker'}
+EXCLUDED_MEMBERS = EXCLUDED_ATTRIBUTES | _SPECIAL_CALLABLE_NAMES


Please keep the diff as minimal as possible. As far as I can tell, splitting the _SPECIAL_NAMES frozenset into a _SPECIAL_NAMES set and a _SPECIAL_CALLABLE_NAMES set shouldn't make any difference here

For my solution to work, I need both, because for things defined directly inside the protocol body, I only exclude EXCLUDED_ATTRIBUTES, but not _SPECIAL_CALLABLE_NAMES. (Hence, we can test for __class_getitem__). However, for general classes we need to exclude it (otherwise test.test_typing.ProtocolTests.test_collections_protocols_allowed fails).

See the difference in function body between _get_local_members and _get_local_protocol_members.

Lib/typing.py

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

Lib/test/test_typing.py

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

randolf-scholz · 2023-11-23T17:21:30Z

Lib/typing.py

+            attrs |= getattr(base, "__protocol_attrs__", set())
+        else:
+            attrs |= _get_local_members(base.__dict__)
+    return attrs


 def _get_protocol_attrs(cls):


After the modifications here, _get_protocol_attrs is not used anymore, and it will even give some wrong results (e.g. if the Protocol defines __class_getitem__). So maybe just delete it?

If it's not used anywhere anymore, then yes, it should be deleted. It's an undocumented, private function; if anybody's using it and their code is broken by us deleting it, that's on them. I can't find any uses of it on grep.app anyway, so I don't think anybody will have their code broken by us deleting it.

randolf-scholz added 2 commits November 23, 2023 16:29

wip reworked setting __protocol_attrs__

36b59cb

Merge branch 'main' into rework_protocol_attrs

9147fd1

bedevere-app bot mentioned this pull request Nov 23, 2023

@runtime_checkable Protocol doesn't check if __class_getitem__ exists. #112319

Open