Add introspection helper functions for third-party libraries

[AlexWaygood]

This is a proof-of-concept for the idea that I mentioned in #185 (comment). The proposal is that we could provide two simple introspection helper functions for third-party libraries, and could recommend that they use these utilities as building blocks for their own introspection functions.

For example, pydantic has a function called is_literal_type, and typing_inspect does also. Both functions broke with the release of typing_extensions 4.6.0:

• Fix Literal bug with typing-extension==4.6.0 pydantic/pydantic#5826
• Fix new version of typing_extensions ilevkivskyi/typing_inspect#101

With these introspection helpers, both libraries could rewrite their is_literal_type functions as:

from typing_extensions import get_origin, is_typing_name

def is_literal_type(tp) -> bool:
    return is_typing_name(tp, "Literal") or is_typing_name(get_origin(tp), "Literal")

And you could use the utility functions in this PR as building blocks for all kinds of similar introspection capabilities, if you're a library that's doing that kind of thing. We could advertise and recommend these functions in the docs as the preferred way to do this kind of thing while avoiding breakage.

I'll add docs to this PR if there's interest in this idea -- didn't want to invest too much time into it if it's not something we want to pursue :)

[AlexWaygood]

We can also bikeshed over the names, etc. As I say, this is a proof-of-concept.

Cc. @samuelcolvin / @adriangb / @ilevkivskyi -- curious if you'd find these useful? If you wouldn't, there's obviously no point :)

[JelleZijlstra]

For what it's worth, in pyanalyze (https://github.com/quora/pyanalyze/blob/master/pyanalyze/safe.py#L148) I found the following two functions useful:

• is_typing_name(obj, name) equivalent to obj is getattr(typing_extensions, name) or obj is getattr(typing, name) (filtering for whether those two exist). This I use frequently, often in conjunction with get_origin().
• is_instance_of_typing_name(obj, name) equivalent to isinstance(obj, getattr(typing_extensions, name)) or isinstance(obj, getattr(typing, name)). This one I use less frequently, and a few existing uses are questionable (e.g. they use private names).

I don't know when I would use your typing_extensions_reexports_name. I don't generally care whether typing_extensions has a thing, just whether the thing I have is some special object from typing(_extensions).

get_typing_objects_by_name_of could be used to implement my two helpers, but it's important to use is instead of ==, which the API you propose doesn't make easy.

[AlexWaygood]

I don't know when I would use your typing_extensions_reexports_name. I don't generally care whether typing_extensions has a thing, just whether the thing I have is some special object from typing(_extensions).

Yeah, you're right -- that one was useless. I've removed it, and added an implementation of is_typing_name.

[JelleZijlstra]

I think the idea (to help runtime users interact with typing-extensions better) is good.

However, as a consumer of typing-extensions (in pyanalyze) I don't think I would use these functions. I need two things that this API doesn't give me:

• Look at mypy_extensions in addition to typing_extensions (well, I could possibly drop this, but backwards compatibility). However, I don't think typing_extensions should be aware of mypy_extensions.
• These functions turn out to be very hot in pyanalyze, so I want to cache them so I don't have to do all the getattr() calls every time. This is possibly something typing_extensions could do too, but it might be better for individual applications to manage their own caches.

I wouldn't want to merge this PR unless I hear from some other maintainers of runtime typing libraries who say they'll definitely use these functions.

An alternative could be to put a recipe in the docs that suggests copying code like what you put in this PR and modifying it for your library's needs.

[AlexWaygood]

• These functions turn out to be very hot in pyanalyze, so I want to cache them so I don't have to do all the getattr() calls every time. This is possibly something typing_extensions could do too, but it might be better for individual applications to manage their own caches.

FWIW my PR already has a cache on one of the two functions, and the other function is just a thin wrapper over the function that has the cache.

I wouldn't want to merge this PR unless I hear from some other maintainers of runtime typing libraries who say they'll definitely use these functions.

Fully agreed, and I agree we've seen no evidence of that yet. @ilevkivskyi / @adriangb / @agronholm, would any of you folks actually use these functions if we added these to typing_extensions? If not, we should just close this, and pursue better documentation as Jelle suggests

[JelleZijlstra]

Good point about the cache. Pyanalyze uses a somewhat different caching scheme but likely your lru_cache is enough. I do have another idiosyncratic need: my functions need to support strings (like is_typing_name("typing.Protocol", "Protocol") in order to interact with pyanalyze's support for stub files. That's another behavior that wouldn't make much sense in typing_extensions.

[AlexWaygood]

Closing due to lack of interest among users

[adriangb]

Hey sorry I never gave any feedback here. We're getting into beta releases for Pydantic v2 so it's been pedal to the metal the last couple of weeks.

An alternative could be to put a recipe in the docs that suggests copying code like what you put in this PR and modifying it for your library's needs.

I agree that this would be a better first step than this PR. As part of the docs you can always change or even delete these examples without worrying about backward compatibility, and something like this is small enough that copy-pasting it into a codebase wouldn't be a huge concern. There's a good chance that if I put these into Pydantic's codebase I'd want to tweak something, I'm just not sure what yet.

[ilevkivskyi]

Sorry for late reply, here is some input from my side: Although in general I like the idea to of having some introspection helpers in typing_extensions there are two issues that would prevent them from being used in typing_inspect:

• The mypy_extensions thing mentioned above
• typing_inspect supports very old versions of Python (currently 3.5 is supported, and even Python 2, although it is not tested in CI anymore)

[AlexWaygood]

Thanks @ilevkivskyi, that makes sense! We've gone for adding recipes to the docs instead — see #225.