twisted.web.template._flattenElement spends a significant amount of runtime in typing.py

[dreid]

Describe the incorrect behavior you saw

_flattenElement defines a closure keepGoing which is used to recursively call _flattenElement. keepGoing's type definition includes multiple Callable[…]s.

Since the file does not include from __future__ import annotations the type definitions are evaluated at function definition time, which is on every call to _flattenElement.

typing.Callable.__getitem__ does multiple isinstance checks before deferring to the the @_tp_cacheed implementation of Callable.__getitem_inner__. This causes evaluating the types for the closure to make up a significant portion of _flattenElement's runtime.

Describe how to cause this behavior

This behavior was discovered while profiling the a documentation build using pydoctor under the austin profiler.

Speedscope URL for the profile is here: https://www.speedscope.app/#profileURL=https%3A%2F%2Fgist.githubusercontent.com%2Fdreid%2F197566471f39a96523f5065d19d0bf7f%2Fraw%2F3e7ec92a17bc82d40acceb1e2efcaa3ef7c8ef07%2Ftwisted-austin-trunk.speedscope

In the profile you can see that the inner function in _tp_cache accounts for 1m25s of the total runtime, and the Callable.__getitem__ accounts for 25s of total runtime.

Describe the correct behavior you'd like to see
A clear and concise description of what you expected to happen, or what you believe should be happening instead.

Testing environment

• Operating System and Version; paste the output of these commands:
  • on Linux, uname -a ; cat /etc/lsb-release
  • on Windows, systeminfo | Findstr /i "OS"
  • on macOS, sw_vers
• Twisted version [e.g. 22.2.0]
  • please paste the output of twist --version and pip --freeze
• Reactor [e.g. select, iocp]

Additional context
Add any other context about the problem here.