Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-103921: Document PEP 695 #104642

Merged
merged 69 commits into from May 26, 2023
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
786a436
First docs work
JelleZijlstra May 13, 2023
c1f266e
More words
JelleZijlstra May 15, 2023
040c4fd
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 19, 2023
e3dfb27
ast changes, fix build
JelleZijlstra May 19, 2023
f5b1d86
the big PEP list
JelleZijlstra May 19, 2023
5798631
dis
JelleZijlstra May 19, 2023
0f76679
default roles
JelleZijlstra May 19, 2023
cb76021
TODOs in compound_stmts
JelleZijlstra May 19, 2023
12c1249
minor
JelleZijlstra May 19, 2023
94a4da2
Remove broken link
JelleZijlstra May 19, 2023
815a021
update dis.dis docs
JelleZijlstra May 19, 2023
17ac785
TypeAlias is deprecated
JelleZijlstra May 19, 2023
1311399
Type aliases should now use the type statement
JelleZijlstra May 19, 2023
123752c
syntax
JelleZijlstra May 19, 2023
9d6e0c0
Merge branch 'main' into pep695docs
JelleZijlstra May 19, 2023
e955a45
Expand typing.rst
JelleZijlstra May 20, 2023
d7f2b55
Finish the type statement
JelleZijlstra May 20, 2023
53c4916
What's New
JelleZijlstra May 20, 2023
edec8c3
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 20, 2023
bc1c1d2
typo
JelleZijlstra May 20, 2023
b1a843c
FQNs in ast.rst
JelleZijlstra May 20, 2023
e690aff
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 21, 2023
cc5b51c
Document TypeVar constructor signature, discuss variance
JelleZijlstra May 21, 2023
b0e1ce3
TypeVarTuple signature
JelleZijlstra May 21, 2023
382ff2c
Fix sig
JelleZijlstra May 21, 2023
db94e4f
Document TypeVar attributes, fix indentation
JelleZijlstra May 21, 2023
18b9d63
Fix TypeVarTuple indentation
JelleZijlstra May 21, 2023
b7ba811
Merge branch 'main' into pep695docs
AlexWaygood May 22, 2023
fd1d0a8
s/typeparams/type_params/
JelleZijlstra May 22, 2023
62ba887
Apply suggestions from code review
JelleZijlstra May 22, 2023
200eba1
Reword long dis sentence
JelleZijlstra May 22, 2023
adaf874
Manual changes from Alex review
JelleZijlstra May 22, 2023
4481d6d
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 22, 2023
365ae7b
Improve ast.TypeAlias text
JelleZijlstra May 22, 2023
96e5d16
Document __type_params__
JelleZijlstra May 22, 2023
0204d54
Add to datamodel.rst
JelleZijlstra May 22, 2023
f426d39
Wording change
JelleZijlstra May 22, 2023
60d7783
Update Doc/library/typing.rst
JelleZijlstra May 22, 2023
63d4d84
Apply suggestions from code review
JelleZijlstra May 22, 2023
fd6751a
fix indentation
JelleZijlstra May 22, 2023
499dae1
Link to generic functions
JelleZijlstra May 22, 2023
b4172c9
Move up variance paragraph
JelleZijlstra May 22, 2023
e9bc4a8
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 23, 2023
0d37675
Some Alex comments
JelleZijlstra May 23, 2023
c94ee7c
Fix alignment
JelleZijlstra May 23, 2023
1630342
doctest maybe
JelleZijlstra May 23, 2023
aade45a
More Alex suggestions
JelleZijlstra May 23, 2023
57b9992
Feedback from Carl
JelleZijlstra May 23, 2023
7953d2c
Section on lazy eval
JelleZijlstra May 23, 2023
9128c72
More cross-refs
JelleZijlstra May 23, 2023
0511b6a
Can't have nice things
JelleZijlstra May 23, 2023
4fd3f02
Respond to some feedback
JelleZijlstra May 24, 2023
fff0df4
Undo unnecessary typing.rst changes
JelleZijlstra May 24, 2023
ba2a103
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 24, 2023
1c6807d
more words
JelleZijlstra May 24, 2023
6386475
or
JelleZijlstra May 24, 2023
cd4206c
Update Doc/reference/compound_stmts.rst
JelleZijlstra May 24, 2023
e307177
Merge branch 'main' into pep695docs
AlexWaygood May 25, 2023
d7e1c6c
Apply suggestions from code review
JelleZijlstra May 26, 2023
f878970
More feedback from Alex
JelleZijlstra May 26, 2023
a4bd32a
fixes
JelleZijlstra May 26, 2023
ee8f952
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 26, 2023
7a709d7
Rewrite and extend some docstrings
JelleZijlstra May 26, 2023
0cfeb92
More comments
JelleZijlstra May 26, 2023
f978766
Docstring fixes
JelleZijlstra May 26, 2023
bba092d
tables
JelleZijlstra May 26, 2023
57ddbef
more fixes
JelleZijlstra May 26, 2023
67acd98
Merge remote-tracking branch 'upstream/main' into pep695docs
JelleZijlstra May 26, 2023
988fc7c
Merge branch 'main' into pep695docs
AlexWaygood May 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
102 changes: 100 additions & 2 deletions Doc/library/ast.rst
Expand Up @@ -43,12 +43,12 @@

.. class:: AST

This is the base of all AST node classes. The actual node classes are

Check warning on line 46 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:mod reference target not found: _ast
derived from the :file:`Parser/Python.asdl` file, which is reproduced
:ref:`above <abstract-grammar>`. They are defined in the :mod:`_ast` C
module and re-exported in :mod:`ast`.

There is one class defined for each left-hand side symbol in the abstract

Check warning on line 51 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.stmt

Check warning on line 51 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.expr

Check warning on line 51 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.expr
grammar (for example, :class:`ast.stmt` or :class:`ast.expr`). In addition,
there is one class defined for each constructor on the right-hand side; these
classes inherit from the classes for the left-hand side trees. For example,
Expand All @@ -64,7 +64,7 @@
Each concrete class has an attribute :attr:`_fields` which gives the names
of all child nodes.

Each instance of a concrete class has one attribute for each child node,

Check warning on line 67 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:attr reference target not found: left

Check warning on line 67 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.expr
of the type as defined in the grammar. For example, :class:`ast.BinOp`
instances have an attribute :attr:`left` of type :class:`ast.expr`.

Expand All @@ -79,7 +79,7 @@
end_lineno
end_col_offset

Instances of :class:`ast.expr` and :class:`ast.stmt` subclasses have

Check warning on line 82 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.expr

Check warning on line 82 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.stmt
:attr:`lineno`, :attr:`col_offset`, :attr:`end_lineno`, and
:attr:`end_col_offset` attributes. The :attr:`lineno` and :attr:`end_lineno`
are the first and last line numbers of source text span (1-indexed so the
Expand All @@ -93,9 +93,9 @@
one can get the source segment of a one-line expression node using
``source_line[node.col_offset : node.end_col_offset]``.

The constructor of a class :class:`ast.T` parses its arguments as follows:

Check warning on line 96 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:class reference target not found: ast.T

* If there are positional arguments, there must be as many as there are items

Check warning on line 98 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs

py:attr reference target not found: T._fields
in :attr:`T._fields`; they will be assigned as attributes of these names.
* If there are keyword arguments, they will set the attributes of the same
names to the given values.
Expand Down Expand Up @@ -917,6 +917,25 @@
type_ignores=[])


.. class:: TypeAlias(name, type_params, value)

A :ref:`type alias <type-aliases>` created through the :keyword:`type`
statement. ``name`` is the name of the alias, ``type_params`` is a list of
:ref:`type parameters <ast-type-params>`, and ``value`` is the value of the
type alias.

.. doctest::

>>> print(ast.dump(ast.parse('type Alias = int'), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[],
value=Name(id='int', ctx=Load()))],
type_ignores=[])


Other statements which are only applicable inside functions or loops are
described in other sections.

Expand Down Expand Up @@ -1644,15 +1663,93 @@
value=Constant(value=Ellipsis))])])],
type_ignores=[])

.. _ast-type-params:

Type parameters
^^^^^^^^^^^^^^^

:ref:`Type parameters <type-params>` can exist on classes, functions, and type
aliases.

.. class:: TypeVar(name, bound)

A :class:`typing.TypeVar`. ``name`` is the name of the type variable.
``bound`` is the bound or constraints, if any. If ``bound`` is a :class:`Tuple`,
it represents constraints; otherwise it represents the bound.

.. doctest::

>>> print(ast.dump(ast.parse("type Alias[T: int] = list[T]"), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[
TypeVar(
name='T',
bound=Name(id='int', ctx=Load()))],
value=Subscript(
value=Name(id='list', ctx=Load()),
slice=Name(id='T', ctx=Load()),
ctx=Load()))],
type_ignores=[])

.. class:: ParamSpec(name)

A :class:`typing.ParamSpec`. ``name`` is the name of the parameter specification.

.. doctest::

>>> print(ast.dump(ast.parse("type Alias[**P] = Callable[P, int]"), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[
ParamSpec(name='P')],
value=Subscript(
value=Name(id='Callable', ctx=Load()),
slice=Tuple(
elts=[
Name(id='P', ctx=Load()),
Name(id='int', ctx=Load())],
ctx=Load()),
ctx=Load()))],
type_ignores=[])

.. class:: TypeVarTuple(name)

A :class:`typing.TypeVarTuple`. ``name`` is the name of the type variable tuple.

.. doctest::

>>> print(ast.dump(ast.parse("type Alias[*Ts] = tuple[*Ts]"), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[
TypeVarTuple(name='Ts')],
value=Subscript(
value=Name(id='tuple', ctx=Load()),
slice=Tuple(
elts=[
Starred(
value=Name(id='Ts', ctx=Load()),
ctx=Load())],
ctx=Load()),
ctx=Load()))],
type_ignores=[])

Function and class definitions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. class:: FunctionDef(name, args, body, decorator_list, returns, type_comment)
.. class:: FunctionDef(name, type_params, args, body, decorator_list, returns, type_comment)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realised that type_params is in the wrong place here for the AST as it currently exists on main, where type_params is at the end. But probably not worth fixing that until questions about the ABI are resolved in #104974.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, looks like you might have added type_params to the constructor for FunctionDef and ClassDef, but not AsyncFunctionDef!


A function definition.

* ``name`` is a raw string of the function name.
* ``type_params`` is a list of :ref:`type parameters <ast-type-params>`.
* ``args`` is an :class:`arguments` node.
* ``body`` is the list of nodes inside the function.
* ``decorator_list`` is the list of decorators to be applied, stored outermost
Expand Down Expand Up @@ -1820,11 +1917,12 @@
type_ignores=[])


.. class:: ClassDef(name, bases, keywords, starargs, kwargs, body, decorator_list)
.. class:: ClassDef(name, type_params, bases, keywords, starargs, kwargs, body, decorator_list)

A class definition.

* ``name`` is a raw string for the class name
* ``type_params`` is a list of :ref:`type parameters <ast-type-params>`.
* ``bases`` is a list of nodes for explicitly specified base classes.
* ``keywords`` is a list of :class:`keyword` nodes, principally for 'metaclass'.
Other keywords will be passed to the metaclass, as per `PEP-3115
Expand Down
111 changes: 91 additions & 20 deletions Doc/library/dis.rst
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may want to get somebody else (Brandt? Irit?) to look at the changes to this file; I'm far from an expert on dis

Expand Up @@ -188,9 +188,9 @@ operation is being performed, so the intermediate analysis object isn't useful:
For a module, it disassembles all functions. For a class, it disassembles
all methods (including class and static methods). For a code object or
sequence of raw bytecode, it prints one line per bytecode instruction.
It also recursively disassembles nested code objects (the code of
comprehensions, generator expressions and nested functions, and the code
used for building nested classes).
It also recursively disassembles nested code objects. These can include
generator expressions, nested functions, the bodies of nested classes,
and the code objects used for :ref:`annotation scopes <annotation-scopes>`.
Strings are first compiled to code objects with the :func:`compile`
built-in function before being disassembled. If no object is provided, this
function disassembles the last traceback.
Expand Down Expand Up @@ -926,6 +926,27 @@ iterations of the loop.
.. opcode:: LOAD_NAME (namei)

Pushes the value associated with ``co_names[namei]`` onto the stack.
The name is looked up within the locals, then the globals, then the builtins.


.. opcode:: LOAD_LOCALS

Pushes a reference to the locals dictionary onto the stack. This is used
to prepare namespace dictionaries for :opcode:`LOAD_FROM_DICT_OR_DEREF`
and :opcode:`LOAD_FROM_DICT_OR_GLOBALS`.

.. versionadded:: 3.12


.. opcode:: LOAD_FROM_DICT_OR_GLOBALS (i)

Pops a mapping off the stack and looks up the value for ``co_names[namei]``.
If the name is not found there, looks it up in the globals and then the builtins,
similar to :opcode:`LOAD_GLOBAL`.
This is used for loading global variables in
:ref:`annotation scopes <annotation-scopes>` within class bodies.

.. versionadded:: 3.12


.. opcode:: BUILD_TUPLE (count)
Expand Down Expand Up @@ -1243,16 +1264,17 @@ iterations of the loop.
``i`` is no longer offset by the length of ``co_varnames``.


.. opcode:: LOAD_CLASSDEREF (i)
.. opcode:: LOAD_FROM_DICT_OR_DEREF (i)

Much like :opcode:`LOAD_DEREF` but first checks the locals dictionary before
consulting the cell. This is used for loading free variables in class
bodies.

.. versionadded:: 3.4
Pops a mapping off the stack and looks up the name associated with
slot ``i`` of the "fast locals" storage in this mapping.
If the name is not found there, loads it from the cell contained in
slot ``i``, similar to :opcode:`LOAD_DEREF`. This is used for loading
free variables in class bodies (which previously used
:opcode:`!LOAD_CLASSDEREF`) and in
:ref:`annotation scopes <annotation-scopes>` within class bodies.

.. versionchanged:: 3.11
``i`` is no longer offset by the length of ``co_varnames``.
.. versionadded:: 3.12


.. opcode:: STORE_DEREF (i)
Expand Down Expand Up @@ -1504,13 +1526,45 @@ iterations of the loop.

The operand determines which intrinsic function is called:

* ``0`` Not valid
* ``1`` Prints the argument to standard out. Used in the REPL.
* ``2`` Performs ``import *`` for the named module.
* ``3`` Extracts the return value from a ``StopIteration`` exception.
* ``4`` Wraps an aync generator value
* ``5`` Performs the unary ``+`` operation
* ``6`` Converts a list to a tuple
+-----------------------------------+-----------------------------------+
| Operand | Description |
+===================================+===================================+
| ``INTRINSIC_1_INVALID`` | Not valid |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_PRINT`` | Prints the argument to standard |
| | out. Used in the REPL. |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_IMPORT_STAR`` | Performs ``import *`` for the |
| | named module. |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_STOPITERATION_ERROR`` | Extracts the return value from a |
| | ``StopIteration`` exception. |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_ASYNC_GEN_WRAP`` | Wraps an aync generator value |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_UNARY_POSITIVE`` | Performs the unary ``+`` |
| | operation |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_LIST_TO_TUPLE`` | Converts a list to a tuple |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_TYPEVAR`` | Creates a :class:`typing.TypeVar` |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_PARAMSPEC`` | Creates a |
| | :class:`typing.ParamSpec` |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_TYPEVARTUPLE`` | Creates a |
| | :class:`typing.TypeVarTuple` |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_SUBSCRIPT_GENERIC`` | Returns :class:`typing.Generic` |
| | subscripted with the argument |
+-----------------------------------+-----------------------------------+
| ``INTRINSIC_TYPEALIAS`` | Creates a |
| | :class:`typing.TypeAliasType`; |
| | used in the :keyword:`type` |
| | statement. The argument is a tuple|
| | of the type alias's name, |
| | type parameters, and value. |
+-----------------------------------+-----------------------------------+

.. versionadded:: 3.12

Expand All @@ -1522,8 +1576,25 @@ iterations of the loop.

The operand determines which intrinsic function is called:

* ``0`` Not valid
* ``1`` Calculates the :exc:`ExceptionGroup` to raise from a ``try-except*``.
+----------------------------------------+-----------------------------------+
| Operand | Description |
+========================================+===================================+
| ``INTRINSIC_2_INVALID`` | Not valid |
+----------------------------------------+-----------------------------------+
| ``INTRINSIC_PREP_RERAISE_STAR`` | Calculates the |
| | :exc:`ExceptionGroup` to raise |
| | from a ``try-except*``. |
+----------------------------------------+-----------------------------------+
| ``INTRINSIC_TYPEVAR_WITH_BOUND`` | Creates a :class:`typing.TypeVar` |
| | with a bound. |
+----------------------------------------+-----------------------------------+
| ``INTRINSIC_TYPEVAR_WITH_CONSTRAINTS`` | Creates a |
| | :class:`typing.TypeVar` with |
| | constraints. |
+----------------------------------------+-----------------------------------+
| ``INTRINSIC_SET_FUNCTION_TYPE_PARAMS`` | Sets the ``__type_params__`` |
| | attribute of a function. |
+----------------------------------------+-----------------------------------+

.. versionadded:: 3.12

Expand Down
8 changes: 8 additions & 0 deletions Doc/library/stdtypes.rst
Expand Up @@ -5476,6 +5476,14 @@ types, where they are relevant. Some of these are not reported by the
.. versionadded:: 3.3


.. attribute:: definition.__type_params__

The :ref:`type parameters <type-params>` of generic classes, functions,
and :ref:`type aliases <type-aliases>`.

.. versionadded:: 3.12


.. attribute:: class.__mro__

This attribute is a tuple of classes that are considered when looking for
Expand Down