autodoc preserve defaults leads to exception on multiline lambda

[jaraco]

Describe the bug

In cssutils, I've stumbled into an issue where the docs builds are failing (jaraco/cssutils#36).

After some investigation, I learned that the issue seems to be related to the use of autodoc with autodoc_preserve_defaults = True and the use of property(lambda) where the lambda is on a different line from the property.

How to Reproduce

 draft $ cat mod.py
class X:
  foo = property(
    lambda self: None, doc="Foo.")
 draft $ cat conf.py
extensions = [
    'sphinx.ext.autodoc',
]

master_doc = "index"

# Preserve authored syntax for defaults
autodoc_preserve_defaults = True
 draft $ cat index.rst
.. automodule:: mod
    :members:
    :undoc-members:
 draft $ pip-run sphinx -- -m sphinx . build
Running Sphinx v7.0.1
making output directory... done
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 1 source files that are out of date
updating environment: [new config] 1 added, 0 changed, 0 removed
reading sources... [100%] index                                                                                                                
WARNING: error while formatting arguments for mod.X.foo: Handler <function update_defvalue at 0x102c2b100> for event 'autodoc-before-process-signature' threw an exception (exception: unmatched ')' (<unknown>, line 2))
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                                 
generating indices... genindex py-modindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 1 warning.

The HTML pages are in build.

Environment Information

draft $ pip-run sphinx -- -m sphinx --bug-report
Please paste all output below into the bug report template



Platform:              darwin; (macOS-13.4-arm64-arm-64bit)
Python version:        3.11.3 (main, Apr  7 2023, 20:13:31) [Clang 14.0.0 (clang-1400.0.29.202)])
Python implementation: CPython
Sphinx version:        7.0.1
Docutils version:      0.20.1
Jinja2 version:        3.1.2
Pygments version:      2.15.1

Sphinx extensions

sphinx.ext.autodoc

Additional context

Weirdly, removing the carriage return after property( suppresses the error. Also, converting to a traditional @property decorator or replacing the lambda with a simple function also suppresses the error:

class X:
  def _f(self):
    return
  foo = property(
    _f, doc="Foo.")

[picnixz]

TL;DR autodoc_preserve_defaults does not work with lambdas.

Even the following simple example

X = object()
Z = lambda x=X: x
"""The Z function."""

miserably fails. The reason is because the object being documented is a lambda function. Because of that, the extracted AST node is not a function definition node:

sphinx/sphinx/ext/autodoc/preserve_defaults.py

Lines 35 to 41 in d3c91f9

	try:
	source = inspect.getsource(obj)
	if source.startswith((' ', r'\t')):
	# subject is placed inside class or block. To read its docstring,
	# this adds if-block before the declaration.
	module = ast.parse('if True:\n' + source)
	return module.body[0].body[0] # type: ignore

Your example also highlights another issue. The corresponding source is

    lambda self: None, doc="Foo.")

You'll wrap it in a fake block and would ask to parse:

if True:
    lambda self: None, doc="Foo.")

and obviously it fails. The reason why it works when no lambda is used is because the source code of your function is correctly found. For the simple example above, the corresponding source code is

Z = lambda x=X: x

which is an assignment and not a function definition. Note that module.body[0].body[0] is still incorrect because it is an expression node whose value is the lambda function (module.body[0].body[0].value). So we need to change the implementation of get_function_def.

---

I can work on the issue today but won't be able to guarantee that it can be easily solved. I think I'll just consider the case of a lambda function differently and handle the case when the source code of the lambda function is wrapped or decorated and whether there are trailing braces to close.

[jaraco]

Thanks for the analysis.

I understand this report is fairly obscure, so I'll be surprised and impressed if you're able to solve it elegantly or quickly.

From the sounds of things, my best bet would for now be to accept that this form isn't supported and work to port the usage in the library to use naturally-decorated property methods, which I'll do.

[picnixz]

I understand this report is fairly obscure, so I'll be surprised and impressed if you're able to solve it elegantly or quickly.

Well, it should not be that hard actually. It's just that I don't necessarily have all the corner cases in mind (nor the time to look carefully). I'll only be able to look at it in mid July

[picnixz]

The PR that was merged should fix the issue (but Sphinx 7.2 is currently breaking other things so I'll advise waiting for 7.2.1).