Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Improve traceback report in parallel build. #11451

Closed
picnixz opened this issue Jun 4, 2023 · 0 comments · Fixed by #11452
Closed

[FEATURE] Improve traceback report in parallel build. #11451

picnixz opened this issue Jun 4, 2023 · 0 comments · Fixed by #11452

Comments

@picnixz
Copy link
Member

picnixz commented Jun 4, 2023

After having looked at #11449, I observed that the traceback for exceptions raised during a parellel build (excluding warnings as errors) in general not useful. Consider the following setup:

$ cat <<-EOF > index.rst 
Index
-----
.. fail::
EOF
$ cat <<-EOF > conf.py
from sphinx.directives import SphinxDirective

class FailDirective(SphinxDirective):
    def run(self):
        raise ValueError()

def setup(app):
    app.add_directive('fail', FailDirective)
EOF
$ python3 -m sphinx -M html . '_build' -ET -j4

Then the output is something like:

Running Sphinx v7.1.0+/d3c91f951
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 7 source files that are out of date
updating environment: [new config] 7 added, 0 changed, 0 removed

Traceback (most recent call last):
  File "/python/sphinx/sphinx/cmd/build.py", line 285, in build_main
    app.build(args.force_all, args.filenames)
  File "/python/sphinx/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/python/sphinx/sphinx/builders/__init__.py", line 294, in build_update
    self.build(to_build,
  File "/python/sphinx/sphinx/builders/__init__.py", line 311, in build
    updated_docnames = set(self.read())
  File "/python/sphinx/sphinx/builders/__init__.py", line 416, in read
    self._read_parallel(docnames, nproc=self.app.parallel)
  File "/python/sphinx/sphinx/builders/__init__.py", line 472, in _read_parallel
    tasks.join()
  File "/python/sphinx/sphinx/util/parallel.py", line 99, in join
    if not self._join_one():
  File "/python/sphinx/sphinx/util/parallel.py", line 119, in _join_one
    raise SphinxParallelError(*result)
sphinx.errors.SphinxParallelError: ValueError

Sphinx parallel build error:
ValueError

Process finished with exit code 2

So we know that an exception occurred but we have no idea where it was. The issue lies in:

if args.verbosity or args.traceback:
traceback.print_exc(None, stderr)
print(file=stderr)

When calling traceback.print_exc(), the latest exception is a SphinxParallelError object with the error message and the saved traceback. However, when printing, the traceback being used is the traceback of the SphinxParallelError object as an exception and not the SphinxParallelError.traceback value (which is actually a string). Unfortunately, traceback objects are not pickable so, this issue addresses this concern.

Environment

Platform:              linux; (Linux-5.3.18-lp152.106-default-x86_64-with-glibc2.26)
Python version:        3.10.3 (main, Jan 31 2023, 10:47:25) [GCC 7.5.0])
Python implementation: CPython
Sphinx version:        7.0.1
Docutils version:      0.18.1
Jinja2 version:        3.1.2
Pygments version:      2.15.1
picnixz added a commit to picnixz/sphinx that referenced this issue Jun 4, 2023
picnixz added a commit to picnixz/sphinx that referenced this issue Jun 4, 2023
picnixz added a commit to picnixz/sphinx that referenced this issue Jun 4, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant