IPython directive can segfault in sphinx parallel build

[ngoldbaum]

See sphinx-doc/sphinx#11449 (comment) and replies for some context.

Today I happened to get into a state where a sphinx parallel build would reproducibly segfault trying to connect to the IPython history database, with the following traceback reported by faulthandler:

Fatal Python error: Segmentation faultnerated/numpy.lib.npyio.NpzFile .. reference/generated/numpy.linalg.vector_norm

Current thread 0x00000001dd5dd000 (most recent call first):
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/core/history.py", line 247 in init_db
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/core/history.py", line 77 in catch_corrupt_db
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/decorator.py", line 232 in fun
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/core/history.py", line 222 in __init__
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/core/history.py", line 542 in __init__
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 1884 in init_history
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 619 in __init__
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/traitlets/config/configurable.py", line 583 in instance
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/sphinxext/ipython_directive.py", line 364 in __init__
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/sphinxext/ipython_directive.py", line 964 in setup
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/IPython/sphinxext/ipython_directive.py", line 1006 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2154 in run_directive
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2104 in directive
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2367 in explicit_construct
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2355 in explicit_markup
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 445 in check_line
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 233 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 195 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 279 in nested_parse
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 391 in new_subsection
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 325 in section
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2785 in underline
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 445 in check_line
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 233 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 195 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 279 in nested_parse
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 391 in new_subsection
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 325 in section
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2785 in underline
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 445 in check_line
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 233 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 195 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 279 in nested_parse
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 391 in new_subsection
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 325 in section
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2785 in underline
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 445 in check_line
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/statemachine.py", line 233 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 169 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/parsers.py", line 81 in parse
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/readers/__init__.py", line 76 in parse
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/io.py", line 105 in read
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/docutils/core.py", line 234 in publish
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 498 in read_doc
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 459 in read_process
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/util/parallel.py", line 76 in _process
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/multiprocessing/process.py", line 108 in run
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/multiprocessing/process.py", line 314 in _bootstrap
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/multiprocessing/popen_fork.py", line 71 in _launch
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/multiprocessing/popen_fork.py", line 19 in __init__
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/multiprocessing/context.py", line 281 in _Popen
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/multiprocessing/process.py", line 121 in start
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/util/parallel.py", line 135 in _join_one
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/util/parallel.py", line 102 in join
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 474 in _read_parallel
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 418 in read
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 313 in build
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 293 in build_update
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/application.py", line 355 in build
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/cmd/build.py", line 298 in build_main
  File "/Users/goldbaum/.pyenv/versions/3.11.6/lib/python3.11/site-packages/sphinx/cmd/build.py", line 341 in main
  File "/Users/goldbaum/.pyenv/versions/3.11.6/bin/sphinx-build", line 8 in <module>

Sphinx uses multiprocessing (fork on unix and macos) to orchestrate parallel builds and sqlite3 isn't fork-safe. I see old issues and PRs related to similar issues (#8888 made the history db in-memory for the directive).

I did ipython history clear and now I'm no longer hitting segfaults and unfortunately can no longer reproduce the problem.

[ivanov]

Thanks for the report, @ngoldbaum - I think you're on the right track about sqlite3, because the top of the traceback line is in
IPython/core/history.py, line 247 in init_db

247         self.db = sqlite3.connect(str(self.hist_file), **kwargs)

(it moved down a line recently, currently at 248

ipython/IPython/core/history.py

Line 248 in 6046a5f

self.db = sqlite3.connect(str(self.hist_file), **kwargs)

)
So I'm not sure how or why we ended up down the code path but it sounds like this should not have been happening.