latest insiders build crashes my system

[janvanveldhuizen]

Context

No response

Bug description

Since I built the latest container, my laptop crashes when starting the website. The CPU fan is blowing full speed, and the system becomes unresponsive.
When I leave the machine running, then it finally ends with a console screen printing endless lines of out-of-memory error messages.
I keep looking for something on my side that is causing this, but I cannot find it.

Related links

• Reporting a bug

Reproduction

multi-language-blog.zip

Steps to reproduce

1. Take the latest insiders repo
2. git pull
3. docker build -t mkdocs-material .
4. extract the file multi-language-blog.zip to a new directory
5. docker run -p 8000:8000 -v ./:/docs mkdocs-material

Browser

Chrome

Before submitting

• I have read and followed the bug reporting guidelines.
• I have attached links to the documentation, and possibly related issues and discussions.
• I assure that I have removed all customizations before submitting this bug report.
• I have attached a .zip file with a minimal reproduction.

[janvanveldhuizen]

Maybe this helps:

I just ran a test by limiting the memory docker can use.

docker run --memory="1024m" -p 8000:8000 -v ./:/docs mkdocs-material 

The CPU fan blew for a while, and in htop I saw the cpu being busy with python processes.
After a while docker stopped with:

~/dev/example                                                                                                                                                                           
▶ docker run --memory="1024m" -p 8000:8000 -v ./:/docs mkdocs-material 
WARNING -  Config value 'dev_addr': The use of the IP address '0.0.0.0' suggests a production environment or the use of a proxy to connect to the MkDocs server. However, the MkDocs' server is intended for local development purposes only. Please use a third party production-ready server instead.
INFO    -  Building documentation...

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp0bkb_oqs'

[janvanveldhuizen]

I rolled back to commit 055bf86, cleaned up the docker environment and built a new container. Now, it works properly.

cd ~/dev/mkdocs-material-insiders
git checkout 055bf86
docker system prune
docker build -t mkdocs-material .
cd ~/dev/example
docker run --memory="1024m" -p 8000:8000 -v ./:/docs mkdocs-material

WARNING -  Config value 'dev_addr': The use of the IP address '0.0.0.0' suggests a production environment or the use of a proxy to connect to the MkDocs server. However, the MkDocs' server is intended for local development purposes only. Please use a third party production-ready server instead.
INFO    -  Building documentation...
INFO    -  Cleaning site directory
INFO    -  [project://.en] Cleaning site directory
INFO    -  [project://.en] Building documentation to directory: /docs/languages/en/site
INFO    -  [project://.de] Cleaning site directory
INFO    -  [project://.de] Building documentation to directory: /docs/languages/de/site
INFO    -  [project://.en] Documentation built in 0.17 seconds
INFO    -  [project://.de] Documentation built in 0.19 seconds
INFO    -  Documentation built in 0.27 seconds
INFO    -  [06:55:36] Watching paths for changes: 'docs', 'mkdocs.yml', 'languages/de/docs', 'languages/de/mkdocs.yml', 'languages/en/docs', 'languages/en/mkdocs.yml'
INFO    -  [06:55:36] Serving on http://0.0.0.0:8000/
INFO    -  [06:56:10] Browser connected: http://localhost:8000/en/

Then, I went back to master again

cd ~/dev/mkdocs-material-insiders
git checkout master
docker system prune
docker build -t mkdocs-material .
cd ~/dev/example
docker run --memory="1024m" -p 8000:8000 -v ./:/docs mkdocs-material

~/dev/example                                                                                                                                                                          ▶ docker run --memory="1024m" -p 8000:8000 -v ./:/docs mkdocs-material 
WARNING -  Config value 'dev_addr': The use of the IP address '0.0.0.0' suggests a production environment or the use of a proxy to connect to the MkDocs server. However, the MkDocs' server is intended for local development purposes only. Please use a third party production-ready server instead.
INFO    -  Building documentation...

~/dev/example                                                                                                                                                                          

So, it stops immediately without any message.

[squidfunk]

Thanks for reporting! Indeed, there seems to be some problem. Note that I did an entire rewrite of the plugin to solve the open issues and prepare for more advanced features to be added, so this is definitely related. I'll investigate.

[squidfunk]

Okay. So this is weird. It seems that on Linux (or Docker?) the address space between Python Processes is shared when using the ProcessPoolExecutor, as the child process gets the exact same instance of the ProjectsBuilder class, which is visible from the debug logging I inserted into the plugin:

build /docs/mkdocs.yml <material.plugins.projects.builder.ProjectsBuilder object at 0xffffaa0e6b90>
build /docs/languages/en/mkdocs.yml <material.plugins.projects.builder.ProjectsBuilder object at 0xffffaa0e6b90>

It looks like on Linux (or Docker?) the process is forked, while on macOS it is spawned. Looking at the multiprocessing context, this is indeed what is happening:

• macOS: <multiprocessing.context.SpawnContext object at 0x105bf4c10>
• Linux: <multiprocessing.context.ForkContext object at 0xffffaa05d210>

Thus, with afbb230, we now explicitly set the context to spawn, which fixes the issue on my machine. In fact, this also happened in our Examples GitHub Actions pipeline, so many thanks for reporting and creating an excellent reproduction. Being able to fire up Docker to check it made fixing this simple.

[janvanveldhuizen]

Great that you found and fixed it!

[squidfunk]

Released as part of 9.5.3+insiders-4.48.0, which also brings a brand new tags plugin!