3.42.0 breaks automatic reload after a change is detect

[mangiucugna]

Describe the bug

in Gradio 3.42.0 the autoreload when a change is detected breaks with an error and the only way to continue is to kill and restart gradio again.

My only action was to upgrade the package from 3.41.2 to 3.42.0 (side note: why those packages are not present in the releases in github?)

Have you searched existing issues? 🔎

• I have searched and found no existing issues

Reproduction

Upgrade Gradio to 3.42.0 and run gradio app.py, then change something in app.py

Screenshot

No response

Logs

Changes detected in: /Users/stefanobaccianella/Repositories/difficult-conversations-bot/app.py
/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/components/annotated_image.py:7: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
Reloading app failed with the following exception: 
ImportError: PyO3 modules may only be initialized once per interpreter process
Changes detected in: /Users/stefanobaccianella/Repositories/difficult-conversations-bot/app.py
/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/components/annotated_image.py:7: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
Reloading app failed with the following exception: 
ImportError: PyO3 modules may only be initialized once per interpreter process

System Info

Running python 3.11.4

pip freeze:
aiofiles==23.2.1
aiohttp==3.8.5
aiosignal==1.3.1
alt-profanity-check==1.3.0
altair==5.0.1
annotated-types==0.5.0
anyio==3.7.1
async-timeout==4.0.3
attrs==23.1.0
cachetools==5.2.0
certifi==2023.7.22
cfgv==3.4.0
charset-normalizer==3.2.0
click==8.1.7
contourpy==1.1.0
cycler==0.11.0
dataclasses-json==0.5.14
distlib==0.3.7
ecdsa==0.18.0
fastapi==0.101.1
ffmpy==0.3.1
filelock==3.12.2
fonttools==4.42.1
frozenlist==1.4.0
fsspec==2023.6.0
google-api-core==2.11.1
google-auth==2.22.0
googleapis-common-protos==1.60.0
gradio==3.42.0
gradio_client==0.5.0
h11==0.14.0
httpcore==0.17.3
httpx==0.24.1
huggingface-hub==0.16.4
identify==2.5.27
idna==3.4
importlib-resources==6.0.1
iniconfig==2.0.0
Jinja2==3.1.2
joblib==1.3.2
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
kiwisolver==1.4.5
langchain==0.0.279
langsmith==0.0.26
MarkupSafe==2.1.3
marshmallow==3.20.1
matplotlib==3.7.2
mona-openai==0.1.3
mona-sdk==0.0.51
multidict==6.0.4
mypy==1.5.1
mypy-extensions==1.0.0
nest-asyncio==1.5.7
nodeenv==1.8.0
numexpr==2.8.5
numpy==1.25.2
openai==0.28.0
orjson==3.9.5
packaging==23.1
pandas==2.0.3
phonenumberslite==8.13.19
Pillow==10.0.0
platformdirs==3.10.0
pluggy==1.3.0
pre-commit==3.3.3
protobuf==4.24.1
pyasn1==0.5.0
pyasn1-modules==0.3.0
pydantic==2.3.0
pydantic_core==2.6.3
pydub==0.25.1
PyJWT==1.7.1
pyparsing==3.0.9
pytest==7.4.0
pytest-mock==3.11.1
python-dateutil==2.8.2
python-dotenv==1.0.0
python-jose==3.3.0
python-multipart==0.0.6
pytz==2023.3
PyYAML==6.0.1
referencing==0.30.2
regex==2023.8.8
requests==2.31.0
requests-mock==1.11.0
rpds-py==0.9.2
rsa==4.9
scikit-learn==1.3.0
scipy==1.11.2
semantic-version==2.10.0
six==1.16.0
sniffio==1.3.0
SQLAlchemy==2.0.20
starlette==0.27.0
tenacity==8.2.3
threadpoolctl==3.2.0
tiktoken==0.4.0
toolz==0.12.0
tqdm==4.66.1
typing-inspect==0.9.0
typing_extensions==4.7.1
tzdata==2023.3
urllib3==1.26.16
uvicorn==0.23.2
virtualenv==20.24.4
websockets==11.0.3
yarl==1.9.2

Severity

I can work around it

[abidlabs]

Hi @mangiucugna we changed the way reload mode works, which may be leading to this. Can you share the code for app.py so that we can reproduce?

[mangiucugna]

Sure thing, the repo is public https://huggingface.co/spaces/mangiucugna/difficult-conversations-bot/tree/main

just upgrade gradio to latest to reproduce

[mangiucugna]

Ok I have dug a bit deeper and the problem seems to be related to tiktoken, imported by langchain, that is importing the rust bindings, and those will definitely fail if reimported again.
There is an issue opened with tiktoken about it openai/tiktoken#141
I don't think I am the only one using langchain and tiktoken with Gradio so imho it would be wise to revert PR 5267, as it also causes the double import of Numpy that is in the output I shared (although not a blocking error).
I couldn't find a solution in python to avoid re-initializing a module using importlib but there are smarter people around that can find a solution for this problem I am sure.

Tagging @freddyaboulton as he is the author of the PR.

[kiri-i]

It seems that this bug is still unresolved. The same issue occurs in version 3.50.2.
When I downgraded to 3.41.2 as a test, the error no longer occurred.
It appears to be related to the openai module.

$ gradio main.py 
Watching: '/home/kiri/Code/ChatRead/.venv/lib/python3.10/site-packages/gradio', '/home/kiri/Code/ChatRead'

Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

Changes detected in: /home/kiri/Code/ChatRead/main.py
/home/kiri/Code/ChatRead/.venv/lib/python3.10/site-packages/openai/datalib/numpy_helper.py:4: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy
Reloading main failed with the following exception: 
ImportError: PyO3 modules may only be initialized once per interpreter process

[mangiucugna]

It’s definitely a bug in tiktoken, but is triggered by the PR I mentioned above. Not sure it’s possible to fix without reverting that PR (or we all try to fix tiktoken but not sure if that’s a good use of our collective time)

[mangiucugna]

btw, so that we are clear, PyO3 does not consider this a bug.
The changelog says that since version 0.17 this is an expected behaviour, we can argue it's not great but for that project it isn't a bug.
Ideally there would be a way to exclude some modules from being reloaded, numpy and PyO3 are the ones throwing warnings/errors for me, but I am not sure it's possible (or at least my python abilities aren't advanced enough to come up with a solution, I even asked GPT-4)

[freddyaboulton]

I think what we'll have to do is introduce a --relaunch-server flag or something similar to shut off and start the server on changes. It would be slower development experience but I can't think of another way to solve the issue of extension modules not liking being reloaded in the general sense. Thanks @mangiucugna !

[mangiucugna]

yeah I agree, I have been going back and forth on a possible solution but there are too many libraries that use Rust bindings (some also inside gradio through orjson) so a flag sounds right to me.
Cheers

[GeorgeStrakhov]

happening for me as well

[freddyaboulton]

@mangiucugna @GeorgeStrakhov Can you guys try developing in reload mode in a jupyter notebook if you are using libs with rust bindings?

You %load_ext gradio and then do %%blocks at the top of the cell with your gradio app. Define all of your imports outside of the cell with %%blocks and do not call launch. The magic command will take care of that for you. It seems to work well with tiktoken and numpy.

I started implementing the --relaunch-server flag but maybe there's a way to make reload mode for scripts work the same as it does for jupyer notebooks. Like maybe you add a @reload decorator to the things you want to reload only?

[mangiucugna]

I will try to make time in the coming days to test. I like the idea of reloading only some parts, in fact my understanding is that if we could just skip the imports the problem would be solved.
Not sure if it's feasible though

[freddyaboulton]

Thank you!

[mangiucugna]

So I tried it, it worked fine this way. Not sure how this tests the reload feature for you but it worked as expected

[freddyaboulton]

Thanks @mangiucugna - will think about how we can make reload mode work like a jupyter notebook from the command line after the 4.0 launch next week!

[will-hill]

+1 to please implementing this feature. Very cool system, otherwise.

[patrolli]

So I tried it, it worked fine this way. Not sure how this tests the reload feature for you but it worked as expected

I faced the same issue, could you tell me how you solve this problem? I have no idea about how to add the reload decorator...

[mangiucugna]

An update here, the folks over at tiktoken have upgraded to the latest pyo3 that seems to solve this problem openai/tiktoken#141

I think now we need to wait for all rust-binded libraries to upgrade (orjson comes to mind for gradio)

[BramVanroy]

Any updates on this? Perhaps dependencies should be pinned to unaffected versions?

[abidlabs]

I believe this should be resolved as of gradio==4.14.0. Can someone please let us know if that's not the case and provide a simple repro?

[mangiucugna]

just tried with latest and the error changed, so I guess progress :)

UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_runtime.py:184: UserWarning: Pydantic is installed but cannot be imported. Please check your installation. `huggingface_hub` will default to not using Pydantic. Error message: '{e}'
  warnings.warn(
Reloading app failed with the following exception: 
Traceback (most recent call last):
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/utils.py", line 207, in watchfn
    module = importlib.import_module(reloader.watch_module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/app.py", line 3, in <module>
    import gradio as gr
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/__init__.py", line 3, in <module>
    import gradio._simple_templates
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/_simple_templates/__init__.py", line 1, in <module>
    from .simpledropdown import SimpleDropdown
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/_simple_templates/simpledropdown.py", line 6, in <module>
    from gradio.components.base import FormComponent
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/components/__init__.py", line 1, in <module>
    from gradio.components.annotated_image import AnnotatedImage
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/components/annotated_image.py", line 11, in <module>
    from gradio import processing_utils, utils
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/processing_utils.py", line 22, in <module>
    from gradio.data_classes import FileData, GradioModel, GradioRootModel
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/gradio/data_classes.py", line 12, in <module>
    from fastapi import Request
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/__init__.py", line 7, in <module>
    from .applications import FastAPI as FastAPI
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/applications.py", line 16, in <module>
    from fastapi import routing
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 22, in <module>
    from fastapi import params
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/params.py", line 5, in <module>
    from fastapi.openapi.models import Example
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/openapi/models.py", line 4, in <module>
    from fastapi._compat import (
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/_compat.py", line 20, in <module>
    from fastapi.exceptions import RequestErrorModel
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/fastapi/exceptions.py", line 3, in <module>
    from pydantic import BaseModel, create_model
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/pydantic/__init__.py", line 3, in <module>
    import pydantic_core
  File "/Users/stefanobaccianella/Repositories/difficult-conversations-bot/.venv/lib/python3.11/site-packages/pydantic_core/__init__.py", line 6, in <module>
    from ._pydantic_core import (
ImportError: PyO3 modules may only be initialized once per interpreter process

[rob]

I believe this should be resolved as of gradio==4.14.0. Can someone please let us know if that's not the case and provide a simple repro?

I'm still having the same issue as above, even on 4.14.0.

[zZxztxZz]

Same issue on 4.15.0

code

import gradio as gr
import os
print(gr.__version__)
os.environ["no_proxy"] = "localhost,127.0.0.1,::1"
def run_webui():
    with gr.Blocks() as demo:
        with gr.Row():
            gpt_dropdown = gr.Dropdown()
            sovits_dropdown = gr.Dropdown()
            model_refresh_button = gr.Button("Refresh Model" ,variant="primary")
            
    demo.launch(server_port=2777)

if __name__ == "__main__":
    run_webui()

gradio it, change the code and save, then the issue occur

logs

PS C:\ _MYPATH_> gradio .\_FILENAME_.py
Watching: 'C:\_MYPATH_\myenv\Lib\site-packages\gradio', 'C:\_MYPATH_', 'C:\_MYPATH_'

4.15.0
Running on local URL:  http://127.0.0.1:2777

To create a public link, set `share=True` in `launch()`.
Changes detected in: _MYPATH_\_FILENAME_.py
C:\_PATH_\myenv\Lib\site-packages\gradio\components\annotated_image.py:7: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues 
and is discouraged.
  import numpy as np
4.15.0
4.15.0
Exception in thread Thread-5:
Traceback (most recent call last):
  File "C:\Python312\Lib\threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "C:\Python312\Lib\threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "C:\_PATH_\myenv\Lib\site-packages\gradio\utils.py", line 217, in watchfn
    demo = getattr(module, reloader.demo_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module '_FILENAME_' has no attribute 'demo'

[freddyaboulton]

Hi @zZxztxZz ! That's expected. Can you please create demo outside of a function. Something like

demo = create_webui()

if __name__ == "__main__":
    demo.launch(...)

[zZxztxZz]

Thank you @freddyaboulton! I changed my code like this,and it won't crash when automatic reload.

code

import gradio as gr
import os
print(gr.__version__)
os.environ["no_proxy"] = "localhost,127.0.0.1,::1"
def create_webui():
    with gr.Blocks() as demo:
        with gr.Row():
            gpt_dropdown = gr.Dropdown()
            sovits_dropdown = gr.Dropdown()
            model_refresh_button = gr.Button("Refresh Model" ,variant="primary")
    return demo    
    
demo = create_webui()    

if __name__ == "__main__":
    demo.launch(server_port=2777)

logs

PS C:\ _MYPATH_> gradio .\_FILENAME_.py
Watching: 'C:\_MYPATH_\myenv\Lib\site-packages\gradio', 'C:\_MYPATH_', 'C:\_MYPATH_'

4.15.0
Running on local URL:  http://127.0.0.1:2777

To create a public link, set `share=True` in `launch()`.
Changes detected in: _MYPATH_\_FILENAME_.py
C:\_PATH_\myenv\Lib\site-packages\gradio\components\annotated_image.py:7: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues 
and is discouraged.
  import numpy as np
4.15.0
4.15.0

[yiyiiiiiii]

I'm still having issues with not being able to autoreload, also with numpy @freddyaboulton

code
`import gradio as gr
import produce_audio
import produce_video

def create():
with gr.Blocks() as demo:
with gr.Row():
with gr.Column():
txt = gr.Textbox(label="请输入文字")
txt2audio = gr.Button("Create audio", variant="primary")
audio = gr.Audio(type="filepath")
txt2audio.click(fn=produce_audio.produce_audio, inputs=txt, outputs=audio)
with gr.Column():
video = gr.Video(label="上传要转换视频", width=100)
qua = gr.Radio(["Fast", "Improved", "Enhanced"], value="Enhanced", label="模式")
wav2lip = gr.Button("create video", variant="primary")
with gr.Column():
video_output = gr.Video(label="最终生成视频")
wav2lip.click(fn=produce_video.process_audio_video, inputs=[audio, video, qua], outputs=video_output)
return demo

demo = create()

if name == "main":
demo.launch()
logs UserWarning: The NumPy module was reloaded (imported a second time). This can in some case
s result in small but subtle issues and is discouraged.
import numpy as np
Reloading ui failed with the following exception:
Traceback (most recent call last):
File "D:\AI\Wav2lip\Easy-Wav2Lip\venv\lib\site-packages\gradio\utils.py", line 207, in watchfn
module = importlib.import_module(reloader.watch_module_name)
File "C:\Users\16558\AppData\Local\Programs\Python\Python310\lib\importlib_init_.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "D:\AI\Wav2lip\Easy-Wav2Lip\ui.py", line 3, in
import produce_video
File "D:\AI\Wav2lip\Easy-Wav2Lip\produce_video.py", line 4, in
from easy_functions import (format_time,
File "D:\AI\Wav2lip\Easy-Wav2Lip\easy_functions.py", line 1, in
import torch
File "D:\AI\Wav2lip\Easy-Wav2Lip\venv\lib\site-packages\torch_init.py", line 1119, in
from ._tensor import Tensor
File "D:\AI\Wav2lip\Easy-Wav2Lip\venv\lib\site-packages\torch_tensor.py", line 21, in
from torch.overrides import (
File "D:\AI\Wav2lip\Easy-Wav2Lip\venv\lib\site-packages\torch\overrides.py", line 1591, in
has_torch_function = _add_docstr(
RuntimeError: function '_has_torch_function' already has a docstring
`

[freddyaboulton]

Can you please share your code as well as the python and gradio versions your are using @yiyiiiiiii ?

[freddyaboulton]

Hi all, I am working on a fix for this in #7684 . It introduces a gr.no_reload() context manager. Any code in that context will not be reloaded so you can put any import statements that break on reload there (like numpy and tiktoken). Can you please test it out to see if it works? You can install that PR with the following command. The PR also has some sample code to showcase how to use gr.no_reload()

pip install https://gradio-builds.s3.amazonaws.com/e18d2583269511a252ad3e9d610efcaee09e79d3/gradio-4.21.0-py3-none-any.whl

If you encounter any problems please share all code and library versions needed to reproduce so I can investigate.

cc @mangiucugna