Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utf8 mode seems not working while loading site module with Python 3.11/3.12 in Windows #117793

Closed
wkwkw opened this issue Apr 12, 2024 · 1 comment
Assignees
Labels
3.11 only security fixes 3.12 bugs and security fixes stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error

Comments

@wkwkw
Copy link

wkwkw commented Apr 12, 2024

Bug report

Bug description:

I installed python 3.12.2 with rye 0.32.0, and it created a pth file that contains non-ascii file path.

$ cat .venv/Lib/site-packages/_tesuto.pth
C:\Users\XXXXXXXXX\Documents\tmp\テスト\src

Processing this file causes a UnicodeDecodeError and python prompt fails to open. (#87380)
I tried running python in utf-8 mode via $ python -X utf8 and PYTHONUTF8=1 environment variable, but not worked.
(not also with 3.11.8)

Error message is like following:

$ python -X utf8
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1176, in exec_module
  File "<frozen site>", line 624, in <module>
  File "<frozen site>", line 607, in main
  File "<frozen site>", line 539, in venv
  File "<frozen site>", line 394, in addsitepackages
  File "<frozen site>", line 236, in addsitedir
  File "<frozen site>", line 188, in addpackage
UnicodeDecodeError: 'cp932' codec can't decode byte 0x86 in position 35: illegal multibyte sequence

related part in messages from -v option:

Processing global site-packages
Adding directory: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv'
Adding directory: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv\\Lib\\site-packages'
Processing .pth file: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv\\Lib\\site-packages\\_tesuto.pth'
# C:\Users\XXXXXXXX\.rye\py\cpython@3.12.2\Lib\encodings\__pycache__\cp932.cpython-312.pyc matches C:\Users\XXXXXXXX\.rye\py\cpython@3.12.2\Lib\encodings\cp932.py

tried but not worked

  • installing python via pyenv

worked

  • using python 3.10 or earlier
  • changing the encoding of the pth file to cp932

CPython versions tested on:

3.12

Operating systems tested on:

Windows

@wkwkw wkwkw added the type-bug An unexpected behavior, bug, or error label Apr 12, 2024
@arhadthedev arhadthedev added 3.11 only security fixes 3.12 bugs and security fixes stdlib Python modules in the Lib dir topic-unicode labels Apr 12, 2024
@methane methane self-assigned this Apr 12, 2024
@methane
Copy link
Member

methane commented Apr 12, 2024

It seems duplicate of #77102

@methane methane closed this as completed Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes 3.12 bugs and security fixes stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants