Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest trying to check if custom argument is a file crashes due to filename being too long #10169

Closed
4 tasks done
jdckmz opened this issue Jul 25, 2022 · 6 comments · Fixed by #10988
Closed
4 tasks done
Labels
good first issue easy issue that is friendly to new contributor type: bug problem that needs to be addressed

Comments

@jdckmz
Copy link

jdckmz commented Jul 25, 2022

I have a custom flag defined in conftest.py, and when I try to assign it to a value that is too long pytest crashes before ever getting to my code. This reproduces even if the flag isn't defined, and even if the current working directory is /.

Failing example:

/> pytest --xxxxx_flags=" --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx"     
Traceback (most recent call last):
  File "/home/ANT.AMAZON.COM/jdckmz/.local/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 188, in console_main
    code = main()
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 146, in main
    config = _prepareconfig(args, plugins)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 325, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/helpconfig.py", line 102, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1013, in pytest_cmdline_parse
    self.parse(args)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1301, in parse
    self._preparse(args, addopts=addopts)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1203, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1080, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 525, in _set_initial_conftests
    if anchor.exists():  # we found some file object
  File "/usr/lib/python3.8/pathlib.py", line 1407, in exists
    self.stat()
  File "/usr/lib/python3.8/pathlib.py", line 1198, in stat
    return self._accessor.stat(self)
OSError: [Errno 36] File name too long: '/--xxxxx_flags= --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx'

If I reduce the length of the flag, I get the expected behavior for my project, and this different and expected error from my pytest MVP:

/> pytest --xxxxx_flags=" --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx"
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.10, pytest-7.0.0, pluggy-1.0.0
rootdir: /
plugins: flaky-3.7.0, colcon-core-0.10.0, cov-2.8.1
collected 0 items                                                                                                                                                          

============================================================================= warnings summary =============================================================================
home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/cacheprovider.py:433
  /home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/stepwise.py:52
  /home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================================================================== 2 warnings in 0.01s ============================================================================
ERROR: file or directory not found: --xxxxx_flags= --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx

I did a little digging into my version of pytest (7.0.0) to make sure I wasn't doing something wrong, but it looks like there is a blind call to pathlib.Path.exists() with a path constructed from the argument in __init__.py:

    #
    # Internal API for local conftest plugin handling.
    #
    def _set_initial_conftests(
        self, namespace: argparse.Namespace, rootpath: Path
    ) -> None:
    ...
    testpaths = namespace.file_or_dir
    foundanchor = False
    for testpath in testpaths:
            path = str(testpath)
            i = path.find("::")
            if i != -1:
                path = path[:i]
            anchor = absolutepath(current / path)
            if anchor.exists():  # this throws OSError which is never caught

It seems to me like there should be a try or something here, since in cases like mine the argument may not be a file at all, and that can cause OS level errors.

Operating System: Ubuntu 20.04 LTS

> pytest --version
pytest 7.0.0
> python3 --version
Python 3.8.10
> pip list
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Package                       Version
----------------------------- --------------------
aiohttp                       3.8.1
aiosignal                     1.2.0
alabaster                     0.7.12
apturl                        0.5.2
argcomplete                   1.8.1
astroid                       2.9.3
async-timeout                 4.0.2
atomicwrites                  1.4.0
attrs                         21.4.0
autobahn                      17.10.1
Automat                       0.8.0
aws-requests-auth             0.4.3
awscli                        1.22.52
awscrt                        0.13.0
awsiotsdk                     1.9.0
Babel                         2.9.1
bcrypt                        3.2.0
beautifulsoup4                4.8.2
black                         22.1.0
blinker                       1.4
boto3                         1.20.52
botocore                      1.23.52
Brlapi                        0.7.0
cached-property               1.5.1
catkin-pkg-modules            0.5.2
cbor                          1.0.0
certifi                       2021.10.8
cffi                          1.15.0
chardet                       4.0.0
charset-normalizer            2.0.11
click                         8.0.3
cmakelang                     0.6.13
cmakelint                     1.4.2
colcon-argcomplete            0.3.3
colcon-bash                   0.4.2
colcon-cd                     0.1.1
colcon-cmake                  0.2.26
colcon-common-extensions      0.3.0
colcon-core                   0.10.0
colcon-defaults               0.2.6
colcon-devtools               0.2.3
colcon-library-path           0.2.1
colcon-metadata               0.2.5
colcon-notification           0.2.13
colcon-output                 0.2.12
colcon-package-information    0.3.3
colcon-package-selection      0.2.10
colcon-parallel-executor      0.2.4
colcon-pkg-config             0.1.0
colcon-powershell             0.3.7
colcon-python-setup-py        0.2.7
colcon-recursive-crawl        0.2.1
colcon-ros                    0.3.23
colcon-test-result            0.3.8
colcon-zsh                    0.4.0
colorama                      0.4.3
command-not-found             0.3
constantly                    15.1.0
control                       0.9.1
cov-core                      1.15.0
coverage                      4.5.2
cryptography                  36.0.1
cupshelpers                   1.0
cycler                        0.11.0
Cython                        0.29.14
dbus-python                   1.2.16
defer                         1.0.6
distlib                       0.3.4
distro                        1.4.0
distro-info                   0.23ubuntu1
docker                        5.0.3
docker-compose                1.25.0
dockerpty                     0.4.1
docopt                        0.6.2
docutils                      0.15.2
duplicity                     0.8.12.0
EasyCluster                   0.22.2
empy                          3.3.2
entrypoints                   0.3
evdev                         1.3.0
fasteners                     0.14.1
filelock                      3.7.1
filemagic                     1.6
flake8                        3.7.9
flaky                         3.7.0
fonttools                     4.29.1
frozenlist                    1.3.0
future                        0.18.2
gitdb                         4.0.9
gitdb2                        4.0.2
github.py                     0.5.0
GitPython                     3.1.26
gpg                           1.13.1-unknown
greenlet                      1.1.2
html5lib                      1.0.1
httplib2                      0.14.0
hyperlink                     19.0.0
idna                          3.3
ifcfg                         0.18
imagesize                     1.3.0
importlib-metadata            4.10.1
incremental                   16.10.1
influxdb                      5.3.1
iniconfig                     1.1.1
isort                         5.10.1
Jinja2                        3.0.3
jmespath                      0.10.0
jsonschema                    3.2.0
keyring                       18.0.1
keyrings.alt                  3.4.0
kiwisolver                    1.3.2
language-selector             0.1
lark-parser                   0.8.1
launchpadlib                  1.10.13
lazr.restfulclient            0.14.2
lazr.uri                      1.0.3
lazy-object-proxy             1.7.1
lockfile                      0.12.2
louis                         3.12.0
lxml                          4.5.0
lz4                           3.0.2+dfsg
macaroonbakery                1.3.1
Mako                          1.1.0
MarkupSafe                    2.0.1
matplotlib                    3.5.1
mccabe                        0.6.1
mock                          3.0.5
monotonic                     1.5
more-itertools                8.12.0
mpi4py                        3.0.3
msgpack                       1.0.3
multi-key-dict                2.0.3
multidict                     6.0.2
mypy-extensions               0.4.3
netifaces                     0.10.4
nose2                         0.9.1
notify2                       0.3
numpy                         1.22.2
oauthlib                      3.1.0
olefile                       0.46
packaging                     21.3
pandas                        1.4.0
paramiko                      2.9.2
pathspec                      0.9.0
pbr                           5.8.1
pexpect                       4.8.0
Pillow                        9.0.1
pip                           22.1.2
pipenv                        2022.6.7
platformdirs                  2.5.0
pluggy                        1.0.0
protobuf                      3.19.4
psutil                        5.8.0
ptyprocess                    0.7.0
py                            1.11.0
py-ubjson                     0.14.0
pyasn1                        0.4.8
pyasn1-modules                0.2.1
pybind11                      2.8.0
pycairo                       1.16.2
pycodestyle                   2.8.0
pycparser                     2.21
pycrypto                      2.6.1
pycups                        1.9.73
pydocstyle                    2.1.1
pydot                         1.4.1
pyelftools                    0.28
pyflakes                      2.1.1
Pygments                      2.11.2
PyGObject                     3.36.0
PyHamcrest                    1.9.0
PyJWT                         1.7.1
pylint                        2.12.2
pymacaroons                   0.13.0
PyNaCl                        1.5.0
pyOpenSSL                     19.0.0
pyparsing                     3.0.7
pypng                         0.0.20
PyQRCode                      1.2.1
PyQt5                         5.14.1
pyquaternion                  0.9.9
pyRFC3339                     1.1
pyrsistent                    0.15.5
pyserial                      3.5
pytest                        7.0.0
pytest-cov                    2.8.1
python-apt                    2.0.0+ubuntu0.20.4.7
python-dateutil               2.8.2
python-debian                 0.1.36ubuntu1
python-dotenv                 0.19.2
python-jenkins                1.7.0
python-magic                  0.4.16
python-snappy                 0.5.3
PyTrie                        0.2
pytz                          2021.3
pyxdg                         0.26
PyYAML                        5.3.1
reportlab                     3.5.34
requests                      2.27.1
requests-unixsocket           0.2.0
roman                         2.0.0
rosdistro-modules             0.9.0
rospkg-modules                1.4.0
rplidar                       0.9.2
rsa                           4.7.2
s3transfer                    0.5.1
scipy                         1.8.0
screen-resolution-extra       0.0.0
SecretStorage                 2.3.1
service-identity              18.1.0
setproctitle                  1.1.10
setuptools                    45.2.0
simplejson                    3.16.0
sip                           4.19.21
six                           1.16.0
smmap                         5.0.0
smmap2                        3.0.1
snowballstemmer               2.2.0
soupsieve                     1.9.5
Sphinx                        4.4.0
sphinx-autoapi                1.8.4
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-dotnetdomain    0.4
sphinxcontrib-golangdomain    0.2.0.dev0
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport      1.2.4
SQLAlchemy                    1.4.35
ssh-import-id                 5.10
tensorrt                      8.0.1.6
texttable                     1.6.2
toml                          0.10.2
tomli                         2.0.1
tripy                         1.0.0
Twisted                       18.9.0
txaio                         2.10.0
typed-ast                     1.5.2
typing_extensions             4.0.1
u-msgpack-python              2.1
ubuntu-advantage-tools        27.9
ubuntu-drivers-common         0.0.0
ufw                           0.36
unattended-upgrades           0.1
Unidecode                     1.3.2
urllib3                       1.26.8
usb-creator                   0.3.7
virtualenv                    20.14.1
virtualenv-clone              0.5.7
wadllib                       1.3.3
wcwidth                       0.1.8
webencodings                  0.5.1
websocket-client              1.2.3
wheel                         0.34.2
wrapt                         1.13.3
wsaccel                       0.6.2
xdot                          1.1
xkit                          0.0.0
xmltodict                     0.12.0
yarl                          1.7.2
zipp                          3.7.0
zope.interface                4.7.1
zstandard                     0.17.0
  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible
@nicoddemus nicoddemus added type: bug problem that needs to be addressed good first issue easy issue that is friendly to new contributor labels Jul 26, 2022
@nicoddemus
Copy link
Member

Thanks for the report @jdckmz!

@PurityLake
Copy link
Contributor

From what I can tell, if we have got to this point there is no pluginmanager has picked up the command line arg, so is there really anything we can do?

@jdckmz
Copy link
Author

jdckmz commented Jul 28, 2022

The args that I've added aren't part of a plugin but are part of pytest itself as I understand, documented here:
In conftest.py:

def pytest_addoption(parser):
    """pytest hook to add command line options."""

    group = parser.getgroup("Canvas Test Options")
    group.addoption(
        "--xxxxx_flags",
        default=None,
        help="Extra flags to pass to the launched process.",
    )

@PurityLake
Copy link
Contributor

@jdckmz

Ok after some investigation, you need to make --xxxxx_flags into a pytest.fixture for it to not register as a path

@arichardson
Copy link

I am seeing the same thing, there is even a comment in _set_initial_conftests that suggests this could be an issue:

        """Load initial conftest files given a preparsed "namespace".

        As conftest files may add their own command line options which have
        arguments ('--my-opt somepath') we might get some false positives.
        All builtin and 3rd party plugins will have been loaded, however, so
        common options will not confuse our logic here.
        """

Maybe there should be a try/except around the exists call? Or alternatively run this logic after trying to parse the arguments?

arichardson added a commit to CTSRD-CHERI/cheritest that referenced this issue May 11, 2023
Due to pytest-dev/pytest#10169 we can't have
command line flags with long names since pytest will check if any argument
is a valid path (even if it's registered as a command line option) and
Path.exist() may raise an exception which kills the entire pytest process.
Work around this by splitting the single list of unsupported options into
one option per feature.
@nicoddemus
Copy link
Member

@arichardson funny you posted this today, as I did a workaround for that just yesterday in https://github.com/pytest-dev/pytest/pull/10988/files#diff-df52f8f6a3544754cc8ebdf903594738e68a18dc9ac3c959f646cf4705a9afed.

Seems however we should remove that guard against Python 3.7 and always do the try/except to also solves this.

nicoddemus added a commit to nicoddemus/pytest that referenced this issue May 12, 2023
`_set_initial_conftests` could break on some systems if a very long
option was passed, because the `Path.exists()` call raises an
`OSError` instead of returning `False`.

Fix pytest-dev#10169
nicoddemus added a commit to nicoddemus/pytest that referenced this issue May 12, 2023
`_set_initial_conftests` could break on some systems if a very long
option was passed, because the `Path.exists()` call raises an
`OSError` instead of returning `False`.

Fix pytest-dev#10169
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue easy issue that is friendly to new contributor type: bug problem that needs to be addressed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants