--no-network test option not recognised

[drew-parsons]

I'm trying to test a build of h5py 3.9.0 using the --no-network option defined in h5py/tests/conftest.py

But the option is not being recognised when launching tests inside python using h5py.run_tests(), for instance,

$ python3 -c 'import h5py; h5py.run_tests("-v -k ros3 --no-network")' 
ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]
__main__.py: error: unrecognized arguments: --no-network
  inifile: None
  rootdir: ~/test

The option does work successfully if I run as pytest from the command line, $ pytest-3 -v --no-network ~/.local/lib/python3.11/site-packages/h5py/tests/test_ros3.py

And the option does work from h5py.run_tests(), but only when enclosed in square brackets,

$ python3 -c 'import h5py; h5py.run_tests("-v -k ros3 [--no-network]")' 
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.0.0+repack -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('~/test/.hypothesis/examples')
rootdir: ~/test
plugins: xvfb-2.0.0, remotedata-0.4.0, asyncio-0.20.3, arraydiff-0.5.0, filter-subpackage-0.1.2, flaky-3.7.0, mock-3.11.1, mpi-0.6, hypothesis-6.80.0, astropy-header-0.2.2, cov-4.1.0, openfiles-0.5.0, astropy-0.10.0, doctestplus-0.13.0
asyncio: mode=Mode.STRICT
collected 741 items / 738 deselected / 3 selected                                                                                                                                                                 

tests/test_ros3.py::test_ros3 SKIPPED (ros3 driver not available)                                                                                                                                           [ 33%]
tests/test_ros3.py::test_ros3_s3_fails SKIPPED (ros3 driver not available)                                                                                                                                  [ 66%]
tests/test_ros3.py::test_ros3_s3uri SKIPPED (ros3 driver not available)                                                                                                                                     [100%]

================================================================================================ warnings summary =================================================================================================
../.local/lib/python3.11/site-packages/h5py/tests/test_ros3.py:21
  ~/.local/lib/python3.11/site-packages/h5py/tests/test_ros3.py:21: PytestUnknownMarkWarning: Unknown pytest.mark.nonetwork - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    pytest.mark.nonetwork

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== 3 skipped, 738 deselected, 1 warning in 0.39s ==================================================================================

This use of square brackets is not intuitive, I almost did not find it. The -v and -k options were accepted without square brackets. Does it mean there is some bug with the option configuration? Is there some existing documentation (possibly pytest documentation) for it? Or should the behaviour be documented e.g. in h5py/tests/conftest.py or in h5py docs?

System:

• Debian unstable
• Python 3.11.4
• debian package python3.11 3.11.4-1
• h5py 2.9.0 install by pip (not the debian package)
• HDF5 1.10.8

$ python3 -c 'import h5py; print(h5py.version.info)'
Summary of the h5py configuration
---------------------------------

h5py    3.9.0
HDF5    1.12.2
Python  3.11.4 (main, Jun  7 2023, 10:13:09) [GCC 12.2.0]
sys.platform    linux
sys.maxsize     9223372036854775807
numpy   1.24.2
cython (built with) 0.29.35
numpy (built against) 1.23.2
HDF5 (built against) 1.12.2

[ajelenak]

I implemented the --no-network option. The commands below work for me if run from the folder where the h5py's pytest.ini file is:

• python -c 'import h5py; h5py.run_tests("-v -k ros3 --no-network")'
• python -m pytest --pyargs h5py -v -k ros3 --no-network

[drew-parsons]

That's a good hint. The nonetwork marker is indeed defined in pytest.ini. So running tests from a different directory might be related to my issue (though in that case still strange that I got the option working with the square brackets). I'll investigate further.

[ajelenak]

The pytest skip messages indicate your libhdf5 was not built with the ros3 virtual file driver. Only if ros3 is available the --no-network pytest option will be checked.

[drew-parsons]

I see. That must mean the pip build didn't have ros. I'm actually trying to build (and test) from source, my system hdf5 does support ros3. I was using the pip build as a reference since I was getting the problem with the --no-network test option from the source build. The ros3 tests do pass for my source build, but I'm trying to set up a build configuration where --no-network can be specified explicitly (for building on systems without network access)

[drew-parsons]

Nope, I can't get it just with the pytest.ini file. I can't run the test from the source dir where pytest.ini comes from anyway, since that gives the ImportError You cannot import h5py from inside the install directory defined in __init__.py.

If I try the test from an empty directory then the behaviour is the same unrecognized arguments: --no-network even if I copy pytest.ini to that dir.

You're right, I was mislead by the SKIPPED (ros3 driver not available) message. The square brackets don't actually help. I'll update the issue title accordingly.

As a workaround I can effectively skip the net-related tests by accessing the marker directly via pytest -m "not nonetwork". That achieves the result I want, manually skipping network tests, but the logic of it is wrong (it would be more logical in that case if the marker was called network or needs_network, so that the not operator skips it).

But I'm otherwise stuck. Could the problem be a matter of pytest version? My pytest is 7.4.0. v7.4.0 was only recently released, it does break some things. pytest-dev/pytest#11134

[drew-parsons]

One more observation, running tests from an installed build.

Each of python3 -m pytest --pyargs h5py -v -k ros3 --no-network
python3 -m pytest -v -k ros3 --no-network
and python3 -m pytest --pyargs h5py -v -k ros3 --no-network /usr/lib/python3/dist-packages/h5py/tests/ (specifying the installed tests dir) all fail with unrecognized arguments: --no-network

But the tests skip successfully without --pyargs h5py, while specifying the tests dir

$ python3 -m pytest -v -k ros3 --no-network /usr/lib/python3/dist-packages/h5py/tests/
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.0.0+repack -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/projects/python/build/test/blank/.hypothesis/examples')
rootdir: /usr/lib/python3/dist-packages/h5py//tests
plugins: xvfb-2.0.0, remotedata-0.4.0, asyncio-0.20.3, arraydiff-0.5.0, filter-subpackage-0.1.2, flaky-3.7.0, mock-3.11.1, mpi-0.6, hypothesis-6.80.0, astropy-header-0.2.2, cov-4.1.0, openfiles-0.5.0, astropy-0.10.0, doctestplus-0.13.0
asyncio: mode=Mode.STRICT
collecting ... conftest.py detected option --no-network
collected 741 items / 738 deselected / 3 selected                                                                                                                                                                 

../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py::test_ros3 SKIPPED (No Internet)                                                                             [ 33%]
../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py::test_ros3_s3_fails SKIPPED (No Internet)                                                                    [ 66%]
../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py::test_ros3_s3uri SKIPPED (No Internet)                                                                       [100%]

================================================================================================ warnings summary =================================================================================================
../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py:21
  /usr/lib/python3/dist-packages/h5py/tests/test_ros3.py:21: PytestUnknownMarkWarning: Unknown pytest.mark.nonetwork - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    pytest.mark.nonetwork

../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py:28
  /usr/lib/python3/dist-packages/h5py/tests/test_ros3.py:28: PytestUnknownMarkWarning: Unknown pytest.mark.nonetwork - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    print("pytest.mark.nonetwork=",pytest.mark.nonetwork)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================= 3 skipped, 738 deselected, 4 warnings in 0.68s ==================================================================================

Specifying the tests dir like this without the --no-network test option, the tests pass:

...
../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py::test_ros3 PASSED                                                                                            [ 33%]
../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py::test_ros3_s3_fails PASSED                                                                                   [ 66%]
../../../../../../../usr/lib/python3/dist-packages/h5py/tests/test_ros3.py::test_ros3_s3uri PASSED                                                                                      [100%]

I get the PytestUnknownMarkWarning even if I copy pytest.ini to the empty dir I'm launching the tests from.

[ajelenak]

I am not a pytest expert but I cannot replicate your problem. I created a new conda environment and installed h5py from conda-forge channel. Both of the commands below work without any unknown marker warnings:

• python -m pytest -v -k ros3 --no-network $CONDA_PREFIX/lib/python3.11/site-packages/h5py/tests
• python -m pytest --pyargs h5py -v -k ros3 --no-network

Using pytest 7.4.0.

[drew-parsons]

The first version works for me, specifying the directory. The second one fails. I guess its related to the pytest conf discovery rules, which the pytest issues raise as a problem several times. Possibly the problem is triggered by the extra layer in the debian builds providing both serial and mpi variants. I'll try a direct build without the debian layering.