Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest.warns() asserts when pytest.skip() is called #11907

Closed
4 tasks done
mgorny opened this issue Feb 1, 2024 · 3 comments · Fixed by #11920
Closed
4 tasks done

pytest.warns() asserts when pytest.skip() is called #11907

mgorny opened this issue Feb 1, 2024 · 3 comments · Fixed by #11920
Labels
plugin: warnings related to the warnings builtin plugin type: bug problem that needs to be addressed type: regression indicates a problem that was introduced in a release which was working previously

Comments

@mgorny
Copy link
Contributor

mgorny commented Feb 1, 2024

When pytest.skip() is used inside pytest.warns() (and no warning is emitted), the test fails. I'm not sure if this is intentional but it feels a bit off, given that the test would normally be skipped instead. I've hit this in the wild in urllib3's test suite.

Quick reproducer:

import pytest


def test_foo():
    with pytest.warns(UserWarning):
        pytest.skip("skipping")

Yields:

$ python -m pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.11.7, pytest-8.1.0.dev116+g4546d5445, pluggy-1.4.0
rootdir: /tmp/test
collected 1 item                                                                                                                      

test_foo.py F                                                                                                                   [100%]

============================================================== FAILURES ===============================================================
______________________________________________________________ test_foo _______________________________________________________________

    def test_foo():
        with pytest.warns(UserWarning):
>           pytest.skip("skipping")
E           Skipped: skipping

test_foo.py:6: Skipped

During handling of the above exception, another exception occurred:

    def test_foo():
>       with pytest.warns(UserWarning):
E       Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
E        Emitted warnings: [].

test_foo.py:5: Failed
======================================================= short test summary info =======================================================
FAILED test_foo.py::test_foo - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
========================================================== 1 failed in 0.02s ==========================================================

pip list:

Package    Version
---------- -----------------------
iniconfig  2.0.0
packaging  23.2
pip        23.3.2
pluggy     1.4.0
pytest     8.1.0.dev116+g4546d5445
setuptools 69.0.3

  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible
@bluetech
Copy link
Member

bluetech commented Feb 4, 2024

Hmm this is from 15524f3. One might argue that the test should indeed fail because it actually did not warn, but I'd say the previous behavior is the more expected one. Probably pytest exceptions (fail, skip, xfail, exit) need to be special-cased here.

cc @Cheukting @Zac-HD

@bluetech bluetech added type: bug problem that needs to be addressed type: regression indicates a problem that was introduced in a release which was working previously plugin: warnings related to the warnings builtin plugin labels Feb 4, 2024
@Zac-HD
Copy link
Member

Zac-HD commented Feb 4, 2024

I agree that we should add a special case for these exceptions.

@bluetech
Copy link
Member

bluetech commented Feb 4, 2024

Posted a fix in #11920. I opted to let all base exceptions (e.g. keyboard interrupt, sys.exit, etc.) propagate, not only pytest ones.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: warnings related to the warnings builtin plugin type: bug problem that needs to be addressed type: regression indicates a problem that was introduced in a release which was working previously
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants