Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suppressed deprecations incorrectly stop test execution when execution should be stopped on deprecation #6100

Closed
DannyMeyer opened this issue Jan 10, 2025 · 0 comments
Assignees
Labels
feature/test-runner CLI test runner type/bug Something is broken version/11 Something affects PHPUnit 11

Comments

@DannyMeyer
Copy link

Q A
PHPUnit version 11.5.2
PHP version 8.3
Installation Method Composer

Summary

With the following flags:

stopOnDeprecation="true"
failOnDeprecation="true"
failOnPhpunitDeprecation="true"
displayDetailsOnPhpunitDeprecations="true"
displayDetailsOnTestsThatTriggerDeprecations="true"
displayDetailsOnTestsThatTriggerErrors="true"
displayDetailsOnTestsThatTriggerNotices="true"
displayDetailsOnTestsThatTriggerWarnings="true"

phpunit will stop the further execution of the tests after the first suppressed deprecation occurs. In this case the test will be shown as OK without any notices.

Current behavior

When suppressed deprecations occurs with active stopOnDeprecation="true" flag, phpunit stops further tests without any notice and the test will be successful.
Note:
Suppressed deprecations are used by Symfony: https://github.com/symfony/deprecation-contracts/blob/main/function.php#L25

How to reproduce

phpunit.xml

<phpunit
    ...

    stopOnDeprecation="true"
    failOnDeprecation="true"
    failOnPhpunitDeprecation="true"
    displayDetailsOnPhpunitDeprecations="true"
    displayDetailsOnTestsThatTriggerDeprecations="true"
    displayDetailsOnTestsThatTriggerErrors="true"
    displayDetailsOnTestsThatTriggerNotices="true"
    displayDetailsOnTestsThatTriggerWarnings="true"
>

Test:

<?php

declare(strict_types=1);

namespace unit;

use PHPUnit\Framework\Attributes\CoversNothing;
use PHPUnit\Framework\TestCase;

#[CoversNothing]
class Test extends TestCase
{
    public function testFirst(): void
    {
        @trigger_error('test', E_USER_DEPRECATED);
        $this->assertTrue(true);
    }

    public function testSecond(): void
    {
        $this->assertTrue(true);
    }
}

Result:

PHPUnit 11.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.13
Configuration: phpunit.xml.dist
Time: 00:00.005, Memory: 8.00 MB

OK (1 test, 1 assertion)

Process finished with exit code 0

Expected behavior

Result:

PHPUnit 11.5.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.13
Configuration: phpunit.xml.dist
Time: 00:00.006, Memory: 10.00 MB

1 test triggered 1 deprecation:

1) test/unit/Test.php:15
test

OK, but there were issues!
Tests: 1, Assertions: 1, Deprecations: 1.

Process finished with exit code 1

Or alternatively: Don't stop when it's suppressed.

@DannyMeyer DannyMeyer added the type/bug Something is broken label Jan 10, 2025
@sebastianbergmann sebastianbergmann added the feature/test-runner CLI test runner label Jan 10, 2025
@sebastianbergmann sebastianbergmann self-assigned this Jan 10, 2025
@sebastianbergmann sebastianbergmann added the version/11 Something affects PHPUnit 11 label Jan 10, 2025
@sebastianbergmann sebastianbergmann changed the title Tests stopped silently on suppressed deprecation Suppressed deprecations incorrectly stop test execution when execution should be stopped on deprecation Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/test-runner CLI test runner type/bug Something is broken version/11 Something affects PHPUnit 11
Projects
None yet
Development

No branches or pull requests

2 participants