Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Details about deprecations, notices, and warnings are not displayed when the TestDox result printer is used #5518

Closed
sebastianbergmann opened this issue Sep 20, 2023 · 1 comment
Assignees
Labels
feature/test-runner CLI test runner type/bug Something is broken

Comments

@sebastianbergmann
Copy link
Owner

Display deprecations, notices, and warnings in TestDox output when --display-deprecations (or displayDetailsOnTestsThatTriggerDeprecations="true"), --display-notices (or displayDetailsOnTestsThatTriggerNotices="true"), --display-warnings (or displayDetailsOnTestsThatTriggerWarnings="true") is used, respectively.

@sebastianbergmann
Copy link
Owner Author

sebastianbergmann commented Dec 30, 2023

PHPUnit 10 introduced a clear separation between the outcome (errored, failed, incomplete, skipped, or passed) of a test and the issues (considered risky, triggered a deprecation/notice/warning, etc.) of a test.

While a test does only have outcome, it may have >= 0 issues. I do not think that the information about issues can be displayed in TestDox format in a useful way. I therefore propose that we only consider test outcome for the TestDox format and display details about test issues by re-using the default result printer.

Consider the following example:

ExampleTest.php

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class ExampleTest extends TestCase
{
    public function testNoIssues(): void
    {
        $this->assertTrue(true);
    }

    public function testFailure(): void
    {
        $this->assertTrue(false);
    }

    public function testDeprecation(): void
    {
        trigger_error('deprecation', E_USER_DEPRECATED);

        $this->assertTrue(true);
    }
}

Current Behaviour

$ phpunit --display-deprecations --display-notices --display-warnings --testdox ExampleTest.php
PHPUnit 10.5.5-5-gd2a23b7cb9 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.1

.FD                                                                 3 / 3 (100%)

Time: 00:00.017, Memory: 4.00 MB

Example
 ✔ No issues
 ✘ Failure
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/ExampleTest.php:13
   │
 ✔ Deprecation

FAILURES!
Tests: 3, Assertions: 3, Failures: 1, Deprecations: 1.

Proposed Behaviour

$ phpunit --display-deprecations --display-notices --display-warnings --testdox ExampleTest.php
PHPUnit 10.5.5-5-gd2a23b7cb9 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.1

.FD                                                                 3 / 3 (100%)

Time: 00:00.017, Memory: 4.00 MB

Example
 ✔ No issues
 ✘ Failure
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/ExampleTest.php:13
   │
 ✔ Deprecation

1 test triggered 1 deprecation:

1) /home/sb/ExampleTest.php:18
deprecation

Triggered by:

* ExampleTest::testDeprecation
  /home/sb/ExampleTest.php:16

FAILURES!
Tests: 3, Assertions: 3, Failures: 1, Deprecations: 1.

I will implement the above ASAP for PHPUnit 10.5.

@sebastianbergmann sebastianbergmann removed this from the PHPUnit 11.0 milestone Dec 30, 2023
@sebastianbergmann sebastianbergmann added type/bug Something is broken and removed type/enhancement A new idea that should be implemented labels Dec 30, 2023
@sebastianbergmann sebastianbergmann changed the title Display deprecations, notices, and warnings in TestDox output Details about deprecations, notices, and warnings are not displayed when the TestDox result printer is used Dec 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/test-runner CLI test runner type/bug Something is broken
Projects
None yet
Development

No branches or pull requests

1 participant