[DX] Massive deprecations when upgrading to 10.5.18 and 10.5.20

[gnutix]

Q	A
PHPUnit version	10.5.20
PHP version	8.2.16
Installation Method	Composer

Summary

Hello there! Sorry if this is not really a bug, but I think it deserves a conversation at least.

I've just upgraded PHPUnit on our project from 10.5.15 to 10.5.20, and the CI reported broken tests. Well, not really ; it's "only" deprecations... 3252 of them. I've checked the release notes, and both 10.5.18 and 10.5.20 introduced new deprecations having massive impacts.

I'm wondering if those two are justified, inevitable, and if there's anything that could be done to help users migrate (like a Rector maybe) ? Or at the very least, is there an option that would allow PHPUnit to report a zero status code when there's no failure/errors but only deprecations ? (that may be related to our setup of Symfony/PHPUnit-bridge and so on...)

Curious of your thoughts and if I'm the only one in this scenario.
gnutix

[sebastianbergmann]

As of #5812 and since PHPUnit 10.5.18, PHPUnit 10.5 warns about tests that "only work by happenstance". These tests, that may already work incorrectly when run with PHPUnit 10.5, no longer work with PHPUnit 11 due to #4964.

PHPUnit 10.5.20 did not introduce any new deprecations, but I guess you are referring to #5819. This, too, is a warning about tests that, if at all, "only work by happenstance".

is there an option that would allow PHPUnit to report a zero status code when there's no failure/errors but only deprecations

What you ask for is the default behaviour. You need to explictly configure that deprecations lead to a non-zero exit code, for instance using the failOnDeprecation attribute in the XML configuration file.

[gnutix]

failOnDeprecation fixes the status code indeed. Is there a way to not report the thousands of deprecations at the end of each run too ?

Your answer and closing the issue right afterwards doesn't give me a feeling that you really care about the effort fixing these deprecations will require of the developers using your library. Maybe I misunderstand "happenstance", but if it means "by a happy coincidence", I beg to disagree : it worked because there never was any constraint on how data providers keys were to be named, and suddenly you've added such a constraint to fit the plans for the future version.

I'd be curious to hear of other developers if they had such impacts on their test suites, but closing the issue means no one will ever see it...

[Ma-ve]

@gnutix We've seen this too. While it's annoying, as it happened between minor versions, we're not really that worried.

Either:

• stick with 10.5.18, as PHPUnit 11 is already available
• remove all array keys from data providers
• update array keys to match the argument name

We've decided on half/half: removing array keys from data providers that weren't providing additional information, and for those with more complex structures (true/false/true/true), we've updated the array keys to match the argument names.

As for the 'happenstance', I wouldn't much call it that if it weren't for PHPUnit introducing support for named arguments somewhere along the way. PHPUnit doing->{$method}(...array_values($dataProviderData)) worked perfectly fine for many years

[mondrake]

@gnutix

Is there a way to not report the thousands of deprecations at the end of each run too ?

you may find #5841 interesting for this case.