Baseline for E_(USER_)DEPRECATED, E_(USER_)NOTICE, E_STRICT, and E_(USER_)WARNING

[sebastianbergmann]

PHPUnit's test runner registers an error handler and processes E_DEPRECATED, E_USER_DEPRECATED, E_NOTICE, E_USER_NOTICE, E_STRICT, E_WARNING, and E_USER_WARNING errors, for instance, that are triggered while a test is executed. This error handler emits events that are, for instance, subscribed to and used by output printers and loggers.

Since #5293, the reporting of deprecations, notices, and warnings can be limited to specified directories. Since #5328, there is an option to ignore the suppression of deprecations, notices, and warnings. Since #5428, the #[WithoutErrorHandler] attribute can be used to disable PHPUnit's error handler for a test method.

Following the example of static analysis tools such as Psalm and PHPStan, PHPUnit should support the concept of a baseline to

declare the currently reported list of errors as “the baseline” and cause it not being reported in subsequent runs. It allows you to be interested in [issues] only in new and changed code

• Implement baseline attribute on the <source> element of PHPUnit's XML configuration file to configure the path to the baseline file to be used
• Implement --generate-baseline CLI option to generate a baseline file (at the location configured in the XML configuration file, see above)
• Make paths in Issue objects relative to baseline.xml location
• Use the baseline (when it is configured) to not display information about E_(USER_)DEPRECATED, E_(USER_)NOTICE, E_STRICT, and E_(USER_)WARNING issues that are "baselined"
• Change --generate-baseline CLI option (see above) to require the specification of the path to where the baseline file should be written
• Implement --use-baseline <path> CLI option to specify the baseline file to be used
• Implement --ignore-baseline CLI option to ignore the baseline