4.0 | Proposal: introduce class constants to (eventually) replace the static Tokens arrays

[jrfnl]

Motivation

Sniffs should always be able to rely on the token arrays declared as static properties in the Tokens to be stable.

These token arrays should never be changed/updated by sniffs.

However, as they are declared as static properties, there is a risk that creative sniff writers will update the token arrays, thereby breaking sniffs from other standards, including PHPCS native sniffs.

Bug reports caused by this kind of creative sniff writing will be hard to debug and waste the time of maintainers.

By changing the token arrays from static properties to class constants, this risk is mitigated.

Note: in case anyone is wondering - no, this change can not be made in the 3.x branch, as declaring class constants with an array value is not supported until PHP 5.6 and for the 3.x branch, the minimum PHP version is still 5.4.

Proposal in detail

Original	Will become:
Tokens::$weightings	Tokens::WEIGHTINGS [*]
Tokens::$assignmentTokens	Tokens::ASSIGNMENT_TOKENS
Tokens::$equalityTokens	Tokens::EQUALITY_TOKENS
Tokens::$comparisonTokens	Tokens::COMPARISON_TOKENS
Tokens::$arithmeticTokens	Tokens::ARITHMETIC_TOKENS
Tokens::$operators	Tokens::OPERATORS
Tokens::$booleanOperators	Tokens::BOOLEAN_OPERATORS
Tokens::$castTokens	Tokens::CAST_TOKENS
Tokens::$parenthesisOpeners	Tokens::PARENTHESIS_OPENERS
Tokens::$scopeOpeners	Tokens::SCOPE_OPENERS
Tokens::$scopeModifiers	Tokens::SCOPE_MODIFIERS
Tokens::$methodPrefixes	Tokens::METHOD_MODIFIERS
Tokens::$blockOpeners	Tokens::BLOCK_OPENERS
Tokens::$emptyTokens	Tokens::EMPTY_TOKENS
Tokens::$commentTokens	Tokens::COMMENT_TOKENS
Tokens::$phpcsCommentTokens	Tokens::PHPCS_ANNOTATION_TOKENS
Tokens::$stringTokens	Tokens::STRING_TOKENS
Tokens::$textStringTokens	Tokens::TEXT_STRINGS
Tokens::$bracketTokens	Tokens::BRACKET_TOKENS
Tokens::$includeTokens	Tokens::INCLUDE_TOKENS
Tokens::$heredocTokens	Tokens::HEREDOC_TOKENS
Tokens::$functionNameTokens	Tokens::FUNCTION_NAME_TOKENS
Tokens::$ooScopeTokens	Tokens::OO_SCOPE_TOKENS
Tokens::$magicConstants	Tokens::MAGIC_CONSTANTS
Tokens::$contextSensitiveKeywords	Tokens::CONTEXT_SENSITIVE_KEYWORDS

I also propose to make the Tokens::WEIGHTINGS constant private as it is only supposed to be used via the Tokens::getHighestWeightedToken() method.

Planning

As this is a breaking change which will impact a significant number of sniffs, the static properties will remain available during the complete 4.x cycle and will be soft deprecated only (docblock annotation only).

This will allow for standards to support both PHPCS 3.x as well as 4.x, until they are ready to drop support for 3.x.

If/when issue #30 has been addressed, the soft deprecation could be changed to a hard deprecation towards the end of the 4.x cycle.

The static token array properties will be removed in the 5.0 release.

As the constants would already be available in the 4.x releases, this would, again, allow standards to support both PHPCS 4.x as well as 5.x, until they are ready to drop support for 4.x.

Open questions

• Any objections ?
• Any feedback on the proposed name changes ?
• Does the time line sound workable ?

/cc @asispts @dingo-d @fredden @GaryJones @greg0ire @kukulich @michalbundyra @Ocramius @sirbrillig @stronk7 @weierophinney @wimg

[fredden]

This change makes sense to me. 👍

[sirbrillig]

Sounds like a good idea and the deprecation plan makes sense.

as declaring class constants with an array value is not supported until PHP 5.6 and for the 3.x branch, the minimum PHP version is still 5.4.

Depending on how much you want to keep this support (I think not very much), you could use function calls as the "constants" instead or in addition. eg: Tokens::$weightings could be something like Tokens::getWeightings() which provides immutability without needing any particular version of PHP. Anyway, just a suggestion; constants are great.

[jrfnl]

Depending on how much you want to keep this support (I think not very much), you could use function calls as the "constants" instead or in addition. eg: Tokens::$weightings could be something like Tokens::getWeightings() which provides immutability without needing any particular version of PHP. Anyway, just a suggestion; constants are great.

@sirbrillig Support for PHP < 7.2 will be dropped in 4.0 and I'm currently working towards putting all the prelims in place to allow for making the 4.0 release stable and laying the foundations for the future. Having said that, I really would like to get it to a release sooner rather than later, but it will probably not be before end of the summer/mid autumn.
My availability for PHPCS related work over the next six weeks is going to be spotty at best, due to conferences, so I don't expect to be able to properly get back to it until beginning of July.