New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make it easier to use RuboCop with Prism #12600
Comments
Seems like a good idea! The only strange thing for me is the |
Yeah basically I have to monkey-patch a bunch of stuff to get rubocop to recognize that prism is another possible parser. This is the file that does that: https://github.com/kddnewton/parser-prism/blob/main/lib/parser/prism/rubocop.rb The easiest solution (if you're up for it!) would be to allow a string to be passed to In the very, very long term I would love to have a conversation with you all about the feasibility of making it the main parser. Trust me I know how much of a massive overhaul this would be, and I recognize the amount of work that would go into it. I just wanted to voice that I would be interested in exploring that kind of effort, if you were at all open to it. One small way to explore this would be trying to get just one rubocop cop running directly on the prism parser. I was told 3 years ago when I pitched prism that it would take decades and probably never happen, so you never know! |
I'm definitely open to the idea, although this is something we'll have to consider very carefully - e.g. are there features of
Yeah, we can certainly do some experiments in this direction. That's how I was evaluating the feasibility of |
4x as fast sounds almost too good to be true. That would be an insane improvement indeed. |
Prism's remarkable achievement. In fact, I've been experimenting with parser-prism locally several times. Here's the short to medium-term story first. Multi-Version SupportCurrently lacking, but planned for future support, is multi-version capability. I learned about this in last year's Ruby Prize 2023 award talk (congratulations on the award!). Since RuboCop can distinguish between runtime Ruby version and the analysis Ruby version, multi-version support is important. For example, it's possible to analyze Ruby 3.3 source code with Ruby 3.0 runtime, and vice versa. This is a requirement for RuboCop. parser-prismIn terms of implementation, I think parser-prism is a very good adapter as it allows a smooth transition to Prism while retaining the Parser interface. I haven't dug deep, but it might not pass all of RuboCop's tests at the moment, so this will need to be addressed. Ruby Runtime SupportThis is a note for a separate consideration for RuboCop. Prism requires Ruby 3.0.0 or higher. Currently, RuboCop also includes Ruby 2.7 in its runtime, so if Ruby 2.7 is still running, a fallback to the Parser gem may be necessary. Moreover, since Rails 7.1 supports Ruby 2.7, dropping Ruby 2.7 support might be premature. Universal ParserThis is a longer-term perspective, different from the above, as a Universal parser. Specifically, it relates to the parser generator Lrama, which replaced Bison in Ruby 3.3. Whether Lrama, which generates the Ruby parser based on parse.y, or Prism will become the Universal Parser, has not yet been decided by Matz. With Ruby 3.4 (or Ruby 3.5?), the relationship between Lrama and Prism as a Universal Parser will likely progress further, so until then, it might be better to keep the Parser gem as the default and make Prism an alternative approach of some sort. In any case, Ruby parser community is very active, and as a parser user, I'm very excited about these developments! |
Yeah, I completely agree. I was mostly thinking of just simplifying the usage of |
@koic To address a couple of your points:
|
The main use case is when you have a library that's targeting an older version of Ruby than the current runtime (e.g. you're running Ruby 3.3, but the library that you're working on requires Ruby 2.7+ and you need to lint the code with the Ruby 2.7 parser, as you won't be using newer syntactic features anyways) - that spares you from having to install RuboCop once per runtime and also allows users stuck to old Ruby versions to still have access to the latest RuboCop. Regarding syntax being accretive only - I think a couple of times some niche syntax got removed, but I'll have to double check this. But I'm certain that the meaning of some syntax changed between Ruby versions and thinks that used to be parsed in one way had to be parsed differently in later Ruby versions. |
I've obtained a direction on this. Let's keep the Parser gem as the default and make Prism an experimental option first. Including Prism as a parser option could enhance developer experience, I believe! |
I've just released v0.22.0, which adds support for Ruby 2.7 (running on 2.7, not parsing as 2.7). Hopefully that should make adoption easier. w/r/t multi-versioning, we have an option now to pass in |
I was just thinking about something similar regarding the multi-version support configuration for both existing Parser gem and Prism. The multi-version support API of Prism (i.e., This would allow for the following configuration in .rubocop.yml for parsing Ruby 3.3 with Prism: TargetRubyVersion: 3.3
ParserEngine: prism
TargetRubyVersion: 2.7
ParserEngine: parser This seems to be the simplest for users at the moment. My PoC with @kddnewton Do you have a Prism release plan, which contains |
@koic I like your idea! |
I have an issue open for it here: ruby/prism#2356. But haven't gotten to it yet. |
## Summary I'm working on integrating Prism into RuboCop. This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named in accordance with the following comments. rubocop/rubocop#12600 (comment) Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode. This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified. Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3, the expected name for an `lvar` is `:it`, not `:"0it"`. ### Expected AST The following is an expected AST when parsing Ruby 3.3 code: ```console $ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:send, nil, :it), :do_something)) ``` ### Actual AST The following is an actual AST when parsing Ruby 3.3 code: ```console $ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:lvar, :"0it"), :do_something)) ``` `Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively. And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future, but the behavior will be maintained until RuboCop's support is finalized: rubocop/rubocop#12600 (comment) ## Additional Information A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism. For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`. `Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem. Therefore, logic to maintain compatibility between Parser and Prism is implemented.
## Summary I'm working on integrating Prism into RuboCop. This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named in accordance with the following comments. rubocop/rubocop#12600 (comment) Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode. This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified. Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3, the expected name for an `lvar` is `:it`, not `:"0it"`. ### Expected AST The following is an expected AST when parsing Ruby 3.3 code: ```console $ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:send, nil, :it), :do_something)) ``` ### Actual AST The following is an actual AST when parsing Ruby 3.3 code: ```console $ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:lvar, :"0it"), :do_something)) ``` `Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively. And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future, but the behavior will be maintained until RuboCop's support is finalized: rubocop/rubocop#12600 (comment) ## Additional Information A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism. For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`. `Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem. Therefore, logic to maintain compatibility between Parser and Prism is implemented.
## Summary Fixes ruby#2356. I'm working on integrating Prism into RuboCop. This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named in accordance with the following comments. rubocop/rubocop#12600 (comment) Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode. This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified. Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3, the expected name for an `lvar` is `:it`, not `:"0it"`. ### Expected AST The following is an expected AST when parsing Ruby 3.3 code: ```console $ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:send, nil, :it), :do_something)) ``` ### Actual AST The following is an actual AST when parsing Ruby 3.3 code: ```console $ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:lvar, :"0it"), :do_something)) ``` `Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively. And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future, but the behavior will be maintained until RuboCop's support is finalized: rubocop/rubocop#12600 (comment) ## Additional Information A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism. For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`. `Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem. Therefore, logic to maintain compatibility between Parser and Prism is implemented.
Thank you for your sharing information. I've opened ruby/prism#2419 to support multi-versioning for |
## Summary Fixes ruby#2356. I'm working on integrating Prism into RuboCop. This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named in accordance with the following comments. rubocop/rubocop#12600 (comment) Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode. This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified. Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3, the expected name for an `lvar` is `:it`, not `:"0it"`. ### Expected AST The following is an expected AST when parsing Ruby 3.3 code: ```console $ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:send, nil, :it), :do_something)) ``` ### Actual AST The following is an actual AST when parsing Ruby 3.3 code: ```console $ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:lvar, :"0it"), :do_something)) ``` `Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively. And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future, but the behavior will be maintained until RuboCop's support is finalized: rubocop/rubocop#12600 (comment) ## Additional Information A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism. For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`. `Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem. Therefore, logic to maintain compatibility between Parser and Prism is implemented.
## Summary Fixes ruby#2356. I'm working on integrating Prism into RuboCop. This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named in accordance with the following comments. rubocop/rubocop#12600 (comment) Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode. This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified. Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3, the expected name for an `lvar` is `:it`, not `:"0it"`. ### Expected AST The following is an expected AST when parsing Ruby 3.3 code: ```console $ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:send, nil, :it), :do_something)) ``` ### Actual AST The following is an actual AST when parsing Ruby 3.3 code: ```console $ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:lvar, :"0it"), :do_something)) ``` `Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively. And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future, but the behavior will be maintained until RuboCop's support is finalized: rubocop/rubocop#12600 (comment) ## Additional Information A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism. For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`. `Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem. Therefore, logic to maintain compatibility between Parser and Prism is implemented.
## Summary Fixes ruby/prism#2356. I'm working on integrating Prism into RuboCop. This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named in accordance with the following comments. rubocop/rubocop#12600 (comment) Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode. This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified. Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3, the expected name for an `lvar` is `:it`, not `:"0it"`. ### Expected AST The following is an expected AST when parsing Ruby 3.3 code: ```console $ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision ruby/prism@5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:send, nil, :it), :do_something)) ``` ### Actual AST The following is an actual AST when parsing Ruby 3.3 code: ```console $ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')" ruby 3.3.0 (2023-12-25 revision ruby/prism@5124f9ac75) [x86_64-darwin22] s(:block, s(:send, s(:send, nil, :items), :map), s(:args), s(:send, s(:lvar, :"0it"), :do_something)) ``` `Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively. And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future, but the behavior will be maintained until RuboCop's support is finalized: rubocop/rubocop#12600 (comment) ## Additional Information A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism. For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`. `Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem. Therefore, logic to maintain compatibility between Parser and Prism is implemented. ruby/prism@62d3991e22
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism, as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600. ## Configuration By default, analysis is performed using the Parser gem, so the default value for the newly added `parser_engine` is `parser`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser) ``` This code maintains compatibility, meaning the traditional behavior is preserved: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file) ``` To perform analysis using Prism, specify `parser_engine: :parser_prism`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism) ``` The parameter name `parser_prism` reflects the original parser_prism which was the basis for `Prism::Translation::Parser` (now integrated into Prism): https://github.com/kddnewton/parser-prism This is an experimental introduction, and some incompatibilities still remain. > [!NOTE] > As initially mentioned in rubocop/rubocop#12600 (comment), > the plan was to set `parser_engine: prism`. > > However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`: > ruby/prism#2419 > > `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility. > So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future, > it has been decided that it might be better not to use `ParserEngine: prism` for the time being. > `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`. > > Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for > `Prism::Translation::Parser`. > > This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`, > not `ParserEngine: prism`. ## Compatibility The compatibility issues between Prism and the Parser gem have not been resolved. The failing tests will be skipped with `broken_on: :prism`: - ruby/prism#2454 has been resolved but not yet released. - ruby/prism#2467 is still unresolved. Issues that will be resolved in several upcoming releases of Prism are being skipped with `broken_on: :prism`. Anyway, RuboCop AST can be released independently of the resolution and release of Prism. > [!NOTE] > The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed: > https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb > > If the above interface is accepted, a fix will be proposed on the Prism side. ## Test Tests for RuboCop AST with Prism as the backend can be run as follows: ```console bundle exec rake prism_spec ``` The above is the shortcut alias for: ```console PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec ``` RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+, `internal_investigation` Rake task will not be executed. This task is only run with the Parser gem, which can analyze Ruby versions 2.0+.
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism, as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600. ## Configuration By default, analysis is performed using the Parser gem, so the default value for the newly added `parser_engine` is `parser`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser) ``` This code maintains compatibility, meaning the traditional behavior is preserved: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file) ``` To perform analysis using Prism, specify `parser_engine: :parser_prism`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism) ``` The parameter name `parser_prism` reflects the original parser_prism which was the basis for `Prism::Translation::Parser` (now integrated into Prism): https://github.com/kddnewton/parser-prism This is an experimental introduction, and some incompatibilities still remain. > [!NOTE] > As initially mentioned in rubocop/rubocop#12600 (comment), > the plan was to set `parser_engine: prism`. > > However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`: > ruby/prism#2419 > > `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility. > So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future, > it has been decided that it might be better not to use `ParserEngine: prism` for the time being. > `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`. > > Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for > `Prism::Translation::Parser`. > > This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`, > not `ParserEngine: prism`. ## Compatibility The compatibility issues between Prism and the Parser gem have not been resolved. The failing tests will be skipped with `broken_on: :prism`: - ruby/prism#2454 has been resolved but not yet released. - ruby/prism#2467 is still unresolved. Issues that will be resolved in several upcoming releases of Prism are being skipped with `broken_on: :prism`. Anyway, RuboCop AST can be released independently of the resolution and release of Prism. > [!NOTE] > The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed: > https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb > > If the above interface is accepted, a fix will be proposed on the Prism side. ## Test Tests for RuboCop AST with Prism as the backend can be run as follows: ```console bundle exec rake prism_spec ``` The above is the shortcut alias for: ```console PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec ``` RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+, `internal_investigation` Rake task will not be executed. This task is only run with the Parser gem, which can analyze Ruby versions 2.0+.
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism, as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600. ## Configuration By default, analysis is performed using the Parser gem, so the default value for the newly added `parser_engine` is `parser`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser) ``` This code maintains compatibility, meaning the traditional behavior is preserved: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file) ``` To perform analysis using Prism, specify `parser_engine: :parser_prism`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism) ``` The parameter name `parser_prism` reflects the original parser_prism which was the basis for `Prism::Translation::Parser` (now integrated into Prism): https://github.com/kddnewton/parser-prism This is an experimental introduction, and some incompatibilities still remain. > [!NOTE] > As initially mentioned in rubocop/rubocop#12600 (comment), > the plan was to set `parser_engine: prism`. > > However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`: > ruby/prism#2419 > > `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility. > So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future, > it has been decided that it might be better not to use `ParserEngine: prism` for the time being. > `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`. > > Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for > `Prism::Translation::Parser`. > > This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`, > not `ParserEngine: prism`. ## Compatibility The compatibility issues between Prism and the Parser gem have not been resolved. The failing tests will be skipped with `broken_on: :prism`: - ruby/prism#2454 has been resolved but not yet released. - ruby/prism#2467 is still unresolved. Issues that will be resolved in several upcoming releases of Prism are being skipped with `broken_on: :prism`. Anyway, RuboCop AST can be released independently of the resolution and release of Prism. > [!NOTE] > The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed: > https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb > > If the above interface is accepted, a fix will be proposed on the Prism side. ## Test Tests for RuboCop AST with Prism as the backend can be run as follows: ```console bundle exec rake prism_spec ``` The above is the shortcut alias for: ```console PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec ``` RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+, `internal_investigation` Rake task will not be executed. This task is only run with the Parser gem, which can analyze Ruby versions 2.0+.
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism, as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600. ## Configuration By default, analysis is performed using the Parser gem, so the default value for the newly added `parser_engine` is `parser_whitequark`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_whitequark) ``` This code maintains compatibility, meaning the traditional behavior is preserved: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file) ``` To perform analysis using Prism, specify `parser_engine: :parser_prism`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism) ``` The parameter name `parser_prism` reflects the original parser_prism which was the basis for `Prism::Translation::Parser` (now integrated into Prism): https://github.com/kddnewton/parser-prism This is an experimental introduction, and some incompatibilities still remain. > [!NOTE] > As initially mentioned in rubocop/rubocop#12600 (comment), > the plan was to set `parser_engine: prism`. > > However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`: > ruby/prism#2419 > > `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility. > So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future, > it has been decided that it might be better not to use `ParserEngine: prism` for the time being. > `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`. > > Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for > `Prism::Translation::Parser`. > > This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`, > not `ParserEngine: prism`. ## Compatibility The compatibility issues between Prism and the Parser gem have not been resolved. The failing tests will be skipped with `broken_on: :prism`: - ruby/prism#2454 has been resolved but not yet released. - ruby/prism#2467 is still unresolved. Issues that will be resolved in several upcoming releases of Prism are being skipped with `broken_on: :prism`. Anyway, RuboCop AST can be released independently of the resolution and release of Prism. > [!NOTE] > The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed: > https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb > > If the above interface is accepted, a fix will be proposed on the Prism side. ## Test Tests for RuboCop AST with Prism as the backend can be run as follows: ```console bundle exec rake prism_spec ``` The above is the shortcut alias for: ```console PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec ``` RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+, `internal_investigation` Rake task will not be executed. This task is only run with the Parser gem, which can analyze Ruby versions 2.0+.
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism, as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600. ## Configuration By default, analysis is performed using the Parser gem, so the default value for the newly added `parser_engine` is `parser_whitequark`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_whitequark) ``` This code maintains compatibility, meaning the traditional behavior is preserved: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file) ``` To perform analysis using Prism, specify `parser_engine: :parser_prism`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism) ``` The parameter name `parser_prism` reflects the original parser_prism which was the basis for `Prism::Translation::Parser` (now integrated into Prism): https://github.com/kddnewton/parser-prism This is an experimental introduction, and some incompatibilities still remain. > [!NOTE] > As initially mentioned in rubocop/rubocop#12600 (comment), > the plan was to set `parser_engine: prism`. > > However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`: > ruby/prism#2419 > > `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility. > So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future, > it has been decided that it might be better not to use `ParserEngine: prism` for the time being. > `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`. > > Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for > `Prism::Translation::Parser`. > > This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`, > not `ParserEngine: prism`. ## Compatibility The compatibility issues between Prism and the Parser gem have not been resolved. The failing tests will be skipped with `broken_on: :prism`: - ruby/prism#2454 has been resolved but not yet released. - ruby/prism#2467 is still unresolved. Issues that will be resolved in several upcoming releases of Prism are being skipped with `broken_on: :prism`. Anyway, RuboCop AST can be released independently of the resolution and release of Prism. > [!NOTE] > The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed: > https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb > > If the above interface is accepted, a fix will be proposed on the Prism side. ## Test Tests for RuboCop AST with Prism as the backend can be run as follows: ```console bundle exec rake prism_spec ``` The above is the shortcut alias for: ```console PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec ``` RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+, `internal_investigation` Rake task will not be executed. This task is only run with the Parser gem, which can analyze Ruby versions 2.0+.
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism, as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600. ## Configuration By default, analysis is performed using the Parser gem, so the default value for the newly added `parser_engine` is `parser_whitequark`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_whitequark) ``` This code maintains compatibility, meaning the traditional behavior is preserved: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file) ``` To perform analysis using Prism, specify `parser_engine: :parser_prism`: ```ruby ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism) ``` The parameter name `parser_prism` reflects the original parser_prism which was the basis for `Prism::Translation::Parser` (now integrated into Prism): https://github.com/kddnewton/parser-prism This is an experimental introduction, and some incompatibilities still remain. > [!NOTE] > As initially mentioned in rubocop/rubocop#12600 (comment), > the plan was to set `parser_engine: prism`. > > However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`: > ruby/prism#2419 > > `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility. > So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future, > it has been decided that it might be better not to use `ParserEngine: prism` for the time being. > `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`. > > Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for > `Prism::Translation::Parser`. > > This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`, > not `ParserEngine: prism`. ## Compatibility The compatibility issues between Prism and the Parser gem have not been resolved. The failing tests will be skipped with `broken_on: :prism`: - ruby/prism#2454 has been resolved but not yet released. - ruby/prism#2467 is still unresolved. Issues that will be resolved in several upcoming releases of Prism are being skipped with `broken_on: :prism`. Anyway, RuboCop AST can be released independently of the resolution and release of Prism. > [!NOTE] > The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed: > https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb > > If the above interface is accepted, a fix will be proposed on the Prism side. ## Test Tests for RuboCop AST with Prism as the backend can be run as follows: ```console bundle exec rake prism_spec ``` The above is the shortcut alias for: ```console PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec ``` RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+, `internal_investigation` Rake task will not be executed. This task is only run with the Parser gem, which can analyze Ruby versions 2.0+.
This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `Internal_affairs/location_line_equality_comparison` - `Layout/class_structure` - `Layout/empty_lines` - `Layout/end_of_line` - `Layout/heredoc_indentation` - `Layout/indentation_style` - `Layout/line_length` - `Layout/space_around_operators` - `Layout/space_inside_hash_literal_braces` - `Layout/trailing_whitespace` - `Lint/ambiguous_operator` - `Lint/ambiguous_regexp_literal` - `Lint/circular_argument_reference` - `Lint/deprecated_class_methods` - `Lint/deprecated_constants` - `Lint/erb_new_arguments` - `Lint/non_deterministic_require_order` - `Lint/numbered_parameter_assignment` - `Lint/parentheses_as_grouped_expression` - `Lint/redundant_dir_glob_sort` - `Lint/redundant_require_statement` - `Lint/refinement_import_methods` - `Lint/require_parentheses` - `Lint/syntax` - `Lint/unified_integer` - `Lint/useless_else_without_rescue` - `Naming/block_forwarding` - `Naming/heredoc_delimiter_case` - `Naming/heredoc_delimiter_naming` - `Naming/variable_number` - `Security/yaml_load` - `Style/arguments_forwarding` - `Style/array_intersect` - `Style/block_delimiters` - `Style/collection_compact` - `Style/command_literal` - `Style/comparable_clamp` - `Style/conditional_assignment_assign_in_condition` - `Style/conditional_assignment_assign_to_condition` - `Style/data_inheritance` - `Style/dir_empty` - `Style/file_empty` - `Style/frozen_string_literal_comment` - `Style/guard_clause` - `Style/hash_except` - `Style/hash_syntax` - `Style/hash_transform_keys` - `Style/hash_transform_values` - `Style/if_with_boolean_literal_branches` - `Style/multiline_ternary_operator` - `Style/multiline_when_then` - `Style/mutable_constant` - `Style/nested_file_dirname` - `Style/numeric_literals` - `Style/numeric_predicate` - `Style/object_then` - `Style/quoted_symbols` - `Style/redundant_begin` - `Style/redundant_freeze` - `Style/redundant_heredoc_delimiter_quotes` - `Style/redundant_parentheses` - `Style/rescue_modifier` - `Style/safe_navigation` - `Style/select_by_regexp` - `Style/single_line_methods` - `Style/slicing_with_range` - `Style/string_literals` - `Style/yaml_file_read` - `Style/yoda_condition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `InternalAffairs/LocationLineEqualityComparison` - `Layout/ClassStructure` - `Layout/EmptyLines` - `Layout/EndOfLine` - `Layout/HeredocIndentation` - `Layout/IndentationStyle` - `Layout/LineLength` - `Layout/SpaceAroundKeyword` - `Layout/SpaceAroundOperators` - `Layout/SpaceInsideHashLiteralBraces` - `Layout/TrailingWhitespace` - `Lint/AmbiguousOperator` - `Lint/AmbiguousRegexpLiteral` - `Lint/CircularArgumentReference` - `Lint/DeprecatedClassMethods` - `Lint/DeprecatedConstants` - `Lint/ErbNewArguments` - `Lint/NonDeterministicRequireOrder` - `Lint/NumberedParameterAssignment` - `Lint/ParenthesesAsGroupedExpression` - `Lint/RedundantDirGlobSort` - `Lint/RedundantRequireStatement` - `Lint/RefinementImportMethods` - `Lint/RequireParentheses` - `Lint/Syntax` - `Lint/UnifiedInteger` - `Lint/UselessElseWithoutRescue` - `Naming/BlockForwarding` - `Naming/HeredocDelimiterCase` - `Naming/HeredocDelimiterNaming` - `Naming/VariableNumber` - `Security/YamlLoad` - `Style/ArgumentsForwarding` - `Style/ArrayIntersect` - `Style/BlockDelimiters` - `Style/CollectionCompact` - `Style/CommandLiteral` - `Style/ComparableClamp` - `Style/ConditionalAssignmentAssignInCondition` - `Style/ConditionalAssignmentAssignToCondition` - `Style/DataInheritance` - `Style/DirEmpty` - `Style/FileEmpty` - `Style/FrozenStringLiteralComment` - `Style/GuardClause` - `Style/HashExcept` - `Style/HashSyntax` - `Style/HashTransformKeys` - `Style/HashTransformValues` - `Style/IfWithBooleanLiteralBranches` - `Style/MultilineTernaryOperator` - `Style/MultilineWhenThen` - `Style/MutableConstant` - `Style/NestedFileDirname` - `Style/NumericLiterals` - `Style/NumericPredicate` - `Style/ObjectThen` - `Style/QuotedSymbols` - `Style/RedundantBegin` - `Style/RedundantFreeze` - `Style/RedundantHeredocDelimiterQuotes` - `Style/RedundantParentheses` - `Style/RescueModifier` - `Style/SafeNavigation` - `Style/SelectByRegexp` - `Style/SingleLineMethods` - `Style/SlicingWithRange` - `Style/StringLiterals` - `Style/TernaryParentheses` - `Style/YamlFileRead` - `Style/YodaCondition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves rubocop#12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. > [!CAUTION] > Since the use of Prism is experimental, it is not included in RuboCop's runtime dependencies. > If running through Bundler, please first add `gem 'prism'` to your Gemfile: > > ```ruby > gem 'prism' > ``` There are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `InternalAffairs/LocationLineEqualityComparison` - `Layout/ClassStructure` - `Layout/EmptyLines` - `Layout/EndOfLine` - `Layout/HeredocIndentation` - `Layout/IndentationStyle` - `Layout/LineLength` - `Layout/SpaceAroundKeyword` - `Layout/SpaceAroundOperators` - `Layout/SpaceInsideHashLiteralBraces` - `Layout/TrailingWhitespace` - `Lint/AmbiguousOperator` - `Lint/AmbiguousRegexpLiteral` - `Lint/CircularArgumentReference` - `Lint/DeprecatedClassMethods` - `Lint/DeprecatedConstants` - `Lint/ErbNewArguments` - `Lint/NonDeterministicRequireOrder` - `Lint/NumberedParameterAssignment` - `Lint/ParenthesesAsGroupedExpression` - `Lint/RedundantDirGlobSort` - `Lint/RedundantRequireStatement` - `Lint/RefinementImportMethods` - `Lint/RequireParentheses` - `Lint/Syntax` - `Lint/UnifiedInteger` - `Lint/UselessElseWithoutRescue` - `Naming/BlockForwarding` - `Naming/HeredocDelimiterCase` - `Naming/HeredocDelimiterNaming` - `Naming/VariableNumber` - `Security/YamlLoad` - `Style/ArgumentsForwarding` - `Style/ArrayIntersect` - `Style/BlockDelimiters` - `Style/CollectionCompact` - `Style/CommandLiteral` - `Style/ComparableClamp` - `Style/ConditionalAssignmentAssignInCondition` - `Style/ConditionalAssignmentAssignToCondition` - `Style/DataInheritance` - `Style/DirEmpty` - `Style/FileEmpty` - `Style/FrozenStringLiteralComment` - `Style/GuardClause` - `Style/HashExcept` - `Style/HashSyntax` - `Style/HashTransformKeys` - `Style/HashTransformValues` - `Style/IfWithBooleanLiteralBranches` - `Style/MultilineTernaryOperator` - `Style/MultilineWhenThen` - `Style/MutableConstant` - `Style/NestedFileDirname` - `Style/NumericLiterals` - `Style/NumericPredicate` - `Style/ObjectThen` - `Style/QuotedSymbols` - `Style/RedundantBegin` - `Style/RedundantFreeze` - `Style/RedundantHeredocDelimiterQuotes` - `Style/RedundantParentheses` - `Style/RescueModifier` - `Style/SafeNavigation` - `Style/SelectByRegexp` - `Style/SingleLineMethods` - `Style/SlicingWithRange` - `Style/StringLiterals` - `Style/TernaryParentheses` - `Style/YamlFileRead` - `Style/YodaCondition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Resolves #12600. This is the initial basic implementation to support multiple parser engines. It is still in an experimental phase. There are still incompatibilities with Prism compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support for Prism independently within the RuboCop core. > [!IMPORTANT] > To work this feature, the following patch for RuboCop AST needs to be released: > rubocop/rubocop-ast#277 ## Setting the parser engine RuboCop allows switching the backend parser by specifying either `parser_whitequark` or `parser_prism` for the `ParserEngine`. Here are the parsers used as backends for each value: - `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser - `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`) By default, `parser_whitequark` is implicitly used. `parser_whitequark` can analyze source code from Ruby 2.0 and above: ```yaml AllCops: ParserEngine: parser_whitequark ``` `parser_prism` can analyze source code from Ruby 3.3 and above: ```yaml AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3 ``` `parser_prism` tends to perform analysis faster than `parser_whitequark`. > [!CAUTION] > Since the use of Prism is experimental, it is not included in RuboCop's runtime dependencies. > If running through Bundler, please first add `gem 'prism'` to your Gemfile: > > ```ruby > gem 'prism' > ``` There are some incompatibilities with `parser_whitequark`, making it still considered experimental. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ## Incompatibility At the time of opening this PR, the following cops have incompatibility with Prism: - `Gemspec/RequiredRubyVersion` - `InternalAffairs/LocationLineEqualityComparison` - `Layout/ClassStructure` - `Layout/EmptyLines` - `Layout/EndOfLine` - `Layout/HeredocIndentation` - `Layout/IndentationStyle` - `Layout/LineLength` - `Layout/SpaceAroundKeyword` - `Layout/SpaceAroundOperators` - `Layout/SpaceInsideHashLiteralBraces` - `Layout/TrailingWhitespace` - `Lint/AmbiguousOperator` - `Lint/AmbiguousRegexpLiteral` - `Lint/CircularArgumentReference` - `Lint/DeprecatedClassMethods` - `Lint/DeprecatedConstants` - `Lint/ErbNewArguments` - `Lint/NonDeterministicRequireOrder` - `Lint/NumberedParameterAssignment` - `Lint/ParenthesesAsGroupedExpression` - `Lint/RedundantDirGlobSort` - `Lint/RedundantRequireStatement` - `Lint/RefinementImportMethods` - `Lint/RequireParentheses` - `Lint/Syntax` - `Lint/UnifiedInteger` - `Lint/UselessElseWithoutRescue` - `Naming/BlockForwarding` - `Naming/HeredocDelimiterCase` - `Naming/HeredocDelimiterNaming` - `Naming/VariableNumber` - `Security/YamlLoad` - `Style/ArgumentsForwarding` - `Style/ArrayIntersect` - `Style/BlockDelimiters` - `Style/CollectionCompact` - `Style/CommandLiteral` - `Style/ComparableClamp` - `Style/ConditionalAssignmentAssignInCondition` - `Style/ConditionalAssignmentAssignToCondition` - `Style/DataInheritance` - `Style/DirEmpty` - `Style/FileEmpty` - `Style/FrozenStringLiteralComment` - `Style/GuardClause` - `Style/HashExcept` - `Style/HashSyntax` - `Style/HashTransformKeys` - `Style/HashTransformValues` - `Style/IfWithBooleanLiteralBranches` - `Style/MultilineTernaryOperator` - `Style/MultilineWhenThen` - `Style/MutableConstant` - `Style/NestedFileDirname` - `Style/NumericLiterals` - `Style/NumericPredicate` - `Style/ObjectThen` - `Style/QuotedSymbols` - `Style/RedundantBegin` - `Style/RedundantFreeze` - `Style/RedundantHeredocDelimiterQuotes` - `Style/RedundantParentheses` - `Style/RescueModifier` - `Style/SafeNavigation` - `Style/SelectByRegexp` - `Style/SingleLineMethods` - `Style/SlicingWithRange` - `Style/StringLiterals` - `Style/TernaryParentheses` - `Style/YamlFileRead` - `Style/YodaCondition` Some cop incompatibilities have been resolved in the Prism development line. For known issues, please refer to the Prism issues: https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop ### `Lint/Syntax` cop The messages generated by Lint/Syntax depend on the parser engine used. Parser gem: ```console $ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: unexpected token $end ``` Displays `unexpected token $end`. Prism: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] (string):1:2: error: expected a matching `)` ``` Displays `expected a matching )`. There are differences in the messages between Parser gem and Prism, but since Prism can provide clearer messages in some cases, this incompatibility is accepted. In other words, the messages may vary depending on the parser engine. ## Test To run tests with Prism, the command `bundle exec rake prism_spec` is provided. This task is also executed in GitHub Actions. To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`: ```console $ PARSER_ENGINE=parser_prism path/to/test_spec.rb ``` In the context of testing with Prism, two options for test cases are provided: `broken_on: :prism` and `unsupported_on: :prism`. Both options are utilized to skip tests specifically for Prism. ### `broken_on: :prism` Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`. This indicates an expectation for the issue to be resolved within Prism. ### `unsupported_on: :prism` Prism is designed to parse Ruby versions 3.3+, which means that features unique to Ruby versions 2.0 through 3.2 are not supported. Test cases falling into this category are marked with `unsupported_on: :prism`. This marker is used for cases that are testable with the Parser gem but not with Prism. > [!NOTE] > With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`. > The `ascii_spec` task has not been failing for a while, so it will not be run by default. > However, `ascii_spec` task will continue to be checked in CI. If there are any failures > originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
I've noticed that these days the new super fast Prism parser has a
parser
backend that seems to work with RuboCop (see https://github.com/kddnewton/parser-prism#rubocop), but requires a bit of manual work. I've also noticed one impressive benchmark that implies RuboCop runs 4 times faster with Prism!!!I think we should make it easier for users to use
Prism
- e.g. if it's present it should be used by default or something along those lines. (might also be some configuration option)@rubocop/rubocop-core Any thoughts/ideas on the subject? //cc @kddnewton
The text was updated successfully, but these errors were encountered: