Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easier to use RuboCop with Prism #12600

Closed
bbatsov opened this issue Jan 8, 2024 · 14 comments · Fixed by #12724
Closed

Make it easier to use RuboCop with Prism #12600

bbatsov opened this issue Jan 8, 2024 · 14 comments · Fixed by #12724
Assignees
Labels
enhancement high priority A ticket considered important by RuboCop's Core Team

Comments

@bbatsov
Copy link
Collaborator

bbatsov commented Jan 8, 2024

I've noticed that these days the new super fast Prism parser has a parser backend that seems to work with RuboCop (see https://github.com/kddnewton/parser-prism#rubocop), but requires a bit of manual work. I've also noticed one impressive benchmark that implies RuboCop runs 4 times faster with Prism!!!

I think we should make it easier for users to use Prism - e.g. if it's present it should be used by default or something along those lines. (might also be some configuration option)

@rubocop/rubocop-core Any thoughts/ideas on the subject? //cc @kddnewton

@bbatsov bbatsov added enhancement high priority A ticket considered important by RuboCop's Core Team labels Jan 8, 2024
@dvandersluis
Copy link
Member

Seems like a good idea! The only strange thing for me is the TargetRubyVersion overriding as that's not going to be very user-friendly.

@kddnewton
Copy link
Contributor

Yeah basically I have to monkey-patch a bunch of stuff to get rubocop to recognize that prism is another possible parser. This is the file that does that: https://github.com/kddnewton/parser-prism/blob/main/lib/parser/prism/rubocop.rb

The easiest solution (if you're up for it!) would be to allow a string to be passed to TargetRubyVersion like "prism" that would fall back to require "parser/#{target_ruby_version}". Other than that, anything that gets me out of having to write that horrible monkey-patch would be greatly appreciated.

In the very, very long term I would love to have a conversation with you all about the feasibility of making it the main parser. Trust me I know how much of a massive overhaul this would be, and I recognize the amount of work that would go into it. I just wanted to voice that I would be interested in exploring that kind of effort, if you were at all open to it. One small way to explore this would be trying to get just one rubocop cop running directly on the prism parser. I was told 3 years ago when I pitched prism that it would take decades and probably never happen, so you never know!

@bbatsov
Copy link
Collaborator Author

bbatsov commented Jan 8, 2024

In the very, very long term I would love to have a conversation with you all about the feasibility of making it the main parser. Trust me I know how much of a massive overhaul this would be, and I recognize the amount of work that would go into it. I just wanted to voice that I would be interested in exploring that kind of effort, if you were at all open to it.

I'm definitely open to the idea, although this is something we'll have to consider very carefully - e.g. are there features of parser that are missing or behave differently in prism, can such a migration be done gradually, etc. I did the rewrite from ripper to parser early in RuboCop's life, so I have some idea how we can approach this, but also back then RuboCop was much smaller than today. (I was updating the cops in batches and for a while there were both cops using ripper and parser)

One small way to explore this would be trying to get just one rubocop cop running directly on the prism parser. I was told 3 years ago when I pitched prism that it would take decades and probably never happen, so you never know!

Yeah, we can certainly do some experiments in this direction. That's how I was evaluating the feasibility of parser early on, btw.

@rubyFeedback
Copy link

4x as fast sounds almost too good to be true. That would be an insane improvement indeed.

@koic
Copy link
Member

koic commented Jan 9, 2024

Prism's remarkable achievement. In fact, I've been experimenting with parser-prism locally several times.
When it comes to integrating it into RuboCop, I think a medium to long-term perspective is necessary.

Here's the short to medium-term story first.

Multi-Version Support

Currently lacking, but planned for future support, is multi-version capability. I learned about this in last year's Ruby Prize 2023 award talk (congratulations on the award!).

Since RuboCop can distinguish between runtime Ruby version and the analysis Ruby version, multi-version support is important. For example, it's possible to analyze Ruby 3.3 source code with Ruby 3.0 runtime, and vice versa. This is a requirement for RuboCop.
https://docs.rubocop.org/rubocop/1.59/compatibility.html

parser-prism

In terms of implementation, I think parser-prism is a very good adapter as it allows a smooth transition to Prism while retaining the Parser interface. I haven't dug deep, but it might not pass all of RuboCop's tests at the moment, so this will need to be addressed.

Ruby Runtime Support

This is a note for a separate consideration for RuboCop.

Prism requires Ruby 3.0.0 or higher. Currently, RuboCop also includes Ruby 2.7 in its runtime, so if Ruby 2.7 is still running, a fallback to the Parser gem may be necessary.

Moreover, since Rails 7.1 supports Ruby 2.7, dropping Ruby 2.7 support might be premature.

Universal Parser

This is a longer-term perspective, different from the above, as a Universal parser. Specifically, it relates to the parser generator Lrama, which replaced Bison in Ruby 3.3. Whether Lrama, which generates the Ruby parser based on parse.y, or Prism will become the Universal Parser, has not yet been decided by Matz.

With Ruby 3.4 (or Ruby 3.5?), the relationship between Lrama and Prism as a Universal Parser will likely progress further, so until then, it might be better to keep the Parser gem as the default and make Prism an alternative approach of some sort.

In any case, Ruby parser community is very active, and as a parser user, I'm very excited about these developments!

@bbatsov
Copy link
Collaborator Author

bbatsov commented Jan 9, 2024

With Ruby 3.4 (or Ruby 3.5?), the relationship between Lrama and Prism as a Universal Parser will likely progress further, so until then, it might be better to keep the Parser gem as the default and make Prism an alternative approach of some sort.

Yeah, I completely agree. I was mostly thinking of just simplifying the usage of prism-parser for the people who want to use it, not rebasing RuboCop around it. The performance gains are quite impressive and if this works reasonably well, I can imagine some people wanting to use it for CI, even if they are stuck just to the runtime version (after all most people work on Rails app that target a single runtime version for them that'd be OK).

@kddnewton
Copy link
Contributor

@koic To address a couple of your points:

  • Multi-Version Support: As you said this is planned. I would love to understand the true requirement here for rubocop. Is the actual requirement "be able to parse as any version of Ruby" or could that be amended to "be able to parse as the most recent version of Ruby and/or the current version of Ruby running in this process"? If it's truly "be able to parse as any version of Ruby" I wonder if we could solve this at the library level and not the parser. Because syntax is never removed, we could potentially address this kind of thing by raising an error when more recent syntax than is allowed is found. Overall, I would love more information on this in particular.
  • Ruby Runtime Support: I don't imagine this will actually matter, because by the time we got this fully integrated and tested 2.7 would be EOL anyway 😅.
  • Universal Parser: This shouldn't factor in. Matz has said multiple times that the parser interface going forward will be the prism AST — regardless of the technology used to create it. So the prism interface is stable and you can build on it without worrying that it's going to change out from under you.

@bbatsov
Copy link
Collaborator Author

bbatsov commented Jan 9, 2024

Multi-Version Support: As you said this is planned. I would love to understand the true requirement here for rubocop. Is the actual requirement "be able to parse as any version of Ruby" or could that be amended to "be able to parse as the most recent version of Ruby and/or the current version of Ruby running in this process"? If it's truly "be able to parse as any version of Ruby" I wonder if we could solve this at the library level and not the parser. Because syntax is never removed, we could potentially address this kind of thing by raising an error when more recent syntax than is allowed is found. Overall, I would love more information on this in particular.

The main use case is when you have a library that's targeting an older version of Ruby than the current runtime (e.g. you're running Ruby 3.3, but the library that you're working on requires Ruby 2.7+ and you need to lint the code with the Ruby 2.7 parser, as you won't be using newer syntactic features anyways) - that spares you from having to install RuboCop once per runtime and also allows users stuck to old Ruby versions to still have access to the latest RuboCop.

Regarding syntax being accretive only - I think a couple of times some niche syntax got removed, but I'll have to double check this. But I'm certain that the meaning of some syntax changed between Ruby versions and thinks that used to be parsed in one way had to be parsed differently in later Ruby versions.

@koic
Copy link
Member

koic commented Jan 11, 2024

I've obtained a direction on this. Let's keep the Parser gem as the default and make Prism an experimental option first.
Due to differences in parsers, there might be a loss of full compatibility regarding multi-versioning, but it might be possible to proceed by creating a compromise while experimenting.
Since I'm not familiar with Prism, I will likely send feedback on parser-prism (or Prism) progressively.

Including Prism as a parser option could enhance developer experience, I believe!

@koic koic self-assigned this Feb 6, 2024
@kddnewton
Copy link
Contributor

I've just released v0.22.0, which adds support for Ruby 2.7 (running on 2.7, not parsing as 2.7). Hopefully that should make adoption easier.

w/r/t multi-versioning, we have an option now to pass in version: to Prism.parse which should help with this. I plan on creating more parsers so that you can have Prism::Translation::Parser33 and Prism::Translation::Parser34 and so on. We'll need a way of triggering that, so maybe the rubocop config shouldn't trigger prism based on version but instead a different option? That way you could keep TargetRubyVersion in place.

@koic
Copy link
Member

koic commented Feb 8, 2024

I was just thinking about something similar regarding the multi-version support configuration for both existing Parser gem and Prism.

The multi-version support API of Prism (i.e., Prism::Translation::Parser33 and Prism::Translation::Parser34) is great news and seems ideal for RuboCop!

This would allow for the following configuration in .rubocop.yml for parsing Ruby 3.3 with Prism:

TargetRubyVersion: 3.3
ParserEngine: prism

ParserEngine accepts parser (default) and prism as values. It can be omitted for ParserEngine: parser, maintaining compatibility with traditional RuboCop. To explicitly note the oldest RuboCop support (i.e., Parser::Ruby27), it would look like this:

TargetRubyVersion: 2.7
ParserEngine: parser

This seems to be the simplest for users at the moment.

My PoC with Prism::Translation::Parser appears to be working well so far. I'll think more about the design, but if there seem to be no issues, I will probably be able to open a PR soon.

@kddnewton Do you have a Prism release plan, which contains Prism::Translation::Parser33, Prism::Translation::Parser34 and so on?

@bbatsov
Copy link
Collaborator Author

bbatsov commented Feb 8, 2024

@koic I like your idea!

@kddnewton
Copy link
Contributor

I have an issue open for it here: ruby/prism#2356. But haven't gotten to it yet.

koic added a commit to koic/prism that referenced this issue Feb 15, 2024
## Summary

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.
koic added a commit to koic/prism that referenced this issue Feb 15, 2024
## Summary

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.
koic added a commit to koic/prism that referenced this issue Feb 15, 2024
## Summary

Fixes ruby#2356.

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.
@koic
Copy link
Member

koic commented Feb 15, 2024

Thank you for your sharing information. I've opened ruby/prism#2419 to support multi-versioning for Prism::Translation::Parser. This should at least resolve the issue of being locked into parsing with the current Ruby 3.4.

koic added a commit to koic/prism that referenced this issue Feb 15, 2024
## Summary

Fixes ruby#2356.

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.
koic added a commit to koic/prism that referenced this issue Feb 15, 2024
## Summary

Fixes ruby#2356.

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.
matzbot pushed a commit to ruby/ruby that referenced this issue Feb 15, 2024
## Summary

Fixes ruby/prism#2356.

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision ruby/prism@5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision ruby/prism@5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.

ruby/prism@62d3991e22
koic added a commit to koic/rubocop-ast that referenced this issue Feb 24, 2024
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism,
as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600.

## Configuration

By default, analysis is performed using the Parser gem, so the default value
for the newly added `parser_engine` is `parser`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser)
```

This code maintains compatibility, meaning the traditional behavior is preserved:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file)
```

To perform analysis using Prism, specify `parser_engine: :parser_prism`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism)
```

The parameter name `parser_prism` reflects the original parser_prism which was the basis for
`Prism::Translation::Parser` (now integrated into Prism):
https://github.com/kddnewton/parser-prism

This is an experimental introduction, and some incompatibilities still remain.

> [!NOTE]
> As initially mentioned in rubocop/rubocop#12600 (comment),
> the plan was to set `parser_engine: prism`.
>
> However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`:
> ruby/prism#2419
>
> `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility.
> So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future,
> it has been decided that it might be better not to use `ParserEngine: prism` for the time being.
> `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`.
>
> Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for
> `Prism::Translation::Parser`.
>
> This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`,
> not `ParserEngine: prism`.

## Compatibility

The compatibility issues between Prism and the Parser gem have not been resolved.
The failing tests will be skipped with `broken_on: :prism`:

- ruby/prism#2454 has been resolved but not yet released.
- ruby/prism#2467 is still unresolved.

Issues that will be resolved in several upcoming releases of Prism are being skipped
with `broken_on: :prism`.

Anyway, RuboCop AST can be released independently of the resolution and release of Prism.

> [!NOTE]
> The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed:
> https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb
>
> If the above interface is accepted, a fix will be proposed on the Prism side.

## Test

Tests for RuboCop AST with Prism as the backend can be run as follows:

```console
bundle exec rake prism_spec
```

The above is the shortcut alias for:

```console
PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec
```

RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+,
`internal_investigation` Rake task will not be executed. This task is only run with the Parser gem,
which can analyze Ruby versions 2.0+.
koic added a commit to koic/rubocop-ast that referenced this issue Feb 24, 2024
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism,
as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600.

## Configuration

By default, analysis is performed using the Parser gem, so the default value
for the newly added `parser_engine` is `parser`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser)
```

This code maintains compatibility, meaning the traditional behavior is preserved:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file)
```

To perform analysis using Prism, specify `parser_engine: :parser_prism`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism)
```

The parameter name `parser_prism` reflects the original parser_prism which was the basis for
`Prism::Translation::Parser` (now integrated into Prism):
https://github.com/kddnewton/parser-prism

This is an experimental introduction, and some incompatibilities still remain.

> [!NOTE]
> As initially mentioned in rubocop/rubocop#12600 (comment),
> the plan was to set `parser_engine: prism`.
>
> However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`:
> ruby/prism#2419
>
> `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility.
> So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future,
> it has been decided that it might be better not to use `ParserEngine: prism` for the time being.
> `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`.
>
> Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for
> `Prism::Translation::Parser`.
>
> This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`,
> not `ParserEngine: prism`.

## Compatibility

The compatibility issues between Prism and the Parser gem have not been resolved.
The failing tests will be skipped with `broken_on: :prism`:

- ruby/prism#2454 has been resolved but not yet released.
- ruby/prism#2467 is still unresolved.

Issues that will be resolved in several upcoming releases of Prism are being skipped
with `broken_on: :prism`.

Anyway, RuboCop AST can be released independently of the resolution and release of Prism.

> [!NOTE]
> The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed:
> https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb
>
> If the above interface is accepted, a fix will be proposed on the Prism side.

## Test

Tests for RuboCop AST with Prism as the backend can be run as follows:

```console
bundle exec rake prism_spec
```

The above is the shortcut alias for:

```console
PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec
```

RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+,
`internal_investigation` Rake task will not be executed. This task is only run with the Parser gem,
which can analyze Ruby versions 2.0+.
koic added a commit to koic/rubocop-ast that referenced this issue Feb 24, 2024
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism,
as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600.

## Configuration

By default, analysis is performed using the Parser gem, so the default value
for the newly added `parser_engine` is `parser`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser)
```

This code maintains compatibility, meaning the traditional behavior is preserved:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file)
```

To perform analysis using Prism, specify `parser_engine: :parser_prism`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism)
```

The parameter name `parser_prism` reflects the original parser_prism which was the basis for
`Prism::Translation::Parser` (now integrated into Prism):
https://github.com/kddnewton/parser-prism

This is an experimental introduction, and some incompatibilities still remain.

> [!NOTE]
> As initially mentioned in rubocop/rubocop#12600 (comment),
> the plan was to set `parser_engine: prism`.
>
> However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`:
> ruby/prism#2419
>
> `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility.
> So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future,
> it has been decided that it might be better not to use `ParserEngine: prism` for the time being.
> `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`.
>
> Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for
> `Prism::Translation::Parser`.
>
> This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`,
> not `ParserEngine: prism`.

## Compatibility

The compatibility issues between Prism and the Parser gem have not been resolved.
The failing tests will be skipped with `broken_on: :prism`:

- ruby/prism#2454 has been resolved but not yet released.
- ruby/prism#2467 is still unresolved.

Issues that will be resolved in several upcoming releases of Prism are being skipped
with `broken_on: :prism`.

Anyway, RuboCop AST can be released independently of the resolution and release of Prism.

> [!NOTE]
> The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed:
> https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb
>
> If the above interface is accepted, a fix will be proposed on the Prism side.

## Test

Tests for RuboCop AST with Prism as the backend can be run as follows:

```console
bundle exec rake prism_spec
```

The above is the shortcut alias for:

```console
PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec
```

RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+,
`internal_investigation` Rake task will not be executed. This task is only run with the Parser gem,
which can analyze Ruby versions 2.0+.
koic added a commit to koic/rubocop-ast that referenced this issue Feb 25, 2024
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism,
as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600.

## Configuration

By default, analysis is performed using the Parser gem, so the default value
for the newly added `parser_engine` is `parser_whitequark`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_whitequark)
```

This code maintains compatibility, meaning the traditional behavior is preserved:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file)
```

To perform analysis using Prism, specify `parser_engine: :parser_prism`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism)
```

The parameter name `parser_prism` reflects the original parser_prism which was the basis for
`Prism::Translation::Parser` (now integrated into Prism):
https://github.com/kddnewton/parser-prism

This is an experimental introduction, and some incompatibilities still remain.

> [!NOTE]
> As initially mentioned in rubocop/rubocop#12600 (comment),
> the plan was to set `parser_engine: prism`.
>
> However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`:
> ruby/prism#2419
>
> `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility.
> So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future,
> it has been decided that it might be better not to use `ParserEngine: prism` for the time being.
> `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`.
>
> Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for
> `Prism::Translation::Parser`.
>
> This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`,
> not `ParserEngine: prism`.

## Compatibility

The compatibility issues between Prism and the Parser gem have not been resolved.
The failing tests will be skipped with `broken_on: :prism`:

- ruby/prism#2454 has been resolved but not yet released.
- ruby/prism#2467 is still unresolved.

Issues that will be resolved in several upcoming releases of Prism are being skipped
with `broken_on: :prism`.

Anyway, RuboCop AST can be released independently of the resolution and release of Prism.

> [!NOTE]
> The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed:
> https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb
>
> If the above interface is accepted, a fix will be proposed on the Prism side.

## Test

Tests for RuboCop AST with Prism as the backend can be run as follows:

```console
bundle exec rake prism_spec
```

The above is the shortcut alias for:

```console
PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec
```

RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+,
`internal_investigation` Rake task will not be executed. This task is only run with the Parser gem,
which can analyze Ruby versions 2.0+.
koic added a commit to koic/rubocop-ast that referenced this issue Feb 25, 2024
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism,
as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600.

## Configuration

By default, analysis is performed using the Parser gem, so the default value
for the newly added `parser_engine` is `parser_whitequark`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_whitequark)
```

This code maintains compatibility, meaning the traditional behavior is preserved:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file)
```

To perform analysis using Prism, specify `parser_engine: :parser_prism`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism)
```

The parameter name `parser_prism` reflects the original parser_prism which was the basis for
`Prism::Translation::Parser` (now integrated into Prism):
https://github.com/kddnewton/parser-prism

This is an experimental introduction, and some incompatibilities still remain.

> [!NOTE]
> As initially mentioned in rubocop/rubocop#12600 (comment),
> the plan was to set `parser_engine: prism`.
>
> However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`:
> ruby/prism#2419
>
> `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility.
> So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future,
> it has been decided that it might be better not to use `ParserEngine: prism` for the time being.
> `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`.
>
> Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for
> `Prism::Translation::Parser`.
>
> This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`,
> not `ParserEngine: prism`.

## Compatibility

The compatibility issues between Prism and the Parser gem have not been resolved.
The failing tests will be skipped with `broken_on: :prism`:

- ruby/prism#2454 has been resolved but not yet released.
- ruby/prism#2467 is still unresolved.

Issues that will be resolved in several upcoming releases of Prism are being skipped
with `broken_on: :prism`.

Anyway, RuboCop AST can be released independently of the resolution and release of Prism.

> [!NOTE]
> The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed:
> https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb
>
> If the above interface is accepted, a fix will be proposed on the Prism side.

## Test

Tests for RuboCop AST with Prism as the backend can be run as follows:

```console
bundle exec rake prism_spec
```

The above is the shortcut alias for:

```console
PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec
```

RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+,
`internal_investigation` Rake task will not be executed. This task is only run with the Parser gem,
which can analyze Ruby versions 2.0+.
bbatsov pushed a commit to rubocop/rubocop-ast that referenced this issue Feb 25, 2024
This PR introduces the `parser_engine` option to `ProcessedSource` to support Prism,
as part of the RuboCop AST side effort towards addressing rubocop/rubocop#12600.

## Configuration

By default, analysis is performed using the Parser gem, so the default value
for the newly added `parser_engine` is `parser_whitequark`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_whitequark)
```

This code maintains compatibility, meaning the traditional behavior is preserved:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file)
```

To perform analysis using Prism, specify `parser_engine: :parser_prism`:

```ruby
ProcessedSource.new(@options[:stdin], ruby_version, file, parser_engine: :parser_prism)
```

The parameter name `parser_prism` reflects the original parser_prism which was the basis for
`Prism::Translation::Parser` (now integrated into Prism):
https://github.com/kddnewton/parser-prism

This is an experimental introduction, and some incompatibilities still remain.

> [!NOTE]
> As initially mentioned in rubocop/rubocop#12600 (comment),
> the plan was to set `parser_engine: prism`.
>
> However, the parser engine used in this PR is `Prism::Translation::Parser`, not `Prism`:
> ruby/prism#2419
>
> `Prism::Translation::Parser` and `Prism` have different ASTs, so their migration will definitely cause incompatibility.
> So, considering the possibility of further replacing `Prism::Translation::Parser` with `Prism` in the future,
> it has been decided that it might be better not to use `ParserEngine: prism` for the time being.
> `ParserEngine: prism` is reserved for `Prism`, not `Prism::Translation::Parser`.
>
> Therefore, the parameter value has been set to `parser_engine: parser_prism` specifically for
> `Prism::Translation::Parser`.
>
> This means that the planned way to specify Prism in .rubocop.yml file will be `ParserEngine: parser_prism`,
> not `ParserEngine: prism`.

## Compatibility

The compatibility issues between Prism and the Parser gem have not been resolved.
The failing tests will be skipped with `broken_on: :prism`:

- ruby/prism#2454 has been resolved but not yet released.
- ruby/prism#2467 is still unresolved.

Issues that will be resolved in several upcoming releases of Prism are being skipped
with `broken_on: :prism`.

Anyway, RuboCop AST can be released independently of the resolution and release of Prism.

> [!NOTE]
> The hack in `Prism::Translation::Parser` for `ProcessedSource` needs to be fixed:
> https://github.com/ruby/prism/blob/v0.24.0/lib/prism/translation/parser/rubocop.rb
>
> If the above interface is accepted, a fix will be proposed on the Prism side.

## Test

Tests for RuboCop AST with Prism as the backend can be run as follows:

```console
bundle exec rake prism_spec
```

The above is the shortcut alias for:

```console
PARSER_ENGINE=parser_prism TARGET_RUBY_VERSION=3.3 rake spec
```

RuboCop AST works on Ruby versions 2.6+, but since Prism only targets analysis for Ruby 3.3+,
`internal_investigation` Rake task will not be executed. This task is only run with the Parser gem,
which can analyze Ruby versions 2.0+.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `Internal_affairs/location_line_equality_comparison`
- `Layout/class_structure`
- `Layout/empty_lines`
- `Layout/end_of_line`
- `Layout/heredoc_indentation`
- `Layout/indentation_style`
- `Layout/line_length`
- `Layout/space_around_operators`
- `Layout/space_inside_hash_literal_braces`
- `Layout/trailing_whitespace`
- `Lint/ambiguous_operator`
- `Lint/ambiguous_regexp_literal`
- `Lint/circular_argument_reference`
- `Lint/deprecated_class_methods`
- `Lint/deprecated_constants`
- `Lint/erb_new_arguments`
- `Lint/non_deterministic_require_order`
- `Lint/numbered_parameter_assignment`
- `Lint/parentheses_as_grouped_expression`
- `Lint/redundant_dir_glob_sort`
- `Lint/redundant_require_statement`
- `Lint/refinement_import_methods`
- `Lint/require_parentheses`
- `Lint/syntax`
- `Lint/unified_integer`
- `Lint/useless_else_without_rescue`
- `Naming/block_forwarding`
- `Naming/heredoc_delimiter_case`
- `Naming/heredoc_delimiter_naming`
- `Naming/variable_number`
- `Security/yaml_load`
- `Style/arguments_forwarding`
- `Style/array_intersect`
- `Style/block_delimiters`
- `Style/collection_compact`
- `Style/command_literal`
- `Style/comparable_clamp`
- `Style/conditional_assignment_assign_in_condition`
- `Style/conditional_assignment_assign_to_condition`
- `Style/data_inheritance`
- `Style/dir_empty`
- `Style/file_empty`
- `Style/frozen_string_literal_comment`
- `Style/guard_clause`
- `Style/hash_except`
- `Style/hash_syntax`
- `Style/hash_transform_keys`
- `Style/hash_transform_values`
- `Style/if_with_boolean_literal_branches`
- `Style/multiline_ternary_operator`
- `Style/multiline_when_then`
- `Style/mutable_constant`
- `Style/nested_file_dirname`
- `Style/numeric_literals`
- `Style/numeric_predicate`
- `Style/object_then`
- `Style/quoted_symbols`
- `Style/redundant_begin`
- `Style/redundant_freeze`
- `Style/redundant_heredoc_delimiter_quotes`
- `Style/redundant_parentheses`
- `Style/rescue_modifier`
- `Style/safe_navigation`
- `Style/select_by_regexp`
- `Style/single_line_methods`
- `Style/slicing_with_range`
- `Style/string_literals`
- `Style/yaml_file_read`
- `Style/yoda_condition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Feb 29, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

However, there are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `InternalAffairs/LocationLineEqualityComparison`
- `Layout/ClassStructure`
- `Layout/EmptyLines`
- `Layout/EndOfLine`
- `Layout/HeredocIndentation`
- `Layout/IndentationStyle`
- `Layout/LineLength`
- `Layout/SpaceAroundKeyword`
- `Layout/SpaceAroundOperators`
- `Layout/SpaceInsideHashLiteralBraces`
- `Layout/TrailingWhitespace`
- `Lint/AmbiguousOperator`
- `Lint/AmbiguousRegexpLiteral`
- `Lint/CircularArgumentReference`
- `Lint/DeprecatedClassMethods`
- `Lint/DeprecatedConstants`
- `Lint/ErbNewArguments`
- `Lint/NonDeterministicRequireOrder`
- `Lint/NumberedParameterAssignment`
- `Lint/ParenthesesAsGroupedExpression`
- `Lint/RedundantDirGlobSort`
- `Lint/RedundantRequireStatement`
- `Lint/RefinementImportMethods`
- `Lint/RequireParentheses`
- `Lint/Syntax`
- `Lint/UnifiedInteger`
- `Lint/UselessElseWithoutRescue`
- `Naming/BlockForwarding`
- `Naming/HeredocDelimiterCase`
- `Naming/HeredocDelimiterNaming`
- `Naming/VariableNumber`
- `Security/YamlLoad`
- `Style/ArgumentsForwarding`
- `Style/ArrayIntersect`
- `Style/BlockDelimiters`
- `Style/CollectionCompact`
- `Style/CommandLiteral`
- `Style/ComparableClamp`
- `Style/ConditionalAssignmentAssignInCondition`
- `Style/ConditionalAssignmentAssignToCondition`
- `Style/DataInheritance`
- `Style/DirEmpty`
- `Style/FileEmpty`
- `Style/FrozenStringLiteralComment`
- `Style/GuardClause`
- `Style/HashExcept`
- `Style/HashSyntax`
- `Style/HashTransformKeys`
- `Style/HashTransformValues`
- `Style/IfWithBooleanLiteralBranches`
- `Style/MultilineTernaryOperator`
- `Style/MultilineWhenThen`
- `Style/MutableConstant`
- `Style/NestedFileDirname`
- `Style/NumericLiterals`
- `Style/NumericPredicate`
- `Style/ObjectThen`
- `Style/QuotedSymbols`
- `Style/RedundantBegin`
- `Style/RedundantFreeze`
- `Style/RedundantHeredocDelimiterQuotes`
- `Style/RedundantParentheses`
- `Style/RescueModifier`
- `Style/SafeNavigation`
- `Style/SelectByRegexp`
- `Style/SingleLineMethods`
- `Style/SlicingWithRange`
- `Style/StringLiterals`
- `Style/TernaryParentheses`
- `Style/YamlFileRead`
- `Style/YodaCondition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
koic added a commit to koic/rubocop that referenced this issue Mar 1, 2024
Resolves rubocop#12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

> [!CAUTION]
> Since the use of Prism is experimental, it is not included in RuboCop's runtime dependencies.
> If running through Bundler, please first add `gem 'prism'` to your Gemfile:
>
> ```ruby
> gem 'prism'
> ```

There are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `InternalAffairs/LocationLineEqualityComparison`
- `Layout/ClassStructure`
- `Layout/EmptyLines`
- `Layout/EndOfLine`
- `Layout/HeredocIndentation`
- `Layout/IndentationStyle`
- `Layout/LineLength`
- `Layout/SpaceAroundKeyword`
- `Layout/SpaceAroundOperators`
- `Layout/SpaceInsideHashLiteralBraces`
- `Layout/TrailingWhitespace`
- `Lint/AmbiguousOperator`
- `Lint/AmbiguousRegexpLiteral`
- `Lint/CircularArgumentReference`
- `Lint/DeprecatedClassMethods`
- `Lint/DeprecatedConstants`
- `Lint/ErbNewArguments`
- `Lint/NonDeterministicRequireOrder`
- `Lint/NumberedParameterAssignment`
- `Lint/ParenthesesAsGroupedExpression`
- `Lint/RedundantDirGlobSort`
- `Lint/RedundantRequireStatement`
- `Lint/RefinementImportMethods`
- `Lint/RequireParentheses`
- `Lint/Syntax`
- `Lint/UnifiedInteger`
- `Lint/UselessElseWithoutRescue`
- `Naming/BlockForwarding`
- `Naming/HeredocDelimiterCase`
- `Naming/HeredocDelimiterNaming`
- `Naming/VariableNumber`
- `Security/YamlLoad`
- `Style/ArgumentsForwarding`
- `Style/ArrayIntersect`
- `Style/BlockDelimiters`
- `Style/CollectionCompact`
- `Style/CommandLiteral`
- `Style/ComparableClamp`
- `Style/ConditionalAssignmentAssignInCondition`
- `Style/ConditionalAssignmentAssignToCondition`
- `Style/DataInheritance`
- `Style/DirEmpty`
- `Style/FileEmpty`
- `Style/FrozenStringLiteralComment`
- `Style/GuardClause`
- `Style/HashExcept`
- `Style/HashSyntax`
- `Style/HashTransformKeys`
- `Style/HashTransformValues`
- `Style/IfWithBooleanLiteralBranches`
- `Style/MultilineTernaryOperator`
- `Style/MultilineWhenThen`
- `Style/MutableConstant`
- `Style/NestedFileDirname`
- `Style/NumericLiterals`
- `Style/NumericPredicate`
- `Style/ObjectThen`
- `Style/QuotedSymbols`
- `Style/RedundantBegin`
- `Style/RedundantFreeze`
- `Style/RedundantHeredocDelimiterQuotes`
- `Style/RedundantParentheses`
- `Style/RescueModifier`
- `Style/SafeNavigation`
- `Style/SelectByRegexp`
- `Style/SingleLineMethods`
- `Style/SlicingWithRange`
- `Style/StringLiterals`
- `Style/TernaryParentheses`
- `Style/YamlFileRead`
- `Style/YodaCondition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
bbatsov pushed a commit that referenced this issue Mar 1, 2024
Resolves #12600.

This is the initial basic implementation to support multiple parser engines.
It is still in an experimental phase. There are still incompatibilities with Prism
compared to the Parser gem, but as the RuboCop core, it is possible to begin providing support
for Prism independently within the RuboCop core.

> [!IMPORTANT]
> To work this feature, the following patch for RuboCop AST needs to be released:
> rubocop/rubocop-ast#277

## Setting the parser engine

RuboCop allows switching the backend parser by specifying either
`parser_whitequark` or `parser_prism` for the `ParserEngine`.

Here are the parsers used as backends for each value:

- `ParserEngine: parser_whitequark` ... https://github.com/whitequark/parser
- `ParserEngine: parser_prism` ... https://github.com/ruby/prism (`Prism::Translation::Parser`)

By default, `parser_whitequark` is implicitly used.

`parser_whitequark` can analyze source code from Ruby 2.0 and above:

```yaml
AllCops:
  ParserEngine: parser_whitequark
```

`parser_prism` can analyze source code from Ruby 3.3 and above:

```yaml
AllCops:
  ParserEngine: parser_prism
  TargetRubyVersion: 3.3
```

`parser_prism` tends to perform analysis faster than `parser_whitequark`.

> [!CAUTION]
> Since the use of Prism is experimental, it is not included in RuboCop's runtime dependencies.
> If running through Bundler, please first add `gem 'prism'` to your Gemfile:
>
> ```ruby
> gem 'prism'
> ```

There are some incompatibilities with `parser_whitequark`, making it still considered experimental.
For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

## Incompatibility

At the time of opening this PR, the following cops have incompatibility with Prism:

- `Gemspec/RequiredRubyVersion`
- `InternalAffairs/LocationLineEqualityComparison`
- `Layout/ClassStructure`
- `Layout/EmptyLines`
- `Layout/EndOfLine`
- `Layout/HeredocIndentation`
- `Layout/IndentationStyle`
- `Layout/LineLength`
- `Layout/SpaceAroundKeyword`
- `Layout/SpaceAroundOperators`
- `Layout/SpaceInsideHashLiteralBraces`
- `Layout/TrailingWhitespace`
- `Lint/AmbiguousOperator`
- `Lint/AmbiguousRegexpLiteral`
- `Lint/CircularArgumentReference`
- `Lint/DeprecatedClassMethods`
- `Lint/DeprecatedConstants`
- `Lint/ErbNewArguments`
- `Lint/NonDeterministicRequireOrder`
- `Lint/NumberedParameterAssignment`
- `Lint/ParenthesesAsGroupedExpression`
- `Lint/RedundantDirGlobSort`
- `Lint/RedundantRequireStatement`
- `Lint/RefinementImportMethods`
- `Lint/RequireParentheses`
- `Lint/Syntax`
- `Lint/UnifiedInteger`
- `Lint/UselessElseWithoutRescue`
- `Naming/BlockForwarding`
- `Naming/HeredocDelimiterCase`
- `Naming/HeredocDelimiterNaming`
- `Naming/VariableNumber`
- `Security/YamlLoad`
- `Style/ArgumentsForwarding`
- `Style/ArrayIntersect`
- `Style/BlockDelimiters`
- `Style/CollectionCompact`
- `Style/CommandLiteral`
- `Style/ComparableClamp`
- `Style/ConditionalAssignmentAssignInCondition`
- `Style/ConditionalAssignmentAssignToCondition`
- `Style/DataInheritance`
- `Style/DirEmpty`
- `Style/FileEmpty`
- `Style/FrozenStringLiteralComment`
- `Style/GuardClause`
- `Style/HashExcept`
- `Style/HashSyntax`
- `Style/HashTransformKeys`
- `Style/HashTransformValues`
- `Style/IfWithBooleanLiteralBranches`
- `Style/MultilineTernaryOperator`
- `Style/MultilineWhenThen`
- `Style/MutableConstant`
- `Style/NestedFileDirname`
- `Style/NumericLiterals`
- `Style/NumericPredicate`
- `Style/ObjectThen`
- `Style/QuotedSymbols`
- `Style/RedundantBegin`
- `Style/RedundantFreeze`
- `Style/RedundantHeredocDelimiterQuotes`
- `Style/RedundantParentheses`
- `Style/RescueModifier`
- `Style/SafeNavigation`
- `Style/SelectByRegexp`
- `Style/SingleLineMethods`
- `Style/SlicingWithRange`
- `Style/StringLiterals`
- `Style/TernaryParentheses`
- `Style/YamlFileRead`
- `Style/YodaCondition`

Some cop incompatibilities have been resolved in the Prism development line.

For known issues, please refer to the Prism issues:
https://github.com/ruby/prism/issues?q=is%3Aissue+is%3Aopen+label%3Arubocop

### `Lint/Syntax` cop

The messages generated by Lint/Syntax depend on the parser engine used.

Parser gem:

```console
$ ruby -rparser/ruby33 -ve 'p Parser::Ruby33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: unexpected token $end
```

Displays `unexpected token $end`.

Prism:

```console
$ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("(")'
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
(string):1:2: error: expected a matching `)`
```

Displays `expected a matching )`.

There are differences in the messages between Parser gem and Prism,
but since Prism can provide clearer messages in some cases, this incompatibility is accepted.
In other words, the messages may vary depending on the parser engine.

## Test

To run tests with Prism, the command `bundle exec rake prism_spec` is provided.
This task is also executed in GitHub Actions.

To run tests with Prism specifying files, set the environment variable `PARSER_ENGINE=parser_prism`:

```console
$ PARSER_ENGINE=parser_prism path/to/test_spec.rb
```

In the context of testing with Prism, two options for test cases are provided:
`broken_on: :prism` and `unsupported_on: :prism`.
Both options are utilized to skip tests specifically for Prism.

### `broken_on: :prism`

Test cases failing due to Prism incompatibilities are marked with `broken_on: :prism`.
This indicates an expectation for the issue to be resolved within Prism.

### `unsupported_on: :prism`

Prism is designed to parse Ruby versions 3.3+, which means that features unique to
Ruby versions 2.0 through 3.2 are not supported.
Test cases falling into this category are marked with `unsupported_on: :prism`.
This marker is used for cases that are testable with the Parser gem but not with Prism.

> [!NOTE]
> With `bundle exec rake`, `prism_spec` will be run instead of `ascii_spec`.
> The `ascii_spec` task has not been failing for a while, so it will not be run by default.
> However, `ascii_spec` task will continue to be checked in CI. If there are any failures
> originating from `ascii_spec` in CI, please run `bundle exec ascii_spec` to investigate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement high priority A ticket considered important by RuboCop's Core Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants