Skip to content

Commit

Permalink
Support multi-versioning for Prism::Translation::Parser
Browse files Browse the repository at this point in the history
## Summary

Fixes ruby#2356.

I'm working on integrating Prism into RuboCop.

This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
rubocop/rubocop#12600 (comment)

Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.

Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.

### Expected AST

The following is an expected AST when parsing Ruby 3.3 code:

```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:send, nil, :it), :do_something))
```

### Actual AST

The following is an actual AST when parsing Ruby 3.3 code:

```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
s(:block,
  s(:send,
    s(:send, nil, :items), :map),
  s(:args),
  s(:send,
    s(:lvar, :"0it"), :do_something))
```

`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.

And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
rubocop/rubocop#12600 (comment)

## Additional Information

A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.

`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.

Therefore, logic to maintain compatibility between Parser and Prism is implemented.
  • Loading branch information
koic committed Feb 15, 2024
1 parent cf0369a commit 62d3991
Show file tree
Hide file tree
Showing 5 changed files with 50 additions and 6 deletions.
18 changes: 15 additions & 3 deletions lib/prism/translation/parser.rb
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def parse(source_buffer)
source = source_buffer.source

offset_cache = build_offset_cache(source)
result = unwrap(Prism.parse(source, filepath: source_buffer.name), offset_cache)
result = unwrap(Prism.parse(source, filepath: source_buffer.name, version: convert_for_prism(version)), offset_cache)

build_ast(result.value, offset_cache)
ensure
Expand All @@ -56,7 +56,7 @@ def parse_with_comments(source_buffer)
source = source_buffer.source

offset_cache = build_offset_cache(source)
result = unwrap(Prism.parse(source, filepath: source_buffer.name), offset_cache)
result = unwrap(Prism.parse(source, filepath: source_buffer.name, version: convert_for_prism(version)), offset_cache)

[
build_ast(result.value, offset_cache),
Expand All @@ -75,7 +75,7 @@ def tokenize(source_buffer, recover = false)
offset_cache = build_offset_cache(source)
result =
begin
unwrap(Prism.parse_lex(source, filepath: source_buffer.name), offset_cache)
unwrap(Prism.parse_lex(source, filepath: source_buffer.name, version: convert_for_prism(version)), offset_cache)
rescue ::Parser::SyntaxError
raise if !recover
end
Expand Down Expand Up @@ -168,6 +168,18 @@ def build_range(location, offset_cache)
)
end

# Converts the version format handled by Parser to the format handled by Prism.
def convert_for_prism(version)
case version
when 33
"3.3.0"
when 34
"3.4.0"
else
"latest"
end
end

require_relative "parser/compiler"
require_relative "parser/lexer"

Expand Down
12 changes: 9 additions & 3 deletions lib/prism/translation/parser/rubocop.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,24 @@
module Prism
module Translation
class Parser
# This is the special version number that should be used in rubocop
# This is the special version numbers that should be used in RuboCop
# configuration files to trigger using prism.
# For Ruby 3.3
VERSION_3_3 = 80_82_73_83_77.33
# For Ruby 3.4
VERSION_3_4 = 80_82_73_83_77.34

# This module gets prepended into RuboCop::AST::ProcessedSource.
module ProcessedSource
# Redefine parser_class so that we can inject the prism parser into the
# list of known parsers.
def parser_class(ruby_version)
if ruby_version == Prism::Translation::Parser::VERSION_3_3
require "prism/translation/parser"
Prism::Translation::Parser
require "prism/translation/parser33"
Prism::Translation::Parser33
if ruby_version == Prism::Translation::Parser::VERSION_3_4
require "prism/translation/parser34"
Prism::Translation::Parser34
else
super
end
Expand Down
12 changes: 12 additions & 0 deletions lib/prism/translation/parser33.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
require_relative "parser"

module Prism
module Translation
# This class is the entry-point for Ruby 3.3 of `Prism::Translation::Parser`.
class Parser33 < Parser
def version # :nodoc:
33
end
end
end
end
12 changes: 12 additions & 0 deletions lib/prism/translation/parser34.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
require_relative "parser"

module Prism
module Translation
# This class is the entry-point for Ruby 3.4 of `Prism::Translation::Parser`.
class Parser34 < Parser
def version # :nodoc:
34
end
end
end
end
2 changes: 2 additions & 0 deletions prism.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ Gem::Specification.new do |spec|
"lib/prism/serialize.rb",
"lib/prism/translation.rb",
"lib/prism/translation/parser.rb",
"lib/prism/translation/parser33.rb",
"lib/prism/translation/parser34.rb",
"lib/prism/translation/parser/compiler.rb",
"lib/prism/translation/parser/lexer.rb",
"lib/prism/translation/parser/rubocop.rb",
Expand Down

0 comments on commit 62d3991

Please sign in to comment.