Issue #12507: Support record patterns preview

[nrmancuso]

closes #12507

We extend the Java programming language with nestable record patterns.

The grammar for patterns will become:

Pattern:
  TypePattern
  ParenthesizedPattern
  RecordPattern

TypePattern:
  LocalVariableDeclaration

ParenthesizedPattern:
  ( Pattern )

RecordPattern:
  ReferenceType RecordStructurePattern [ Identifier ]

RecordStructurePattern:
  ( [ RecordComponentPatternList ] )

RecordComponentPatternList : 
  Pattern { , Pattern }

---

Compact view of AST: https://github.com/checkstyle/checkstyle/wiki/Record-Pattern-Preview---AST-strucuture

---

Reports

Check regression reports:
https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/part1-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/part4-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/part5-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/sevntu-check-regression_part_2-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/part2-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/part6-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/part3-report/index.html

https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/sevntu-check-regression_part_1-report/index.html

No differences in check behavior ^

ANTLR regression report:
https://nrmancuso.github.io/reports/issue-12507/2023-04-21-T-13-58-36/antlr-report/index.html

Only change is in exception line number in non-compilable files ^

[nrmancuso]

Notes to self:

• Need to add more inputs with while/do while/for loops with record pattern decomp
• nested decomp in loops

Example:

        if (param instanceof A(var o)) {
            System.out.println(o);
        }

        while (param instanceof A(var o)) {
            System.out.println((String)o);
        }

        for (; param instanceof A(var o);) {
            System.out.println((String)o);
        }

[nrmancuso]

@romani @rnveach @pbludov @strkkk please take a look at latest commit (3dc7ed7) and let's discuss new tokens/ AST structure.

[nrmancuso]

please take a look at latest commit (3dc7ed7) and let's discuss new tokens/ AST structure.

@rnveach @romani ping

[rnveach]

@nrmancuso Do you have a link to the JLS that your changes cover?

[nrmancuso]

@nrmancuso Do you have a link to the JLS that your changes cover?

No, since this is a preview feature, the only specification is in the JEP.

[rnveach]

please take a look at latest commit (3dc7ed7) and let's discuss new tokens/ AST structure.

The only thing I noticed to bring up is I question the need for RECORD_PATTERN_COMPONENTS when it only has 1 child. COMPONENTS to me make it sounds like multiples, but I didn't find any with multiple. Maybe I am just missing it.

[nrmancuso]

@rnveach example of multiple components

[rnveach]

Thanks I figured I was missing it. That was my only real concern. I feel fine with what you are doing.

[nrmancuso]

please take a look at latest commit (3dc7ed7) and let's discuss new tokens/ AST structure.

@romani ping

[nrmancuso]

This is the beginning of a complicated case of record pattern decomposition:

        if (r instanceof Rectangle(ColoredPoint(Point p1,Color c1),
                ColoredPoint lr1)
                && r instanceof Rectangle(ColoredPoint(Point p2,Color c2),
                ColoredPoint lr2) && lr2.c == Color.BLUE) {
            System.out.println(r);
        }

[romani]

it is easier to read at https://github.com/checkstyle/checkstyle/wiki/Record-Pattern-Preview---AST-strucuture

[nrmancuso]

Start of another nested record decomposition, with when usage:

            case Box<Box<String>>(Box<String>(String s)box)
                    when (("test".equals(s) && box.x != 7))  -> 1;

[nrmancuso]

@romani left some comments, but basically we are only adding record patterns to existing pattern grammar. It is a tricky syntax for parsing, which is why I have all the crazy inputs. But - from a reviewing perspective a great deal of the input here very similar.

[nrmancuso]

Also, To help others to understand what is happening with record decomposition:

if (b instanceof Box<String>(String s)) is similar to if (b instanceof String s); the record component pattern allows us to declare and use a pattern variable that is a record component of the record that is in the record pattern. Consider:

➜  src javac --enable-preview -source 19 Test.java
Note: Test.java uses preview features of Java SE 19.
Note: Recompile with -Xlint:preview for details.
➜  src cat Test.java 
class Test<T> {
    record Box<T>(T t) {}

    // example of nested record decomposition with pattern
    // variable declaration from nested record
    T m1(Box<Box<T>> b) {
        // this declaration is equaivlent to the following decomposition
        // in 'if' statement
        final Box<Box<String>> nestedRecordBox =
                new Box<>(new Box<>("some string"));

        if (b instanceof Box<Box<T>>( // <- 'outer' record is decomposed
                Box<T>( // <- 'inner' record is decomposed
                        String s // <- pattern variable is declared
                        ))) {
            // pattern variable s is in scope
            System.out.println(s);
        }
        return null;
    }

    // example of record decomposition with pattern variable declaration
    T m2(Box<T> b) {
        // this declaration is equaivlent to the following decomposition
        // in 'if' statement
        final Box<String> recordBox = new Box<>("some string");

        if (b instanceof Box<T>( // <- record is decomposed
                String s // <- pattern variable is declared
                )) {
            // pattern variable s is in scope
            System.out.println(s);
        }
        return null;
    }

    // example of a pattern variable declaration
    void m3(Object o) {
        // this declaration is equaivlent to the following pattern variable
        // declaration in 'if' statement
        final String str = "some string";

        if (o instanceof String s // <- pattern variable is declared
            ) {
            // pattern variable s is in scope
            System.out.println(s);
        }
    }
}

Of course, I made the actual input files terrible to make sure we can parse all such code, but basically all the inputs are variations on the above.

[romani]

items:

[romani]

I see no problems with AST, all good.
https://github.com/checkstyle/checkstyle/wiki/Record-Pattern-Preview---AST-strucuture
I think we can finalize this PR.

config/projects-to-test/openjdk19-excluded.files

jdk19 to no-execution to make sure we can parse all record patterns that openjdk have in its test inputs - GOOD.

items:

[romani]

Last minor:

[romani]

Ok to merge