Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace LALRPOP parser with hand-written parser #10036

Merged
merged 114 commits into from
Apr 18, 2024
Merged

Replace LALRPOP parser with hand-written parser #10036

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
1475a4d
Replace LALRPOP parser with hand-written parser
LaBatata101 Feb 19, 2024
e4fc0f4
Fix tests and clippy warnings
dhruvmanila Mar 7, 2024
a09b11c
Assert the parser is at augmented assign token (#10269)
dhruvmanila Mar 7, 2024
40ad261
Encapsulate `Program` fields (#10270)
dhruvmanila Mar 7, 2024
7c021af
Rename to include "token" in method name (#10287)
dhruvmanila Mar 8, 2024
804eb0e
Remove deprecated parsing list functions (#10271)
dhruvmanila Mar 8, 2024
691d3eb
Improve various assignment target error (#10288)
dhruvmanila Mar 8, 2024
2aa8f4c
Remove `skip_until` parser method (#10293)
dhruvmanila Mar 8, 2024
1d98cbf
Remove `Pattern::Invalid` variant (#10294)
dhruvmanila Mar 8, 2024
c679764
Move parser quick test to pretty print code
dhruvmanila Mar 11, 2024
cfca1ff
Fix tests and error recovery token sets (#10338)
dhruvmanila Mar 12, 2024
0dfd4dd
Remove `FStringElement::Invalid`, improve parsing logic (#10327)
dhruvmanila Mar 12, 2024
1c7c323
Introduce `LexicalError::InvalidByteLiteral` (#10328)
dhruvmanila Mar 12, 2024
6220e79
Merge string parsing to single entrypoint (#10339)
dhruvmanila Mar 12, 2024
93b9d89
Set new parser for benchmarks
dhruvmanila Mar 13, 2024
5eff87d
Remove f-string `UnclosedLbrace` error checking from the lexer (#10372)
LaBatata101 Mar 14, 2024
60fd525
Use string flags to mark it as invalid (#10385)
dhruvmanila Mar 14, 2024
f23a099
Remove `Expr::Invalid` (#10386)
dhruvmanila Mar 14, 2024
5b63251
Use recovery context to decide on trailing comma (#10405)
dhruvmanila Mar 14, 2024
5399bce
Fix clippy warnings
dhruvmanila Mar 14, 2024
7e1c1ae
Move parser tests to new valid/invalid framework (#10471)
dhruvmanila Mar 19, 2024
08bfd2c
Fix string parsing error ranges (#10406)
dhruvmanila Mar 19, 2024
8da7aba
Remove unreachable error case while f-string parsing (#10474)
dhruvmanila Mar 19, 2024
1bb22d1
Use `TokenId` to track parser progress (#10486)
dhruvmanila Mar 20, 2024
33d1453
Update snapshots from rebase
dhruvmanila Mar 20, 2024
b5ca957
Update with item parsing to avoid infinite lookaheads (#10418)
dhruvmanila Mar 20, 2024
44b2ac4
Remove `peek_nth`, only allow 1 lookahead (#10443)
dhruvmanila Mar 20, 2024
bce8784
Wrap around when incrementing token id (#10544)
dhruvmanila Mar 24, 2024
9fd5fc8
Update `ExprName` ctx, add tests (#10552)
dhruvmanila Mar 25, 2024
8498e27
Complete end expression token set (#10553)
dhruvmanila Mar 25, 2024
e150b1c
Better error recovery for match patterns (#10477)
dhruvmanila Mar 25, 2024
2f9f858
Implement rules around star expressions with different precedence (#1…
dhruvmanila Mar 29, 2024
a1ae0cb
Add tests for list expression parsing (#10554)
dhruvmanila Mar 29, 2024
21f8be9
Add tests for set expression parsing (#10555)
dhruvmanila Mar 29, 2024
79db220
Add tests for dictionary expression parsing (#10556)
dhruvmanila Mar 29, 2024
6e467f4
Limit comprehension as per the grammar (#10633)
dhruvmanila Mar 29, 2024
d77bd9b
Add tests for list comprehension (#10634)
dhruvmanila Mar 29, 2024
c01d432
Add tests for set comprehension (#10635)
dhruvmanila Mar 29, 2024
7345537
Add tests for dictionary comprehension (#10636)
dhruvmanila Mar 29, 2024
21a289a
Add tests for parenthesized expressions, tuple, generator (#10641)
dhruvmanila Mar 29, 2024
a3a0bb0
Add tests for subscript and slice expressions (#10642)
dhruvmanila Mar 29, 2024
9e822da
Implement `Display` for `TokenKind` (#10678)
dhruvmanila Mar 31, 2024
d5fcd07
Add support to write inline tests for the parser (#10681)
dhruvmanila Apr 2, 2024
393a7ec
Remove numbered list for parser contributing doc
dhruvmanila Apr 2, 2024
d6ab61d
Limit `bitwise_or` precedence for starred expression with error (#10657)
dhruvmanila Apr 3, 2024
c5061ff
Report error for `yield` and `starred` in bitwise or parsing (#10748)
dhruvmanila Apr 3, 2024
13fd991
Add tests for yield and yield from expressions (#10658)
dhruvmanila Apr 3, 2024
e5fbf81
Add tests for named expression (#10659)
dhruvmanila Apr 3, 2024
cbdb055
Add tests for conditional expression (#10660)
dhruvmanila Apr 3, 2024
6a8c4f8
Add precedence error handling in parsing LHS expression (#10679)
dhruvmanila Apr 3, 2024
17d763e
Add tests for await expression (#10680)
dhruvmanila Apr 3, 2024
765dd6d
Add tests for attribute expression (#10725)
dhruvmanila Apr 3, 2024
23943f0
Add tests for unary expressions (#10726)
dhruvmanila Apr 3, 2024
ed13e67
Add tests for boolean operation expression (#10727)
dhruvmanila Apr 3, 2024
3465794
Add tests for compare operator expressions (#10730)
dhruvmanila Apr 3, 2024
883a8c6
Add tests for binary operator expressions (#10731)
dhruvmanila Apr 3, 2024
b5b20bd
Add tests for function call arguments (#10733)
dhruvmanila Apr 3, 2024
efd5517
Add tests for call expression (#10734)
dhruvmanila Apr 3, 2024
b89068f
Add tests for lambda expression body (#10737)
dhruvmanila Apr 3, 2024
115e731
Add bitflags dependency
dhruvmanila Apr 4, 2024
1e3a7a2
Add tests for string and f-string literals (#10757)
dhruvmanila Apr 4, 2024
6f86b6e
Add tests for `global`, `nonlocal` (#10758)
dhruvmanila Apr 4, 2024
f415d5d
Add tests for `return` statement (#10767)
dhruvmanila Apr 4, 2024
8f4c618
Add tests for `raise` statement (#10768)
dhruvmanila Apr 4, 2024
db3a0b0
Add tests for `assert` statement (#10769)
dhruvmanila Apr 4, 2024
513b52d
Add tests for `del` statement (#10770)
dhruvmanila Apr 4, 2024
893f093
Add tests for `import` and `from ... import` statement (#10771)
dhruvmanila Apr 4, 2024
bc97554
Remove unreferenced snapshot
dhruvmanila Apr 5, 2024
51269d5
Use `test_ok` for inline tests, merge test report (#10801)
dhruvmanila Apr 6, 2024
03aabcf
Avoid error handling duplication for starred, yield, lambda expressio…
dhruvmanila Apr 9, 2024
18f8b75
Fix with items parsing for yield, starred, named expr (#10811)
dhruvmanila Apr 9, 2024
b16d586
Add tests for type alias statement (#10785)
dhruvmanila Apr 9, 2024
9812eec
Add tests for all kinds of assignment statement (#10786)
dhruvmanila Apr 9, 2024
1407194
Add tests for `if` statement (#10789)
dhruvmanila Apr 9, 2024
5a7eec9
Add test cases for `try` statement (#10815)
dhruvmanila Apr 9, 2024
4de7c69
Add tests for `for` statement (#10816)
dhruvmanila Apr 9, 2024
ca7e861
Add tests for `while` statement (#10817)
dhruvmanila Apr 9, 2024
5f3dae9
Add tests for decorators (#10818)
dhruvmanila Apr 9, 2024
a63878d
Add tests for `async` statement (#10819)
dhruvmanila Apr 9, 2024
3c7a129
Add tests for parameters (#10828)
dhruvmanila Apr 9, 2024
af224e6
Add tests for function definition (#10829)
dhruvmanila Apr 9, 2024
3dfcd74
Add tests for class definition (#10830)
dhruvmanila Apr 9, 2024
d351b8d
Add tests for `match` statement (#10831)
dhruvmanila Apr 9, 2024
d2aff02
fixup! Add tests for `match` statement (#10831) (#10853)
dhruvmanila Apr 10, 2024
e4c53cd
Move IPython test snapshots
dhruvmanila Apr 10, 2024
d3141e9
Remove `is_current_token_postfix` (#10855)
dhruvmanila Apr 10, 2024
a708932
Move remaining tests to new framework (#10856)
dhruvmanila Apr 10, 2024
07cdafd
Remove commented out inline tests (#10857)
dhruvmanila Apr 10, 2024
5edcf46
Use `Semi` as terminator token for simple stmts (#10859)
dhruvmanila Apr 11, 2024
52f3659
Add tests for top level statement parsing (#10860)
dhruvmanila Apr 11, 2024
7b4a1c0
Move arguments and delete target validation in `Parser` (#10871)
dhruvmanila Apr 11, 2024
dbb56b9
Add `display` to `TokenKind` for error message (#10870)
dhruvmanila Apr 11, 2024
1be2041
Review error enum variants and messages (#10872)
dhruvmanila Apr 11, 2024
f45d799
Improve IPython escape command parsing logic (#10873)
dhruvmanila Apr 11, 2024
52aed8d
Fix parameter range if it includes `*`, `**` (#10875)
dhruvmanila Apr 11, 2024
b09a89f
Reject corpus which produces empty range for LALRPOP parser (#10876)
dhruvmanila Apr 11, 2024
02e6345
Expect `for` after `async` instead of `bump` (#10877)
dhruvmanila Apr 11, 2024
08a478f
Pre-compute the module range for LALRPOP parser (#10879)
dhruvmanila Apr 11, 2024
8ef7d36
Move expr token set, docs improvements (#10887)
dhruvmanila Apr 11, 2024
b156d0b
Consider exactly `_` for wildcard pattern (#10896)
dhruvmanila Apr 12, 2024
0780f78
Fix clippy
dhruvmanila Apr 12, 2024
d61c689
fixup! Pre-compute the module range for LALRPOP parser (#10879)
dhruvmanila Apr 12, 2024
dbb0e2f
Add tests for simple target in annotated assign stmt (#10897)
dhruvmanila Apr 12, 2024
f1f7657
Disallow dotted name in `from ... import` statement (#10903)
dhruvmanila Apr 12, 2024
3679535
Expect `,` for parenthesized with items and not at `)` (#10910)
dhruvmanila Apr 15, 2024
1c1d0ec
Avoid panic in invalid pattern recovery (#10966)
dhruvmanila Apr 16, 2024
f3a90a2
Recover from unexpected token at end of expr (#10981)
dhruvmanila Apr 16, 2024
04a70b4
Reset parser context in parenthesized expression (#10994)
dhruvmanila Apr 17, 2024
f464a42
Remove invalid named expr test from formatter (#10995)
dhruvmanila Apr 17, 2024
80f5419
Add `if` as sequence pattern list terminator (#10996)
dhruvmanila Apr 17, 2024
6b73fea
Preserve comma with starred expr in subscript (#10997)
dhruvmanila Apr 17, 2024
e5e3692
Remove LALRPOP based parser (#10998)
dhruvmanila Apr 17, 2024
924ed5e
Remove leftover fuzz target
dhruvmanila Apr 17, 2024
98a7669
Remove leftover `Invalid` enum variants
dhruvmanila Apr 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,7 @@ crates/ruff_linter/resources/test/fixtures/pycodestyle/W391_3.py text eol=crlf
crates/ruff_python_formatter/resources/test/fixtures/ruff/docstring_code_examples_crlf.py text eol=crlf
crates/ruff_python_formatter/tests/snapshots/format@docstring_code_examples_crlf.py.snap text eol=crlf

crates/ruff_python_parser/resources/inline linguist-generated=true

ruff.schema.json linguist-generated=true text=auto eol=lf
*.md.snap linguist-language=Markdown
219 changes: 8 additions & 211 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.