Use empty range when there's "gap" in token source #11032
Merged
+195
−26
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This fixes a bug where the parser would panic when there is a "gap" in
the token source.
What's a gap?
The reason it's
<=
instead of just==
is because there could be whitespaces betweenthe two tokens. For example:
Or, there could tokens that are considered "trivia" and thus aren't emitted by the token
source. These are comments and non-logical newlines. For example:
In either of the above cases, there's a "gap" between the end of the last token and start
of the current token.
Test Plan
Add test cases and update the snapshots.