Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.razor: implement the token type metrics #7895

Open
costin-zaharia-sonarsource opened this issue Aug 25, 2023 · 4 comments · May be fixed by #7924
Open

.razor: implement the token type metrics #7895

costin-zaharia-sonarsource opened this issue Aug 25, 2023 · 4 comments · May be fixed by #7924
Assignees
@antonioaversa
Labels
Area: C# C# rules related issues. Type: New Feature This hasn't been here before.

Comments

@costin-zaharia-sonarsource
Copy link
Member

costin-zaharia-sonarsource commented Aug 25, 2023 •
edited by antonioaversa

Description

See the "Token type" section from the specification document.
More details about the approach here.

Alternative

An alternative to this approach, which only focuses on mapping back tokens without altering the fundamental behavior of the Token Type Analyzer, is to use the RazorSyntaxTree parsing. That approach, that is more powerful but also more complex to implement, is captured by a dedicated issue.

Update

Moving back to "To do" waiting for the issue with location mapping for nested Razor expression to be fixed first, since wrong location mapping can produce wrong and overlapping tokens, and those break tokenization import on SonarQube/SonarCloud by the sensor.
@costin-zaharia-sonarsource costin-zaharia-sonarsource added Type: New Feature This hasn't been here before. Area: C# C# rules related issues. Sprint: MMF-3322 🪒 labels Aug 25, 2023
@costin-zaharia-sonarsource costin-zaharia-sonarsource added this to the 9.9 milestone Aug 25, 2023
@costin-zaharia-sonarsource costin-zaharia-sonarsource removed this from the 9.9 milestone Aug 25, 2023
@antonioaversa antonioaversa moved this from To do to In progress in Best Kanban Aug 28, 2023
@antonioaversa antonioaversa linked a pull request Aug 30, 2023 that will close this issue
.razor: implement the token type metrics #7924
Draft
@antonioaversa antonioaversa linked a pull request Aug 30, 2023 that will close this issue
Draft
@github-actions github-actions bot moved this from In progress to Review in progress in Best Kanban Aug 30, 2023
@antonioaversa antonioaversa moved this from Review in progress to In progress in Best Kanban Aug 30, 2023
@github-actions github-actions bot moved this from In progress to Review in progress in Best Kanban Aug 31, 2023
@antonioaversa antonioaversa moved this from Review in progress to In progress in Best Kanban Sep 1, 2023
@antonioaversa
Copy link
Contributor

Handed over to @costin-zaharia-sonarsource. The underlying PR describes the current status and the next steps.

@costin-zaharia-sonarsource costin-zaharia-sonarsource moved this from In progress to To do in Best Kanban Sep 4, 2023
@costin-zaharia-sonarsource costin-zaharia-sonarsource removed their assignment Sep 4, 2023
@csaba-sagi-sonarsource csaba-sagi-sonarsource self-assigned this Sep 6, 2023
@csaba-sagi-sonarsource csaba-sagi-sonarsource moved this from To do to In progress in Best Kanban Sep 6, 2023
@csaba-sagi-sonarsource csaba-sagi-sonarsource removed their assignment Sep 18, 2023
@antonioaversa
Copy link
Contributor

Took it over from @csaba-sagi-sonarsource.
After discussing with @csaba-sagi-sonarsource and @costin-zaharia-sonarsource, I am going to:

  • rebase onto the current master and fix conflicts
  • extend unit tests to include missing Razor features (e.g. nested components)
  • making sure everything works as expected and have a third round of review from @costin-zaharia-sonarsource
  • if time allows, check locally the performance delta (overall build time on mud-blazor) w.r.t. master (before master validation on peach)

@antonioaversa antonioaversa moved this from In progress to To do in Best Kanban Sep 21, 2023
@antonioaversa
Copy link
Contributor

@costin-zaharia-sonarsource
I am doing backlog cleanup, and I have a question about this issue and its related PR.

The issue has been hanging on the Kanban board, for a while now.
At the time we decided not to proceed with razor tokenization, due to the issues with locations we had at the beginning.
In the end, however, if I recall correctly, we went for our own custom mapping logic, so this task may make more sense now.
There are still problems with location mapping in nested components, as explained in the description of the PR.
As mentioned here, #8040 should be addressed first.

There is quite a lot of code, and the PR went also for a round of review.
However, the branch has become stale, and by now there are 10 conflicts with master.

How would you advise to proceed?

@costin-zaharia-sonarsource
Copy link
Member Author

Since there are still problems with the mapping and the Roslyn APIs don't help us much currently, I would prefer to pause this effort for now. In my opinion, we should wait for the compiler team to finish the current effort in improving the API and continue when the new registrations are available.

@andrei-epure-sonarsource andrei-epure-sonarsource removed this from To do in Best Kanban Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antonioaversa antonioaversa

Labels
Area: C# C# rules related issues. Type: New Feature This hasn't been here before.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

.razor: implement the token type metrics SonarSource/sonar-dotnet
4 participants
@antonioaversa @costin-zaharia-sonarsource @csaba-sagi-sonarsource @cristian-ambrosini-sonarsource