Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Respect Unicode characters in import sorting #10529

Merged
merged 1 commit into from Mar 22, 2024
Merged

Conversation

charliermarsh
Copy link
Member

Summary

Ensures that we use the raw identifier as provided in the source code, rather than the normalized Unicode identifier.

This does mean that we treat these as two separate identifiers, and don't merge them, even though Python will treat them as the same symbol:

import numpy as ℂℇℊℋℌℍℎℐℑℒℓℕℤΩℨKÅℬℭℯℰℱℹℴ
import numpy as CƐgHHHhIILlNZΩZKÅBCeEFio

I think that's fine, this is super rare anyway and would likely be confusing for users.

Closes #10528.

Test Plan

cargo test

@charliermarsh charliermarsh added bug Something isn't working isort Related to import sorting labels Mar 22, 2024
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks correct to me.

I worry that there might be similar issues in other rules that do autofixes -- it seems reasonable to rely on the id of a Name node being the same as in the original source, and then using it for autofixes. So, maybe eagerly normalizing unicode identifiers in the lexer is a bit of a bug magnet now ://

Copy link
Contributor

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@charliermarsh charliermarsh merged commit 61b7982 into main Mar 22, 2024
17 checks passed
@charliermarsh charliermarsh deleted the charlie/isort branch March 22, 2024 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working isort Related to import sorting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Autofix for I001 unexpectedly altering characters from Unicode Block “Letterlike Symbols”
3 participants