ci: Replace lint and formatting tools with ruff

[mitches-got-glitches]

Context

I thought I would have a go at adding ruff to the codebase in anticipation of using it for #1060. My initial thought was to suggest pydocstyle for that ticket, but this project has been deprecated in favour of ruff. The speed benefits of using ruff are pretty significant:

Throughout doing this ticket, I have figured out that we could add ruff and tell it only to check the pydocstyle codes - so that is an alternative if this change is a bit much.

Key changes

• ruff replaces black, flake8, reorder-python-imports and some other pre-commit hooks in .pre-commit-config.yaml
• ruff added to requirements_dev.txt. flake8, pep8-naming, flake8-debugger, flake8-print, flake8-todo are removed.
• Config added to top level ruff.toml to mirror previous configuration as closely as possible.
• Lower level configs added under tests/ruff.toml and tests_functional/ruff.toml to prevent the need for unnecessary changes to these packages. These can be reviewed at a later time.
• pep8 section removed from Makefile - covered instead by the pre-commit section

Notes

• Rules W503 and W504 are not implemented in Ruff - see this issue: flake8's (W503 and) W504 astral-sh/ruff#4125
• Many of the changes made can be explained by looking at ruff's Known Deviations from Black
  • This change happens because ruff looks at Unicode width
  • Adding a line of whitespace after module docstrings seems to be something ruff wants to do (can't find the explanation), and this amounts to quite a lot of the changes
  • This change is because pragma comments are ignored when computing line width
  • This is one change that looks a little buggy to me. The # type: ignore has moved to the second broken line when I'm guessing it has been added for the _ensure_serializable(v). Didn't seem to be picked up by mypy though so it still passes. Should this be moved up a line in this case? I'm not sure this section on pragma comments explains the behaviour we're seeing here enough.
• ruff seems to be one-lining these and this has been raised as an unintentional deviation. Maybe something that the project can live with for now?
• It looks like black was never in requirements_dev.txt?

Extras

• The ignore rules in the ruff.toml can potentially be tuned as per-file-ignores
• I can offer some extra proof that some of the things that have been removed still work with ruff - just let me know what you want to see.

[netlify]

✅ Deploy Preview for dynaconf ready!

Name	Link
🔨 Latest commit	33013a5
🔍 Latest deploy log	https://app.netlify.com/sites/dynaconf/deploys/65f87625b7d7c700081999a8
😎 Deploy Preview	https://deploy-preview-1074--dynaconf.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

Attention: Patch coverage is 91.30435% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 98.75%. Comparing base (b29fc06) to head (33013a5).
Report is 1 commits behind head on master.

Files	Patch %	Lines
dynaconf/validator.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1074      +/-   ##
==========================================
- Coverage   98.80%   98.75%   -0.05%     
==========================================
  Files          23       23              
  Lines        2253     2248       -5     
==========================================
- Hits         2226     2220       -6     
- Misses         27       28       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pedro-psb]

That looks good. I can see it catches TODO, breakpoints and prints, and seems to do a good job at sorting the imports. Also, I don't mind the known style deviations.

Can you hard-comment the reason for tests/ruff.toml and 'test_functional/ruff.toml` ? I think it can be helpful.

[pedro-psb]

We should wait @rochacbruno review as well, as these are pretty significant changes.

And out of curiosity, why would we want per-file-ignores? I'm not saying we don't, I just don't know.

[mitches-got-glitches]

We should wait @rochacbruno review as well, as these are pretty significant changes.

And out of curiosity, why would we want per-file-ignores? I'm not saying we don't, I just don't know.

Just as an example, if I enable a check on E402: Module level import not at the top of the file, we end up with these issues reported back:

Now we might not care too much about the rule contraventions in tests_functional/ but maybe we do care about it in dyanconf/cli.py. So we could have something like this in the ruff.toml:

[lint.per-file-ignores]
"tests_functional/*" = ["E402"]

And we end up with this:

The checks that we currently ignore may be worth applying over the dynaconf/ folder at least. What do you think?

[mitches-got-glitches]

Can you hard-comment the reason for tests/ruff.toml and 'test_functional/ruff.toml` ? I think it can be helpful.

I've added some new commits which actually remove these in favour of per-file-ignores. Let me know what you think of the latest commits where I've implemented per-file-ignores. Tried to do everything on separate commits so things can be rolled back if needed.

[pedro-psb]

Thanks for clarifying per-ignore-files.
Its a nice feature and I prefer it over the extra config files.
This looks good for me! 🎉