feat!: skip running warnings in --quiet mode

[me4502]

Prerequisites checklist

• I have read the contributing guidelines.

What is the purpose of this pull request? (put an "X" next to an item)

[ ] Documentation update
[ ] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofix to a rule
[ ] Add a CLI option
[X] Add something to the core
[ ] Other, please explain:

https://github.com/eslint/rfcs/tree/main/designs/2023-only-run-reporting-rules

Fixes #16450

What changes did you make? (Give an overview)

This PR implements the RFC around only running rules that actually report information in quiet mode. This adds a predicate function that is passed down to the runRules function that filters out all non-error rules when the --quiet flag is entered and maxWarnings is not in use. All rules returning false from the predicate function are skipped during rule running.

TODO:

• Tests
• Documentation changes
• Ignoring of unused disable directives

I believe the --max-warnings flag handling is already covered by the existing tests, as from a test perspective it only cares about if it still works with quiet mode (everything else is an implementation detail). I've added tests that the ruleFilter option does actually filter out rules to the linter.

Is there anything you'd like reviewers to focus on?

[github-actions]

Hi everyone, it looks like we lost track of this pull request. Please review and see what the next steps are. This pull request will auto-close in 7 days without an update.

[Rec0iL99]

Not stale. Work in progress.

[github-actions]

Hi everyone, it looks like we lost track of this pull request. Please review and see what the next steps are. This pull request will auto-close in 7 days without an update.

[me4502]

Sorry am still working on it; have been a bit busy and given this is blocked by the next major release it’s not too high on my list of priorities as I’m assuming that’s still at least a little while away - I’ll try to finish it up sometime soonish then rebase it every so often to keep the bot happy

[github-actions]

Hi everyone, it looks like we lost track of this pull request. Please review and see what the next steps are. This pull request will auto-close in 7 days without an update.

[me4502]

Not stale sorry, I’ve been fairly busy

[github-actions]

Hi everyone, it looks like we lost track of this pull request. Please review and see what the next steps are. This pull request will auto-close in 7 days without an update.

[netlify]

✅ Deploy Preview for docs-eslint canceled.

Name	Link
🔨 Latest commit	504cfa5
🔍 Latest deploy log	https://app.netlify.com/sites/docs-eslint/deploys/658aa57c87fbb40008815f6c

[github-actions]

Hi everyone, it looks like we lost track of this pull request. Please review and see what the next steps are. This pull request will auto-close in 7 days without an update.

[me4502]

I believe this should be ready for review. I looked over it and wrote some tests to validate that the ruleFilter option on the linter functions correctly, and the interactions with the --max-warnings flag appear to be fully covered by existing tests

[me4502]

As it seems some breaking changes for v9 are now in the main branch, should I mark this ready for review?

[mdjermanovic]

As it seems some breaking changes for v9 are now in the main branch, should I mark this ready for review?

Yes, I marked this as ready for review now.

[mdjermanovic]

One thing we forgot to mention in the RFC is caching.
If user runs eslint --cache --quiet, only errors will be stored in the cache because rules set to warn were not run. If the user then runs just eslint --cache without --quiet, results pulled from the cache can be wrong for this command because they don't include warnings that the code might have.
I don't have a concrete solution at the moment, just wanted to note that we likely need to address this.

I think this behavior is acceptable. If you're running ESLint with different command line flags, it seems like different results should be expected? But I'm guessing most people don't change the flags they're using very often.

Okay, I think it's acceptable but it would be better to avoid it if possible. Something like @me4502 suggested in #17274 (comment) might be the solution. However, given the complexity, I think this should be revisited at some later point (not in this PR). Also, it seems that the same problem exists with some other CLI options (e.g., --no-inline-config) so we could try to fix that altogether.

[mdjermanovic]

LGTM, thanks! Leaving open for @nzakas to verify.

[nzakas]

Can we rename to eslint.config-rule-throws.js? We generally don't have underscores in filenames.

[nzakas]

Because I'm blocked, I'm going to go ahead and merge this and then handle the file renaming myself.

Thanks so much for sticking with this feature @me4502! 🙏

[me4502]

Thanks :) Sorry I named the file with underscores as the ones around it had those too

[nzakas]

Yeah no worries, I think those snuck in at some point so I understand the confusion.