Add testifylint linter

[Antonboom]

Hi!

https://github.com/Antonboom/testifylint

1. Supports analysis.SuggestedFix (but no integration in golangci-lint? Add support for SuggestedFixes #1779)
2. Supports analysis.Diagnostic.Category & URL (but no usage of this feature in golangci-lint?)

P.S. For local dev you need 1.21, but the module itself I returned to 1.20 by replacing useful package (slices) with own code 😢

---

Recreated from #4072, because of

[ldez]

In order for a pull request adding a linter to be reviewed, the linter and the PR must follow some requirements.

• The CLA must be signed

Pull Request Description

• It must have a link to the linter repository.
• It must provide a short description of the linter.

Linter

• It must not be a duplicate of another linter or a rule of a linter. (the team will help to verify that)
• It must have a valid license (AGPL is not allowed) and the file must contain the required information by the license, ex: author, year, etc.
• The linter repository must have a CI and tests.
• It must use go/analysis.
• It must have a valid tag, ex: v1.0.0, v0.1.0.
• It must not contain init().
• It must not contain panic().
• It must not contain log.fatal(), os.exit(), or similar.
• It must not modify the AST.
• It must not have false positives/negatives. (the team will help to verify that)
• It must have tests inside golangci-lint.

The Linter Tests Inside Golangci-lint

• They must have at least one std lib import.
• They must work with T=<name of the linter test file>.go make test_linters. (the team will help to verify that)

.golangci.reference.yml

• The linter must be added to the list of available linters (alphabetical case-insensitive order).
  • enable and disable options
• If the linter has a configuration, the exhaustive configuration of the linter must be added (alphabetical case-insensitive order)
  • The values must be different from the default ones.
  • The default values must be defined in a comment.
  • The option must have a short description.

Others Requirements

• The files (tests and linter) inside golangci-lint must have the same name as the linter.
• The .golangci.yml of golangci-lint itself must not be edited and the linter must not be added to this file.
• The linters must be sorted in the alphabetical order (case-insensitive) in the Manager.GetAllSupportedLinterConfigs(...) and .golangci.reference.yml.
• The load mode (WithLoadMode(...)):
  • if the linter doesn't use types: goanalysis.LoadModeSyntax
  • goanalysis.LoadModeTypesInfo required WithLoadForGoAnalysis() in the Manager.GetAllSupportedLinterConfigs(...)
• The version in WithSince(...) must be the next minor version (v1.X.0) of golangci-lint.
• WithURL() must contain the URL of the repository.

Recommendations

• The linter should not use SSA. (SSA can be a problem with generics)
• The linter repository should have a readme and linting.
• The linter should be published as a binary. (useful to diagnose bug origins)

---

The golangci-lint team will edit this comment to check the boxes before and during the review.

The code review will start after the completion of those checkboxes (except for the specific items that the team will help to verify).

If the author of the PR is a member of the golangci-lint team, he should not edit this message.

This checklist does not imply that we will accept the linter.

[ldez]

I will answer here about this #4072 (comment)

Yes, some linters have historically init/panic because they were not designed to be used as a lib.

Not problem, but just for clarification – why the linter doesn't cover PresetStyle and PresetMetaLinter?

not PresetStyle: because the reports of your linter are not (mainly) related to the "taste" of a user or just styling.

not PresetMetaLinter: because you are handling only one topic: testify.

In this case, the tests will not work because the linter ignores non-test files for performance and sanity reasons.

The choice of the scope should rely on golangci-lint configuration if possible.
If I create a dedicated test package, I will not suffix the file with _test because I want to use it outside of the package but your linter can also be useful in this context. Also, golangci-lint controls what the linter will analyze or not.

[Antonboom]

@ldez, thank u for feedback

If I create a dedicated test package

Nice point, I completely forgot about this case.

Summarizing:

1. No init, no panics, analyze all files. Changelog here.
2. T=testifylint.go make test_linters

Output

GL_TEST_RUN=1 go test -v ./test -count 1 -run TestSourcesFromTestdata/testifylint.go
# github.com/golangci/golangci-lint/test.test
ld: warning: -bind_at_load is deprecated on macOS
=== RUN   TestSourcesFromTestdata
    linters_test.go:21: testdata/*.go
=== RUN   TestSourcesFromTestdata/testifylint.go
=== PAUSE TestSourcesFromTestdata/testifylint.go
=== CONT  TestSourcesFromTestdata/testifylint.go
level=info msg="[test] ran [/Users/anthony/golangci-lint/golangci-lint run --internal-cmd-test --no-config --allow-parallel-runners --disable-all --out-format=json --max-same-issues=100 -Etestifylint testifylint.go] in 979.000166ms"
level=info msg="[test] ran [/Users/anthony/golangci-lint/golangci-lint run --internal-cmd-test --no-config --allow-parallel-runners --disable-all --out-format=json --max-same-issues=100 -Etypecheck -Etestifylint testifylint.go] in 294.706667ms"
--- PASS: TestSourcesFromTestdata (10.18s)
    --- PASS: TestSourcesFromTestdata/testifylint.go (1.28s)
=== RUN   TestSourcesFromTestdataSubDir
--- PASS: TestSourcesFromTestdataSubDir (0.00s)
PASS
ok      github.com/golangci/golangci-lint/test  12.382s

3. .golangci.reference.yml updated
4. Run on large projects and happy with the result

testifylint --enable-all -json ./... | jq '.[] | .[] | .[].category' | sort | uniq -c

# Consul
  30 "bool-compare"
 183 "compares"
 295 "empty"
  11 "error-is-as"
 463 "error-nil"
 636 "expected-actual"
  54 "float-compare"
  30 "len"
 106 "require-error"
 181 "suite-dont-use-pkg"

# Grafana
 145 "bool-compare"
  17 "compares"
 152 "empty"
  36 "error-is-as"
 737 "error-nil"
 885 "expected-actual"
 160 "float-compare"
 384 "len"
 362 "require-error"

# Kubernetes
  34 "bool-compare"
  16 "compares"
  46 "empty"
 119 "error-nil"
  74 "expected-actual"
   5 "float-compare"
  89 "len"
 502 "require-error"

Based on the analysis I opened Antonboom/testifylint#5 but it doesn't seem to be a blocker.

[ldez]

LGTM