feat(rule-tester): allow to create empty tests

[azat-io]

PR Checklist

• Addresses an existing open issue: fixes Enhancement: Add the ability to skip valid or invalid tests #7481
• That issue was marked as accepting prs
• Steps in Contributing were taken

Overview

In some cases I only need to test for valid or invalid scenarios.

For example, if I want to check that a rule for the ESLint plugin works with different code stylings. Or that a rule does not apply to a certain file extension.

Example:

ruleTester.run(
  `${RULE_NAME}: works consistently with an empty array or an array with one element`,
  rule,
  {
    valid: ['[].includes(person)', "['Decim'].includes(bartender)"],
    invalid: [],
  },
)

This test is currently crashing with an error: Error: No test found in suite invalid

[typescript-eslint]

Thanks for the PR, @azat-io!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!

---

🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint.

[netlify]

✅ Deploy Preview for typescript-eslint ready!

Name	Link
🔨 Latest commit	50adaa0
🔍 Latest deploy log	https://app.netlify.com/sites/typescript-eslint/deploys/64f5038e23137900077a6b02
😎 Deploy Preview	https://deploy-preview-7467--typescript-eslint.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 50adaa0. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

---

✅ Successfully ran 40 targets

• nx test eslint-plugin --coverage=false
• nx build website
• nx test typescript-estree --coverage=false
• nx run types:build
• nx run repo-tools:postinstall-script
• Node 20 - nx test repo-tools
• Node 20 - nx run types:build
• Node 20 - nx test visitor-keys
• Node 20 - nx test type-utils
• Node 20 - nx test parser
• nx test eslint-plugin-tslint --coverage=false
• nx test eslint-plugin-internal --coverage=false
• Node 20 - nx test typescript-estree
• Node 20 - nx run repo-tools:postinstall-script
• Node 20 - nx test rule-schema-to-typescript-types
• Node 20 - nx test utils
• Node 20 - nx test scope-manager
• Node 20 - nx test eslint-plugin
• Node 20 - nx test eslint-plugin-tslint
• Node 20 - nx test eslint-plugin-internal
• nx run-many --target=typecheck --parallel
• Node 16 - nx test eslint-plugin-internal --coverage=false
• Node 16 - nx run types:build
• Node 16 - nx test visitor-keys --coverage=false
• nx run integration-tests:test
• Node 16 - nx run repo-tools:postinstall-script
• Node 20 - nx test ast-spec
• nx run eslint-plugin:generate:configs
• Node 16 - nx test parser --coverage=false
• nx run-many --target=lint --parallel
• Node 16 - nx test utils --coverage=false
• Node 16 - nx test typescript-estree --coverage=false
• Node 16 - nx test rule-schema-to-typescript-types --coverage=false
• Node 16 - nx test eslint-plugin --coverage=false
• Node 16 - nx test ast-spec --coverage=false
• Node 16 - nx test scope-manager --coverage=false
• Node 16 - nx test type-utils --coverage=false
• Node 16 - nx test eslint-plugin-tslint --coverage=false
• Node 16 - nx test repo-tools --coverage=false
• nx run-many --target=build --parallel --exclude=website --exclude=website-eslint

---

Sent with 💌 from NxCloud.

[JoshuaKGoldberg]

The code changes look nifty but let's discuss in a backing issue. The rule tester is an important part of the public API and I'm not sure we want to do this without talking it over first.

[azat-io]

I would like to be able to skip valid or invalid scenarios in some tests. In some cases, such examples are simply not needed.

Some simple example:

import noLetRule from '../rule/no-let'

ruleTester.run(
  `disallow to use let`,
  rule,
  {
    valid: [],
    invalid: [`let a = 1`],
  },
)

[azat-io]

Issue #7481

[azat-io]

@JoshuaKGoldberg Could you review this PR?

This pull request is the last step to upgrade to version 6 before the big release in our ESLint plugin: azat-io/eslint-plugin-perfectionist#32

The same PR in ESLint also was merged: eslint/eslint#17475

[JoshuaKGoldberg]

Changes look good, thanks for keeping on this!

Could you please add some unit tests for the new logic though? Conditionally skipping tests is a kind of behavior that's easy to trip up over time.

[azat-io]

@JoshuaKGoldberg Hmm. Can you tell me the best way to do that? Since this problem occurs only when using RuleTester with Vitest. Because of this: https://github.com/vitest-dev/vitest/blob/main/packages/runner/src/run.ts#L331-L338

[JoshuaKGoldberg]

See the tests at the bottom of RuleTester.test.ts in the constructor constraints block for reference:

typescript-eslint/packages/rule-tester/tests/RuleTester.test.ts

Line 762 in 6d3d162

describe('constructor constraints', () => {

They show running ruleTester.run('my-rule", ...) and then asserting on the mockedDescribe.mock. You could add a similar block with tests roughly like:

it('does not call describe with valid if no valid tests are provided', () => {
  const ruleTester = new RuleTester();

  ruleTester.run('my-rule', NOOP_RULE, {
    valid: [],
    invalid: [
      {
        code: 'invalid',
        errors: [{ messageId: 'error' }],
      },
    ],
  });

  expect(mockedDescribe.mock.calls) // ...

[azat-io]

@JoshuaKGoldberg Thanks. Done.

[JoshuaKGoldberg]

Awesome! I actually stumbled onto a want for this recently 😄 so it's very exciting to see it come in. Thanks as always @azat-io!