feat(vitest): add github actions reporter

[hi-ogawa]

Description

• closes Native GitHub Annotations support when run on CI #4919

The idea is to reuse the entire printError util used by "default" reporter:

vitest/packages/vitest/src/node/reporters/base.ts

Line 371 in 8cc3eef

await this.ctx.logger.printError(error, { project })

It looks like this mostly works with minor adjustment/fix of printError side (though ideally it might make sense to refactor current printError to cover this kind of use case better).

There are a few minor differences from sapphi-red's vitest-github-actions-reporter:

• @actions/core dependencies are removed since it's only used for simple command formatting utilities (e.g. ::error ...). I only copied minimally necessary code from https://github.com/actions/toolkit/tree/main/packages/core, which is also what Jest did in their reporter.
• error formatting (e.g. assertion diff, stacktrace (filtering and finding relevant stack)) are automatically handled by reusing printError util.

example output

In this commit c89aa5f, I tested demo test cases on CI and it looks like this:

Reveal screenshot

Also comparison to default reporter:

Also the same annotations appear in the summary https://github.com/vitest-dev/vitest/actions/runs/7749165463

todo / tbd

• test unit
• test on github
• doc
• does workspace sub-directory work?
  • it should be fine since full file path is in the output and github will figure it out.
• enable/disable automatically by detecting GITHUB_ACTIONS env var?
  • jest requires github-actions reporter explicitly but disable it automatically if GITHUB_ACTIONS is not defined. I did the same here.
• jest now also does "log folding". it might be nice to have but it looks more tricky (for example, what if tests are in parallel?) I'll explore it separately later Add new default reporter for github actions jestjs/jest#13626
• enable it on Vitest's own CI?

references

• https://github.com/sapphi-red/vitest-github-actions-reporter
• https://github.com/actions/toolkit/blob/f1d9b4b985e6f0f728b4b766db73498403fd5ca3/packages/core/src/command.ts
• https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-error-message
• https://github.com/jestjs/jest/blob/38bb7982d7627ba8376815562bc35599051ce218/packages/jest-reporters/src/GitHubActionsReporter.ts
• https://jestjs.io/docs/configuration#github-actions-reporter
• Add new default reporter for github actions jestjs/jest#13626
• Small improvements to bun test  oven-sh/bun#3071

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
• Ideally, include a test that fails without this PR but passes with it.
• Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

• Run the tests with pnpm test:ci.

Documentation

• If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

• Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

[netlify]

✅ Deploy Preview for fastidious-cascaron-4ded94 ready!

Name	Link
🔨 Latest commit	4f43d5a
🔍 Latest deploy log	https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/65c2dd0330ec360008dfafba
😎 Deploy Preview	https://deploy-preview-5093--fastidious-cascaron-4ded94.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[sheremet-va]

Should we enable it by default if GitHub Actions env is found? What does Bun do in this situation for example? The problem here is that it is impossible to disable then 🤔

[hi-ogawa]

Should we enable it by default if GitHub Actions env is found? What does Bun do in this situation for example? The problem here is that it is impossible to disable then 🤔

So far I've been checking only Jest as a comparison (and make it somewhat compatible). Good point, I'll also check how Bun does this.

[hi-ogawa]

Indeed, Bun automatically enables github actions style output based on GITHUB_ACTIONS=true. It was introduced by this PR:

• Small improvements to bun test  oven-sh/bun#3071

To opt out, users would probably need to manually set GITHUB_ACTIONS=false or unset it on their CI workflow. I don't think Vitest can introduce this feature in this way, so I think it's better to make it opt-in like Jest (at least for starter).

To me, automatically disabling by checking non-existence of GITHUB_ACTIONS already felt not so intuitive, but I think I get this is a tiny nice-to-have trick since otherwise users would need to write a few lines to achieve this like in:

https://github.com/sapphi-red/vitest-github-actions-reporter?tab=readme-ov-file#usage

export default {
  test: {
    reporters: process.env.GITHUB_ACTIONS
      ? ['default', "github-actions"]
      : 'default'
  }
}

[sheremet-va]

I don't think Vitest can introduce this feature in this way, so I think it's better to make it opt-in like Jest (at least for starter).

Why do you think we cannot introduce it? Is it a breaking change?