Scripts: Jest error when running e2e tests

[swissspidy]

Description

For the Preferred Languages feature plugin I am currently using "@wordpress/e2e-test-utils": "^7.8.0", "@wordpress/env": "^4.9.0" and "@wordpress/scripts": "^23.4.0", with no custom configurations or anything.

When running wp-scripts test-e2e all my tests now fail with the following error:

    TypeError: Cannot read property 'enableGlobally' of undefined
      at jestAdapter (node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:39:25)
      at TestScheduler.scheduleTests (node_modules/@wordpress/scripts/node_modules/@jest/core/build/TestScheduler.js:333:13)
      at runJest (node_modules/@wordpress/scripts/node_modules/@jest/core/build/runJest.js:404:19)
      at _run10000 (node_modules/@wordpress/scripts/node_modules/@jest/core/build/cli/index.js:320:7)
      at runCLI (node_modules/@wordpress/scripts/node_modules/@jest/core/build/cli/index.js:173:3)

Example run: https://github.com/swissspidy/preferred-languages/runs/7181995299?check_suite_focus=true

Step-by-step reproduction instructions

1. Check out the https://github.com/swissspidy/preferred-languages repo which is rather small and can be used to reproduce this issue
2. Build the plugin
3. Start up the wp-env
4. Try to run e2e tests
5. See Jest errors

Screenshots, screen recording, code snippet

No response

Environment info

No response

Please confirm that you have searched existing issues in the repo.

Yes

Please confirm that you have tested with all plugins deactivated except Gutenberg.

Yes

[swissspidy]

Seems like the issue is that @wordpress/scripts uses Jest v27 while all other packages use v28:

$ npm ls jest

├─┬ @wordpress/e2e-test-utils@7.8.0
│ └── jest@28.1.2
└─┬ @wordpress/scripts@23.4.0
  ├─┬ @wordpress/jest-preset-default@8.4.0
  │ ├─┬ @wordpress/jest-console@5.3.0
  │ │ └── jest@28.1.2 deduped
  │ └── jest@28.1.2 deduped
  └── jest@27.5.1

Overriding the version in package.json to force v28 helped:

	"overrides": {
		"jest": "28.1.2"
	},

[t-hamano]

Hi,
AFAIK Jest v28 is not yet supported and will need to be adjusted to v27.
Here is the related PR: #33287

I have forked your repository and tested it, and I think the following steps will work.

• remove node_modules/*, package-lock.json
• remove overrides property from package.json
• npm install
• npm run wp-env start
• npm run test:e2e

[swissspidy]

I have tried these steps before and it didn't work, you'll end up with the same situation of mixed Jest versions.

[t-hamano]

Can you test my repository with the forked and modified version?
At least it worked fine in my environment.
https://github.com/t-hamano/preferred-languages Deleted

[swissspidy]

Ah, you're using an older version of npm, that's why. I'm using npm 8.

[t-hamano]

Yes, node 16.x and npm 6.x would be recommended.

[t-hamano]

@swissspidy

Do you think limiting the node version is a fundamental solution to this issue?

I have confirmed that engines allow npm 8 in each package of gutenberg.
However, in the main package.json, the npm version is restricted as follows:

{
	"engines": {
		"npm": ">=6.9.0 <7"
	}
}

Should we introduce version restrictions in each package to prevent errors like this one?
If necessary, I would like to close this issue and open a new issue for discussion.
(I am not familiar with npm...)

I would like to hear your thoughts.

[swissspidy]

Limiting the npm version is just a hacky way to work around the Jest version constraint and would seriously hinder any modern project from using Gutenberg packages, so a big -1 from me on that approach.

IMO the "proper" solution is to ensure that all packages can easily use the same Jest version. Adding it to peerDependencies should work.

Of course updating the version constraint also helps.

Sharing the link to our Slack discussion for context: https://wordpress.slack.com/archives/C02QB2JS7/p1657609316323339

[t-hamano]

IMO the "proper" solution is to ensure that all packages can easily use the same Jest version. Adding it to peerDependencies should work.

Agreed.

If you don't mind, would it be possible to start a new issue?
You should be able to explain better than I can 😅

As far as the following PRs are concerned, the method of unifying with version 27 may be appropriate, since Jest's version upgrade has a large impact:

• Testing: Upgrade Jest to v27 #33287
• Scripts: Upgrade Jest to the new major version (26.x) #27956

[jordesign]

Closing this issue - as the PRs mentioned above seem complete, and there's been no further movement on this in 12 months.

[t-hamano]

I also think this issue should be closed. In the latest Gutenberg, all packages seem to depend on Jest version 29.

npm ls jest
gutenberg@16.6.0 /home/path/to/gutenberg
├─┬ @testing-library/react-native@12.1.2
│ └── jest@29.6.2 deduped
├─┬ @wordpress/e2e-test-utils@10.12.0 -> ./packages/e2e-test-utils
│ └── jest@29.6.2 deduped
├─┬ @wordpress/e2e-tests@7.12.0 -> ./packages/e2e-tests
│ └── jest@29.6.2 deduped
├─┬ @wordpress/jest-console@7.12.0 -> ./packages/jest-console
│ └── jest@29.6.2 deduped
├─┬ @wordpress/jest-preset-default@11.12.0 -> ./packages/jest-preset-default
│ └── jest@29.6.2 deduped
├─┬ @wordpress/jest-puppeteer-axe@6.12.0 -> ./packages/jest-puppeteer-axe
│ └── jest@29.6.2 deduped
├─┬ @wordpress/scripts@26.12.0 -> ./packages/scripts
│ └── jest@29.6.2 deduped
├─┬ eslint-plugin-jest@27.2.3
│ └── jest@29.6.2 deduped
├─┬ jest-watch-typeahead@2.2.2
│ └── jest@29.6.2 deduped
├── jest@29.6.2
└─┬ snapshot-diff@0.10.0
  └── jest@29.6.2 deduped