wp-scripts: add support for Playwright #53108
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
swissspidy
added
[Type] Enhancement
|
Size Change: 0 B Total Size: 1.51 MB ℹ️ View Unchanged
|
|
Thank you for starting this PR and exploring how we can add support for Playwright to
It sounds like a great idea. If folks are fine using the default config provided by
I see that we install browsers on CI with npx playwright install chromium firefox webkit --with-depsYou can replicate how we install browsers when running gutenberg/packages/scripts/scripts/test-e2e.js Lines 31 to 33 in 422d12a This way it magically works for the dev when they run it for the first time. My biggest concern was initially about adding new dependence for Playwright, while we have dependencies present for Puppeteer in 
Both tools have a similar impact, a little bit over 15 MB which isn't surprising as Playwright was forked from Puppeteer. The great news is that both are optimized and they don't install browsers together with npm packages. It all happens afterward lazily when you actually use the tool. At some point, we will have to revisit the architecture of As of today, when you install As for this PR, I guess it's better to start with a simpler approach and live for the time being with two concurrent tools bundled with Now, test-e2e could do what you suggested - detect a config and run a corresponding tool through these packages. We could even set these extracted packages as optional peer dependencies, and possibly even offer a prompt helping to install them on the fly. |
swissspidy
requested review from
kevin940726,
Mamaduka,
gziolo,
ntwb,
nerrad,
ajitbohra and
ryanwelcher
as code owners
gziolo
reviewed
Aug 18, 2023
gziolo
approved these changes
Aug 24, 2023
There was a problem hiding this comment.
This looks great as it stands for the @wordpress/scripts and @wordpress/eslint-plugin with all the remarks I shared in #53108 (comment).
@kevin940726 or @WunderBart, could you help verify that refactoring to ESLint rules and configs work as intended?
| @@ -16,6 +16,7 @@ test.describe( 'Nonce', () => { | |||
| await admin.createNewPost(); | |||
| await page.keyboard.press( 'Enter' ); | |||
| // Wait until the network is idle. | |||
| // eslint-disable-next-line playwright/no-networkidle | |||
There was a problem hiding this comment.
Someone could verify that these exceptions caught after the ESLint package upgrade should be refactored in the follow-up PR.
|
What's blocking this from reviewing / merging? |
|
npm packages got published to npm today, so I say we waited enough to gather additional feedback. We will have two weeks to adjust if necessary before another publishing window. |
|
Flaky tests detected in ab5ca23. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/6040775971
|
|
Thanks for the work! FWIW, I didn't want to move Playwright into I don't feel that strongly about this or we have to revert this though. Just want to share my opinions for future reference 🙂 . |
mikachan
added
[Type] Automated Testing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Adds
wp-scripts test-playwrightwith a default Playwright configuration for running e2e tests.Adds a new
test-playwrightconfig to@wordpress/eslint-plugin.Note
Idea: enhance
test-e2eto automatically detect whether the given configuration is for Puppeteer or Playwright. Needs to accommodate for setups using both together.Why?
Playwright is the future for e2e tests in WordPress. Adding support directly to wp-scripts makes it easier to use Playwright not just for cure, but for everyone.
How?
The setup really follows the one for
test-e2e, though things are a bit simpler because no Jest is involved.Testing Instructions
npm run test:e2e:playwrightandtest:performance:playwrightare changed in the PR to use wp-scripts and the tests still run & passYou can also run
./node_modules/.bin/wp-scripts test-playwrightand then provide a different config to run tests in another project.Testing Instructions for Keyboard
Screenshots or screencast