Component tests failing intermittently with uncaught Vite "failed to fetch dynamically imported module" error in CI #25913
Comments
|
We are having the exact same issue with component tests ( I was just coming here to open a new issue ) . We just migrated from webpack to vite. The error we get is:
We are getting this despite the fact that the file does exist. And this occurs randomly. Vite: 4.1.1 One related issue: our cypress directory is on the same level as our src directory. When running locally, Cypress correctly expects component.ts to be where we have it in the cypress directory. But when we run the tests in docker, Cypress expects component.ts to be in a cypress directory under the src directory (see above error). Even if we use the 'supportFolder' config setting, Cypress still looks for it in the src directory (src is the vite root folder). So I just copied component.ts to the location cypress expects it to be (but still fails randomly despite this). So locally this is not reproducable for us (so far). This only occurs in a docker container. We are also running the tests in parallel. |
|
Update: we found an ugly workaround by using https://docs.cypress.io/guides/guides/module-api#cypressrun and writing our own parallelization mechanism and runner which wraps Cypress. We are checking for errors and we will retry failed tests again. It works but it requires some "balance" in the number of workers. We also made the first group run first, and only when it's successful, we run the other groups in parallel. This seems to help a lot. But there are still random failures from time to time but because we retry them a few times, it will eventually settle down. It looks to be somehow related to "background" Vite's compilation & deps optimization because we observed that it usually gets stable once some of Vite's log messages appear. |
|
Is this only in Docker? We use Vite heavily internally - everything is really stable. I hope someone can reproduce it reliably. I wonder if adding entries to https://vitejs.dev/config/dep-optimization-options.html#optimizedeps-entries helps? |
Yes, it seems to be only happening on CI, which means Docker. And we tried It is somehow related to CI being under load. We observed a few more different errors happening. It just seems that loading of some resources is either failing or incomplete causing various strange issues (sometimes with loading the import, sometimes errors on Cypress level, sometimes in app, usually something about missing context). But all these are usually resolved after 1-3 retries, depending on the CI load. |
|
We are experiencing more or less exactly the same thing. In particular the missing context error from time to time and failing to load component.ts. |
|
Maybe related: #18124 (comment) The OP says "Latest Cypress + Vite" - is this suggesting this is a regression? If so, knowing which version introduced this regression would be very useful. I haven't seen this in our internal suite, but it sounds like I may be able to reproduce by adding lots of large dependencies, like MUI? Or by increasing the load, eg Docker? |
|
We use MUI. We have split approx 700 component tests into 8 groups of varying sizes which run in parallel in docker containters. The machine is sufficient for the load. Since writing the first post above we had 7 successful runs and then today we experienced the same failure across several of the groups. This time it was component.ts in one group, but also random test files in other groups. I tried the suggestion that @synaptiko gave to run one small group first before running the other tests in parallel. I have since had two successful test runs in a row. Hopefully this will stabalize the build more, but I have seen in the past that making a change can can improve things temporarily only to have a regression. I have also tried running all the tests sequentially and get the same error btw. |
We are transitioning from CRA and much older Cypress, so I can really say if it's regression or not. We just started using versions from the last week.
This correlates with our observations. We have MUI but also Syncfusion and a few other bigger dependencies. Our CI uses quite powerful machines but we are running a lot of things in parallel, the load changes over the day, which would explain why it happens randomly (but we observed it's like ~50% of cases). |
|
For anyone interested, here's our "workaround" solution. We have a script called #!/usr/bin/env node
import { Chalk } from 'chalk';
import cypress from 'cypress';
import fs from 'fs';
import path, { dirname } from 'path';
import { fileURLToPath } from 'url';
const chalk = new Chalk({ level: 1 });
const groupId = process.argv[2];
if (groupId === undefined) {
console.error('Please provide a group id parameter');
process.exit(1);
}
const __dirname = dirname(fileURLToPath(import.meta.url));
const groups = JSON.parse(fs.readFileSync(path.join(__dirname, '..', '..', 'component-test-groups.json')));
const group = groups[groupId - 1];
const maxRetries = 5;
let specs = group.join(',');
let shouldFail = false;
let shouldRetry = true;
let retries = 0;
while (shouldRetry) {
if (retries > 0) {
console.log(chalk.bgRed(`Retry number: ${retries}`));
console.log(chalk.bgRed('Retrying failed tests:'));
console.log(
chalk.red(
specs
.split(',')
.map((spec) => `- ${spec}`)
.join('\n')
)
);
console.log();
}
const result = await cypress.run({
spec: specs,
browser: 'chrome',
headless: true,
testingType: 'component',
});
const failedTests = result.runs.reduce((failed, { tests, spec }) => {
return [...failed, ...tests.filter(({ state }) => state === 'failed').map((test) => ({ ...test, spec }))];
}, []);
specs = failedTests
.map(({ spec }) => spec.relative)
.reduce((result, spec) => {
if (!result.includes(spec)) result.push(spec);
return result;
}, [])
.join(',');
if (failedTests.length === 0) {
shouldRetry = false;
} else if (retries >= maxRetries) {
shouldRetry = false;
shouldFail = true;
} else {
retries++;
}
}
if (shouldFail) {
console.log(chalk.bgRed("Couldn't recover these failing tests:"));
console.log(
chalk.red(
specs
.split(',')
.map((spec) => `- ${spec}`)
.join('\n')
)
);
process.exit(1);
} else {
console.log(chalk[retries === 0 ? 'bgBlue' : 'bgRed'](`Finished running tests with ${retries} retries`));
}Under the same path we also have another script which collects paths of all our component test specs and "groups" them randomly, it looks like this: #!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const glob = require('glob');
const groupCount = process.argv[2];
if (groupCount === undefined) {
console.error('Please provide a group count parameter');
process.exit(1);
}
glob('src/**/*.cytest.tsx', (err, files) => {
if (err) {
console.error('An error occurred:', err);
process.exit(1);
}
console.log('Found', files.length, 'files');
// sort files randomly
files.sort(() => Math.random() - 0.5);
// take 1/3 of the files and put them in the first group to warm up Vite's cache
const firstGroupAmount = Math.ceil(files.length / groupCount / 3);
const groups = [files.splice(0, firstGroupAmount)];
// ignore the first group in this loop
for (let i = 0; i < groupCount - 1; i++) {
groups.push(files.slice((i * files.length) / (groupCount - 1), ((i + 1) * files.length) / (groupCount - 1)));
}
console.log(
'Created',
groups.length,
'groups with following distribution:',
groups.map((group) => group.length)
);
fs.writeFileSync(path.join(__dirname, '..', '..', 'component-test-groups.json'), JSON.stringify(groups, null, 2));
});On CI we have a following setup of steps:
Our whole CI flow looks like this: I hope that someone will find this useful until the root cause gets fixed. |
|
Wow, what a hack - the fact you needed to do that really isn't ideal, I hope we can isolate and fix this soon. I wonder if we need to reach out to the Vite team - they'd probably have more insight than we would for the Vite internals. Seems a lot of similar issues in Vite: https://github.com/vitejs/vite/issues?q=is%3Aissue+Failed+to+fetch+dynamically+imported+module
FYI we do the import here: I wonder if we can add some pre-optimzation logic during CI mode to force Vite to pre-compile all dependencies, side stepping this issue entirely (which is sort of what the workarounds here are doing). |
|
Does anyone know any large OSS React + Vite + MUI projects I could try using to reproduce? I tried moving MUI core to Vite but it's not straight forward. |
|
This seems like a duplication of this thread so writing here also: I have tried to run the tests: locally, on our Github Actions CI, cypress IO cloud and currents.dev(cypress cloud competitor). Out project is using React + Typescript. locally: Github Actions: Cypress.io currents.dev(cypress alternative) Haven't mention it until now but on the cloud solution it does seem to work. which leads me to the conclusion that it is something with my environment that I'm doing wrong here is an example of a test I notice that fails more often ApproveRun.spec.cy.tsx @lmiller1990 regarding the complexity of the test - I'd say it is pretty complex and heavy test because we render almost the top component in a SPA application @Murali-Puvvada the bash scripts is a workaround. tests that should take 300ms for example with the cypress run command takes 1000-3000ms instead because it rerun the chrome instance every time. Also: we don't have cypress e2e in our application(only component tests.) I've uploaded the debug output to this public file(700mb log file) when I run all the tests: https://drive.google.com/file/d/1-2KOb6KV1SyOc_hBi2c2DK40gRtFeeop/view?usp=sharing Would love any help regarding the issue. here is the error I'm talking about: |
|
We're seeing this in Vuetify's CI too: https://github.com/vuetifyjs/vuetify/actions/runs/4354367755/jobs/7609586657 |
|
@KaelWD |
|
@lmiller1990 For us it's in docker and component testing. For months now... But what's weird: It only happens on CI/CD (GH actions) where we using exact same docker container as on devs computers. On PC (docker) it works always. I was using Vite from beginning, and had this issue for months, trying multiple Cypress, browers and Vite versions and any workaround I could find. Everything is up to date. It's hard to debug because like I said in never happens locally, just on GH. I was suspecting memory leak, but cypress/docker is running with stable ram usage, low for GH limits. What I tried recently was configuring "attempts" to tests, so even if it fails, it should work fine second time. But this is useless option in that scenario, because when Failed to fetch dynamically imported module happens, Cypress just stops running that test and second attempt never happens........ Recently I added 2 packages that constantly optimized itself to optimizationDeps.entries. And I think that helped a little, but random failed tests still occurs. Now I'm expirecing "failed to fetch" not component file, but cypress index file. |
|
@chojnicki for me it's happening a lot of constantly. Currently the only workaround I've found was this bash script which run the tests separately one by one and if it fails than it retries for two times. Its significantly slower, but it does the job to only fail when a test actually fails. (we're running this bash script in GH actions too) |
|
We have an internal repro 🔥 FINALLY! It's Cypress org private project, but dropping the link here so someone at Cypress can internally... https://cloud.cypress.io/projects/d9sdrd/runs/4193/test-results/2f924501-7fb2-4c80-b69f-819010c67c87 Now we've got a repro I can dig into it... for us it's CI only too, so sounds like a resources/race condition. |
|
Related: vitejs/vite#11804 |
|
FYI It seems I don't have a access to the link @lmiller1990 |
|
@lmiller1990 hope that repo you got will help, but if not, I think I could get access for you/cypress to our repo too. |
|
Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @jordanpowell88 |
|
@chojnicki thanks a lot, I'll ping you if we need access, I think our reproduction should be enough. Is your reproduction consistent? CI only? @matanAlltra sorry this reproduction repo is in our Cypress org but a private repo, I can't make it public it right now, but someone on our team will be able to see it and debug it. If anyone else can share a public repo with this issue, that would sure help, too! |
|
we don't use the |
|
the thing with |
|
I have this issue in local environment (not CI) when I launch a single test.
I solved the problem. In my case it was a wrong relative import '@/...' instead of '../../' inside the test file. |
|
We were seeing this issue in our monorepo, over 1k cyct working well with webpack, but intermittently hanging when running large suites, CI or local. That was with node 14. With node 18 the issue seems to have self healed. |
|
sorry @RadomirNowak haven't had time yet to test your workaround of @muratkeremozcan we use node |
|
We've been stuck on this for about 3 weeks, would really love to hear from anyone with a workaround. |
Just to let you know I am trying your workaround in CI currently, and I should be able to get back to you once its run a sufficient amount of times. |
jennifer-shehane
added
CI
|
Hi. I'm adam-thomas-privitar on my personal account :). So I wouldn't say that my confidence is 100% due to the odd nature of this issue. But I can say so far, the problem has not reoccurred in my branch since adding the test files to the I'm trying not to get my hopes up too much though since the real test will come when this gets merged to main next week and the build volume increases. So ultimately we still need to proceed with caution. I will report back, but I feel there's sufficient data to say that anyone experiencing this should try it so we can add more data points. If it does fix it, @RadomirNowak is a legend :D, as this problem has truly haunted us at scale. If it suddenly fails, I might go and have a little cry somewhere and hang up my keyboard forever. It's perhaps of note that we have our own substantial layer above cypress (test support stuff) that is imported only into tests, in its own package, to abstract common things, and so the proposition that this plays a part is very plausible. I believe nearly all of the discourse about solutions that seem to work then don't is likely due to varying load on the agent that's executing the tests. It's a race condition in its current state, and so any fluctuation in speed affects it, and I've observed this myself. Also, the problem gets worse the more deps you add. If you are running tests in parallel that definitely adds to the randomness also, since that's also causing varying load between runs. Another reason is if you have many agents and some are more highly specced than others. If it does emerge that this is the problem then I guess a PR to the Cypress vite integration package is needed to automatically merge the test files into |
|
Just tried adding {
optimizeDeps: {
entries: ["tests/e2e/**/*.ts", "src/**/*.{ts,vue}"],
},
}( And I'm still getting the import error. Also, I'm able to reproduce this outside of CI using the headless mode. |
|
@dyc3 I had the same issue - turned out I was not catching all files with all import - in my case it was also it could be also a different problem on your side, as I'm using React so maybe there's something different in Vue 🤔 |
|
So its been a couple of days now and the problem has not come back in CI. As others have said, I think this is about capturing everything, including any test-support stuff as well as the test files themselves. Makes sense support file should be added here as well because that is imported separately by cypress and not from the tests. |
|
Just a check in that over 2 weeks now the problem remains gone! |
|
Any updates on this? We tried the latest workarounds suggested in here but still get the error (less frequently, but it still happens at least once a complete component test run). Looks like its always the component.ts for us. |
|
@adamscybot optimizeDeps doesn't work for us. Still getting random errors on CI. |
|
Still a problem. |
|
I was encountering this and I think the message is a red herring. In switching between branches, I'd uninstalled @pinia/testing, which I frequently use on component tests. Once I reinstalled that, it worked. |
|
Chiming in here in the hopes of saving others some time. We had had intermittent issues with component tests failing in CI. The errors always came near a I was able to reproduce the issue locally by clearing optimizeDeps: {
entries: ['cypress/**/*'],
},to our That was fine for a while, but I just now had to address another similar issue, this time with a new dependency ( |
|
I experienced this too - for me it was because I was switching between branches with different dependencies, and one of them somehow hadn't been reinstalled inbetween changing branches (in my case @pinia/testing). So while Cypress was reporting that it couldn't import components.js, it wasn't because that file was missing, it was because that file was erroring because of the missing import of @pinia/testing within it. Hope this helps people who end up here in the future (or me, when this happens again and see my own comment). |
|
I was also unable to see any impact from manipulating the optimizeDeps option in the vite config, but I did want to document what worked for us in case it's useful to someone else as it turned out to be tertiary to the original issue documented. The issue for us turned out to be tied to CircleCI and the use of a static port in our vite config. Our CI config used the same executor to run both E2E tests and Component tests in parallel after a build/install job. The static port was required for the E2E tests to serve reliably for the cypress run, but Cypress component tests were also picking up the static port from the default vite config. Though I initially thought the executors would be isolated, this post made me reconsider. Explicitly overriding the port in our cypress config file fixed this issue and prevented the error from occurring due to host conflicts on the same port. import viteConfig from './vite.config.ts';
// override default config
const customViteConfig = {
...viteConfig({ mode: 'dev' }),
server: {
port: 3001,
},
};
...
// add configuration for component dev server
component: {
devServer: {
framework: 'react',
bundler: 'vite',
viteConfig: customViteConfig,
},
... |
|
I am facing same issue |
|
Running the tests in chrome fixed it for me Update: fixed by excluding |
|
Fixed by deleting node_modules then |
and others
Current behavior
We started migrating our project from CRA to Vite and we have almost 300 component and 40 E2E Cypress tests in place. Unfortunately, after fixing all the other issues, we are still not able to stabilize our tests since there is always 1 or 2 failing randomly on "Failed to fetch dynamically imported module" errors.
We noticed that it's somehow related to the load of CI. Under some conditions more tests are failing like this, in other times it succeeds. But it's random. We checked our tests and we are pretty sure it's not caused by any logic we have.
We've checked some of the existing issues on Cypress & Vite, tried various workaround but no luck with any of them.
What we think is happening is that Cypress is not waiting for Vite to "boot up" properly and retries don't help with it, only when new spec is run, it works.
Note: it only happens with component tests. For E2E tests we had similar stability issues but we solved them by building production version and then just serving it with
vite preview. This made integration tests faster and very stable. Previously they were also timeouting.Note 2: we have a lot of components and lot of dependencies in our project, we also use MUI as our base. But with CRA we were able to have stable tests, it was just around two times slower. That's why we want to use Vite now.
Note 3: we are running "Component tests" in parallel, currently in 4 groups.
Desired behavior
No random problems with Cypress + Vite combo.
Test code to reproduce
Unfortunately can't provide this since our project is private. And I'm afraid that it's related to project's complexity and that's why we can't easily create another one for reproduction.
Cypress Version
v12.6.0
Node version
v16.18.1
Operating System
Docker Hub image: cypress/included for v12.6.0 version
Debug Logs
No response
Other
No response
The text was updated successfully, but these errors were encountered: