Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

webpack@4, node@13, Mac crashes when compiler.watch().close() #10037

Closed
AviVahl opened this issue Nov 26, 2019 · 41 comments
Closed

webpack@4, node@13, Mac crashes when compiler.watch().close() #10037

AviVahl opened this issue Nov 26, 2019 · 41 comments

Comments

@AviVahl
Copy link

AviVahl commented Nov 26, 2019

Bug report

When closing a watching compiler programmatically, the node process crashes.

What is the current behavior?
Getting the following crash in several repositories on our Mac CIs:

Assertion failed: (0), function uv_close, file ../deps/uv/src/unix/core.c, line 187.
error Command failed with signal "SIGABRT".

If the current behavior is a bug, please provide the steps to reproduce.

const webpack = require('webpack');
const compiler = webpack({ mode: 'development' });

function createWatcher() {
    const watcher = compiler.watch({}, () => {
        console.log('watching');
        setTimeout(() => {
            watcher.close(() => {
                console.log('closed watcher');
                createWatcher();
            });
        }, 1000);
    });
}

createWatcher();
  • node a.js

What is the expected behavior?
(for the above code) watch -> wait for 1 sec -> close watcher -> rewatch and repeat process.

Other relevant information:
webpack version: 4.41.2
Node.js version: 13.2.0
Operating System: Mac 10.12.6

@sokra maybe this is because watchpack is using old chokidar with outdated fsevents? :/

@MarcGaumont
Copy link

Have the exact same problem...

@AviVahl
Copy link
Author

AviVahl commented Dec 28, 2019

To be honest, I wanted to re-check whether this is still an issue with fsevents@1.2.11 (which added Node 12 compatibility)... I use a Linux machine, so I need to get my hands on a Mac.

@AviVahl
Copy link
Author

AviVahl commented Dec 29, 2019

I've re-checked the above instructions... still crashes on Mac and Node 13.

@alexander-akait
Copy link
Member

alexander-akait commented Dec 30, 2019

What is version node? 13.0, 13.1, other? Looks like bug on node side

@PandaWhisperer
Copy link

Checking in from the Nuxt.js issue that was just linked. Node.js 13.5.0 with macOS 10.15.2 here.

@KrimoPixelsmith
Copy link

Same issue here node 13.2, npm 6.13.4, macOS 10.15.2

@alexander-akait
Copy link
Member

Can you try to update node to latest version?

@AviVahl
Copy link
Author

AviVahl commented Jan 20, 2020

just re-tested on Node 13.6.0 with latest webpack. re-produced the issue consistently.

Btw, I'm not certain it's a Node issue. Mac is the only platform where watchpack doesn't use native node APIs and instead uses fsevents (via chokidar). The issue could be there...

@KrimoPixelsmith
Copy link

KrimoPixelsmith commented Jan 20, 2020

@evilebottnawi just like @AviVahl said, it's the exact same; from what I could tell looking online, it looks like it's a chokidar/fsevents issue; I first noticed it in using https://github.com/JeffreyWay/laravel-mix, which uses chokidar version 2.0.x and webpack 4 still;

Looks like v3 is out...

@PandaWhisperer
Copy link

Can confirm. Updated to node 13.6.0 and the error still occurs.

@mahnunchik
Copy link

The same error. Node v13.6.0

@vjpr
Copy link

vjpr commented Feb 7, 2020

I am testing with v13.8 now.

@7iomka
Copy link

7iomka commented Feb 8, 2020

Assertion failed: (0), function uv_close, file ../deps/uv/src/unix/core.c, line 187.
Abort trap: 6
$ node -v
v13.6.0

:(

@DevJonTaylor
Copy link

node v13.8.0
macOS 10.15.2 (19C57)
When hot-loading an update it does not crash with this error.
I tested with scss-node | html | and javascript using *.js, *.vue, and *.scss files.

However when closing the node I receive the error again.

 DONE  Compiled successfully in 1153ms                                                                                                      9:15:29 AM


  App running at:
  - Local:   http://localhost:8080/ 
  - Network: http://192.168.1.11:8080/

^CAssertion failed: (0), function uv_close, file ../deps/uv/src/unix/core.c, line 187.
zsh: abort      npm run serve
jontaylor@Jonathans-MacBook-Pro portfolio % 

@sokra
Copy link
Member

sokra commented Feb 10, 2020

For webpack 4 we can't upgrade to chokidar 3, as this would be a breaking change. This would need to be fixed in a chokidar for webpack 4 to benefit from it. I also see these errors randomly occuring in the CI build.

For webpack 5 we migrated away from chokidar and native dependencies. I no longer see these errors in the webpack CI builds.

@kelly-tock
Copy link

i'm getting this as well on node 13.8 with karma and karma webpack. was fine on node v11.12

@Nikodermus
Copy link

Failing as well

node v13.8.0
macos mojave 10.14 (18A391)

@gansdot
Copy link

gansdot commented Mar 15, 2020

Am also getting the below error. npm server crashes after few mins connecting to mongo db. any idea why this is ?

Assertion failed: (0), function uv_close, file ../deps/uv/src/unix/core.c, line 187.
zsh: abort npm run dev

@alexander-akait
Copy link
Member

Anyone experiencing a problem please read - fsevents/fsevents#314

Briefly - not fixable, not sure if even I send a fix for it, it will be merged.

Good news - webpack@5 doesn't use fsevents, so no problems

@AviVahl
Copy link
Author

AviVahl commented Apr 9, 2020

That's a shame, because this started failing with the newly released Node 12.16.2 as well. :(

@alexander-akait
Copy link
Member

@AviVahl You can open a new issue, maybe when the number of issues becomes more, they will change their minds

@aminebenkeroum
Copy link

Same experience : Node v13.8.0, Mac Catalina OS

@slgarkea
Copy link

slgarkea commented Apr 20, 2020

Edit: nope, this not work. This is weird, some build finishes successfully randomly.


I fixed it by forcing chokidar in his last version :

  "resolutions": {
    "chokidar": "3.3.1"
  }

No anormal behaviour seen so far.

@edi
Copy link

edi commented Apr 28, 2020

Same here (using Expo bare-minimum).
Node v13.6.0
Mac Catalina OS

@allaniftrue
Copy link

I'm using node v12.16.3 (npm v6.14.4) and i still get the same error

Assertion failed: (0), function uv_close, file ../deps/uv/src/unix/core.c

@AviVahl
Copy link
Author

AviVahl commented May 7, 2020

Issue got replaced with another:

> fsevents@1.2.13 install /Users/runner/runners/2.169.1/work/mocha-pup/mocha-pup/node_modules/watchpack/node_modules/fsevents
> node install.js

internal/modules/cjs/loader.js:638
    throw err;
    ^

Error: Cannot find module 'nan'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at [eval]:1:1
    at Script.runInThisContext (vm.js:122:20)
    at Object.runInThisContext (vm.js:329:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at evalScript (internal/bootstrap/node.js:590:27)
gyp: Call to 'node -e "require('nan')"' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/Users/runner/hostedtoolcache/node/10.20.1/x64/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Darwin 19.4.0
gyp ERR! command "/Users/runner/hostedtoolcache/node/10.20.1/x64/bin/node" "/Users/runner/hostedtoolcache/node/10.20.1/x64/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/runner/runners/2.169.1/work/mocha-pup/mocha-pup/node_modules/watchpack/node_modules/fsevents
gyp ERR! node -v v10.20.1
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 

Happens for mocha-pup for all node 10/12/14 on Mac.

The new issue appears to already be documented here: fsevents/fsevents#325

Now's the twist. when I tried to reproduce in a separate project using Node v12.16.3 (npm v6.14.4), I've encountered what seems to be a bug in how npm handles optional dependencies.

If I:

mkdir testing
cd testing
npm init -y
npm i webpack

And then look in the package-lock.json file, I can see that fsevents has been resolved to v1.2.13, and that it requires nan:

    "fsevents": {
      "version": "1.2.13",
      "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-1.2.13.tgz",
      "integrity": "sha512-oWb1Z6mkHIskLzEJ/XWX0srkpkTQ7vaopMQkyaEIoq0fmtFVxOthb8cCxeT+p3ynTdkk/RZwbgG4brR5BeWECw==",
      "optional": true,
      "requires": {
        "bindings": "^1.5.0",
        "nan": "^2.12.1"
      }

nan is also found is the lock file.

But if I run the same set of commands, just adding chokidar to the install command:

mkdir testing
cd testing
npm init -y
npm i webpack chokidar

Then the package-lock.json contains:

        "fsevents": {
          "version": "1.2.13",
          "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-1.2.13.tgz",
          "integrity": "sha512-oWb1Z6mkHIskLzEJ/XWX0srkpkTQ7vaopMQkyaEIoq0fmtFVxOthb8cCxeT+p3ynTdkk/RZwbgG4brR5BeWECw==",
          "optional": true
        },

Without the "requires", and without nan appearing anywhere in the lock file.

It seems the existence of another fsevents version in the tree affects how the dependencies of fsevents@1 are resolved by npm, leading to the missing nan install errors.

The above behavior reproduces consistently (try it yourself) and leads me to believe that depending on which dependencies are in your project, you might experience one of two behaviors:

  • Error during postinstall of fsevents just on Mac (on all node versions), which doesn't fail npm install, being an optionalDependency. There would be no runtime crashes, as fsevents isn't used.
  • No errors during postinstall anywhere, but runtime crash when programmatically closing the watch compiler on Node@14/Mac and maybe on latest Node@12 as well.

This also accounts for why some people no longer see the crash, and others do (in comments above).

@paulmillr
Copy link

You should ask maintainers to upgrade to fsevents v2.

@AviVahl
Copy link
Author

AviVahl commented May 8, 2020

@paulmillr already answered is in this thread: #10037 (comment)

webpack@4 needs to support node@6, which newer chokidar/fsevents does not. I can't really argue with that need. I'm just stuck until webpack releases v5 and the loader/plugin ecosystem adapts to the new version.

Well, TBH, not really stuck... we've added workarounds in several of our code-bases to force no watching when in "single-run" mode... :/ This can get a bit hairy, because sometimes it's abstractions like webpack-dev-middleware which call the watcher. We ended up monkey patching the webpack's compiler so every call to .watch() will call .run() instead:
https://github.com/wixplosives/engine/blob/v13.0.2/packages/scripts/src/application.ts#L186
https://github.com/wixplosives/mocha-pup/blob/v3.0.0/src/run-tests.ts#L43

These are just two examples from open source projects.

I hate these workarounds, but we are already using Node 12 in production and started supporting 14. We cannot be held back by such issues.

@paulmillr
Copy link

@AviVahl not really. There is an effort to upgrade to chokidar 3, maintainers merged pull request/s already. Ask them to release new versions asap.

@AviVahl
Copy link
Author

AviVahl commented May 16, 2020

Alright, so the latest releases of watchpack@1 should fix the crash. How, might you ask? It's installing two versions of chokidar with a try/catch to each (newer version gets higher priority). I'll save what I really think about this approach, and just say the "postinstall" errors we get from chokidar@2 are far from ideal. :(

@sokra
Copy link
Member

sokra commented May 16, 2020

@AviVahl I also dislike that. Technically chokidar2 should only be installed when node<8, but npm seem to ignore that. Works fine in yarn.
We will get rid of that in the next major version, but for semver reasons we need to support node 6 in watchpack 1 and webpack 4. (I know that it is out of lifetime, but that doesn't allow us to break semver here)

@paulmillr
Copy link

@sokra see this comment on "removing chokidar in next major version". The file watching task is not as simple as it looks.

@AviVahl
Copy link
Author

AviVahl commented May 17, 2020

@sokra both npm@6.14.4 (comes with latest node 12/14) and yarn@1.22.4 (current "latest" for yarn) warn about and install duplicate chokidar and fsevents versions. I initialized an empty project and just npm i watchpack / yarn add watchpack. So the statement about any of them being ignored may be incorrect.

Alternative possible solution:
Re-release latest webpack@4 as 5, breaking just min Node version. Re-direct current 5 to 6. Supporting it in loaders/plugins would be easy, as there would be no other breaking changes. This is probably the "proper" solution, and will allow webpack to both keep Node 6 support and have clean Node 12/14 support.

All this juggling of dependency versions is starting to get out of hand. You end up using older versions of packages just to keep supporting an unsupported Node version.

Meanwhile, consumer projects using LTS versions of Node end up having duplicate dependencies, weird post-install errors, and failing CI builds due to runtime errors (original issue).

I mean, we've been dealing with this for quite a while now. Consumer projects started having the original issue over 6 months ago.

Having said the above, I wanna add that I appreciate your hard work, and will respect whatever decision you end up taking here.

@briandipalma
Copy link

We're not going to run out of numbers any time soon, releasing Webpack 5 just to drop node 6 support seems reasonable.

@AviVahl
Copy link
Author

AviVahl commented May 18, 2020

Alright, I am closing this issue, as the reported crash should now be fixed (make sure your project has watchpack resolved to v1.7.2 or above).

There's a separate new issue (postinstallation errors) which is expected to resolve itself with webpack@5.

@AviVahl AviVahl closed this as completed May 18, 2020
cowboyd added a commit to cowboyd/msw that referenced this issue Jul 27, 2020
Motivation
----------

fixes mswjs#188

The current version of webpack that we're using has [a bug][1] that
causes the tests to crashes on MacOS with some regularity. That is
because it depends on chokidar 2 which uses the `fsevents` api which
has a [crictical bug][2] on MacOS.

Approach
--------

This manually forces all modules to use chokidar 3 which does not use
the `fsevents` module at all. While this does trigger a semver
violation because webpack 4 depends on chokidar 2, the API is
apparently similar enough that all the tests pass. It does however
give rise to a warning in the console upon installation of depedencies
via a `yarn`.

This resolution can be removed once an upgrade is made to webpack 5 or
to a different depedency bundler like [parcel][3] or [rollup][4].

Learning
--------

* [Selective dependency resolutions ][5]

[1]: webpack/webpack#10037
[2]: fsevents/fsevents#314
[3]: http://parceljs.org
[4]: https://rollupjs.org
[5]: https://classic.yarnpkg.com/en/docs/selective-version-resolutions/
kettanaito pushed a commit to cowboyd/msw that referenced this issue Jul 28, 2020
Motivation
----------

fixes mswjs#188

The current version of webpack that we're using has [a bug][1] that
causes the tests to crashes on MacOS with some regularity. That is
because it depends on chokidar 2 which uses the `fsevents` api which
has a [crictical bug][2] on MacOS.

Approach
--------

This manually forces all modules to use chokidar 3 which does not use
the `fsevents` module at all. While this does trigger a semver
violation because webpack 4 depends on chokidar 2, the API is
apparently similar enough that all the tests pass. It does however
give rise to a warning in the console upon installation of depedencies
via a `yarn`.

This resolution can be removed once an upgrade is made to webpack 5 or
to a different depedency bundler like [parcel][3] or [rollup][4].

Learning
--------

* [Selective dependency resolutions ][5]

[1]: webpack/webpack#10037
[2]: fsevents/fsevents#314
[3]: http://parceljs.org
[4]: https://rollupjs.org
[5]: https://classic.yarnpkg.com/en/docs/selective-version-resolutions/
kettanaito pushed a commit to mswjs/msw that referenced this issue Jul 28, 2020
Motivation
----------

fixes #188

The current version of webpack that we're using has [a bug][1] that
causes the tests to crashes on MacOS with some regularity. That is
because it depends on chokidar 2 which uses the `fsevents` api which
has a [crictical bug][2] on MacOS.

Approach
--------

This manually forces all modules to use chokidar 3 which does not use
the `fsevents` module at all. While this does trigger a semver
violation because webpack 4 depends on chokidar 2, the API is
apparently similar enough that all the tests pass. It does however
give rise to a warning in the console upon installation of depedencies
via a `yarn`.

This resolution can be removed once an upgrade is made to webpack 5 or
to a different depedency bundler like [parcel][3] or [rollup][4].

Learning
--------

* [Selective dependency resolutions ][5]

[1]: webpack/webpack#10037
[2]: fsevents/fsevents#314
[3]: http://parceljs.org
[4]: https://rollupjs.org
[5]: https://classic.yarnpkg.com/en/docs/selective-version-resolutions/
kubkon pushed a commit to kubkon/fluentui that referenced this issue Jan 18, 2021
The fix turns out to be bumping `webpack` from `4.43` to `4.44` in
`packages/web-components`. Prior to this, I'd experience sporadic
errors when running `yarn test` like the ones documented in this
`karma` issue: [karma-runner/karma#3511](karma-runner/karma#3511).
It turns out, that the problem is not with `karma` but rather with
`webpack` as documented in [webpack/webpack#10037](webpack/webpack#10037).
As per the closing comment in that issue, ensure we use `watchpack`
version `^1.7.2` (a transitive dep of `webpack`) is enough to ensure
this sporadic error goes away, and to bump `watchpack` we need to
bump `webpack` by one minor version.
kubkon pushed a commit to kubkon/fluentui that referenced this issue Jan 18, 2021
The fix turns out to be bumping `webpack` from `4.43` to `4.44` in
`packages/web-components`. Prior to this, I'd experience sporadic
errors when running `yarn test` like the ones documented in this
`karma` issue: [karma-runner/karma#3511](karma-runner/karma#3511).
It turns out, that the problem is not with `karma` but rather with
`webpack` as documented in [webpack/webpack#10037](webpack/webpack#10037).
As per the closing comment in that issue, ensure we use `watchpack`
version `^1.7.2` (a transitive dep of `webpack`) is enough to ensure
this sporadic error goes away, and to bump `watchpack` we need to
bump `webpack` by one minor version.
kubkon pushed a commit to kubkon/fluentui that referenced this issue Jan 19, 2021
The fix turns out to be bumping `webpack` from `4.43` to `4.44` in
`packages/web-components`. Prior to this, I'd experience sporadic
errors when running `yarn test` like the ones documented in this
`karma` issue: [karma-runner/karma#3511](karma-runner/karma#3511).
It turns out, that the problem is not with `karma` but rather with
`webpack` as documented in [webpack/webpack#10037](webpack/webpack#10037).
As per the closing comment in that issue, ensure we use `watchpack`
version `^1.7.2` (a transitive dep of `webpack`) is enough to ensure
this sporadic error goes away, and to bump `watchpack` we need to
bump `webpack` by one minor version.
ssaadh added a commit to ssaadh/zoomers-collab-timers that referenced this issue Oct 26, 2022
- There are two issues. One is: webpack/webpack#10037.
- Likely related alongside this, is the socket and frontend not connecting with one another. Was able to drill down to figure out the frontend is the issue. fsevents dependency and two versions on Mac seems to be messing things up

- Above seemed to point to needing to update to webpack 5 in order to use this on Macs.
- Updated most dependencies. Not socket.io-client (4.5.3), nanoid (4.0.0), winston (3.8.2) yet.
- React updated to 18.2 from 17. react-scripts from 4 to 5
- react-app-rewired installed for config-overrides to try to fix new bugs coming up. Installed os-browserify, browserify-zlib for this.

- I think I pointlessly installed react-router. Seems like that wasn’t around before. Downgraded from react-router and react-router-dom 6 to 5. Initially react-router-dom was at 5.3. Likely go back to that next commit and remove react-router.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests