Random errors : cannot start Firefox/FirefoxHeadless

[cedx]

Firefox : v67.0.3
OS : Ubuntu 19.04 and Windows 10
Karma : v4.1.0
Firefox launcher : v1.1.0
Node.js : v12.4.0

I've some projects whose Karma tests work as expected when I'm using the Chrome launcher. But when I use the Firefox launcher, I get the following error:

0 06 2019 19:37:21.060:INFO [karma-server]: Karma v4.1.0 server started at http://0.0.0.0:9876/
20 06 2019 19:37:21.062:INFO [launcher]: Launching browsers Firefox with concurrency unlimited
20 06 2019 19:37:21.067:INFO [launcher]: Starting browser Firefox
20 06 2019 19:37:21.124:ERROR [launcher]: Cannot start Firefox
	
20 06 2019 19:37:21.124:ERROR [launcher]: Firefox stdout: 
20 06 2019 19:37:21.124:ERROR [launcher]: Firefox stderr: 
20 06 2019 19:37:21.127:INFO [launcher]: Trying to start Firefox again (1/2).
20 06 2019 19:37:21.181:ERROR [launcher]: Cannot start Firefox
	
20 06 2019 19:37:21.181:ERROR [launcher]: Firefox stdout: 
20 06 2019 19:37:21.181:ERROR [launcher]: Firefox stderr: 
20 06 2019 19:37:21.183:INFO [launcher]: Trying to start Firefox again (2/2).
20 06 2019 19:37:21.237:ERROR [launcher]: Cannot start Firefox
	
20 06 2019 19:37:21.238:ERROR [launcher]: Firefox stdout: 
20 06 2019 19:37:21.238:ERROR [launcher]: Firefox stderr: 
20 06 2019 19:37:21.239:ERROR [launcher]: Firefox failed 2 times (cannot start). Giving up.
[19:37:21] 'test:browser' errored after 585 ms
[19:37:21] Error: karma: 1
    at ChildProcess.<anonymous> (/repo/js/enum.js/gulpfile.esm.js:93:40)
    at ChildProcess.emit (events.js:200:13)
    at ChildProcess.EventEmitter.emit (domain.js:494:23)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

This error seems to be random: on some projects, the launcher works as expected (for example: https://github.com/cedx/lcov.js). On other projects, the launcher triggers the error (for example: https://github.com/cedx/enum.js).

For the projects that fail, it also seems to be random: the error is triggered on Ubuntu 19.04 and Windows 10, but not on TravisCI with Ubuntu 16.04 (for example: https://travis-ci.com/cedx/enum.js/jobs/209733946).

My Karma config (which is the same on the two mentioned projects):

module.exports = config => config.set({
  basePath: require('path').resolve(__dirname, '..'),
  browsers: ['FirefoxHeadless'],
  files: [
    {pattern: 'lib/**/*.js', type: 'module'},
    {pattern: 'test/**/*.js', type: 'module'}
  ],
  frameworks: ['mocha', 'chai'],
  reporters: ['progress'],
  singleRun: true
});

Using Firefox, instead of FirefoxHeadless, does not change anything (i.e. same error).

[birtles]

This, I suspect, is because Firefox uses a separate launcher process (as of Firefox 64?) which triggers a separate process to avoid DLL injection. The launcher process then exits. Apparently previously -no-remote would prevent this behavior but that changed recently as part of the profile-per-install work (or so I'm told).

As a result, karma launcher will see that the launcher process exited while it is still in its STATE_BEING_CAPTURED state and throw an error. So, I think karma-firefox-launcher needs to override the behavior in the base launcher (i.e. in _execCommand) to not error in this case. However, I'm not sure how it should then find the actual Firefox process to watch so it can clean up the temp directory when it is done.

[birtles]

Actually, it looks like we can fix this just by passing --wait-for-browser. Some details here.

[birtles]

--wait-for-browser works well for starting the browser, but unfortunately killing the launcher process doesn't kill the browser process so the browser remains running after the test has completed.

https://azimi.me/2014/12/31/kill-child_process-node-js.html describes how to kill a process group but it doesn't seem to work for me on Windows, unfortunately (or at least not under WSL).

[birtles]

Looks like for talos we watch to see the first child process generated by the firefox process and then record that as the browser process: https://hg.mozilla.org/mozilla-central/rev/8461e2f532ed

[larsthorup]

With firefox@68 on Windows 10, I see it fairly consistently fail the first time it tries to launch, and then succeed the second time:

First it tries

16 07 2019 11:18:09.605:ERROR [launcher]: Cannot start Firefox

Then it succeeds

16 07 2019 11:18:12.672:INFO [launcher]: Trying to start Firefox again (2/2)

It will also leave an error dialog open:

[birtles]

Right, that's because the browser process will run in the background and on the second time through it will be running when the launcher tries to connect.

Depending on the load on your machine it might connect on the third time or it might fail altogether. With -wait-for-browser it will succeed first time.

I put up PR #95 for this but there seems to be something wrong with the CI for this repository. Looking at other PRs too it looks like no one is maintaining this anymore so I'm not sure if the PR will get merged.

[cedx]

On my side, upgrading Firefox to version 68 fixed the issue on Windows 10 (no more project in error, at least the ones I've tested), but not on Ubuntu 19.10 (exactly the same errors). Using a custom launcher with the -wait-for-browser flag did not change anything.

What I find weird: it's always the same projects that fail, as if it was related to the code rather than Firefox itself. But again: everything is fine if I use Chrome instead of Firefox.

it looks like no one is maintaining this anymore

(off topic) Too bad that this project is a Google one... I also find it really sad that Chrome/Chromium has become the dominant browser (and it will not get better with the new MS Edge).

[ericcarino]

Ubuntu 18.04
Karma 4.2
karma-firefox-launcher 1.1.0
node.js 10.16
Firefox 68.0

We are seeing the same issue when running inside a Docker container. It was working fine until we upgraded to Firefox 68. Now, the issue is reproducible 100% of the time. We downgraded back to Firefox 67 as a workaround in the mean time.

Exact same environment, but not w/ in Docker, we are having no issues. Weird.

[sebavan]

We are having also the issue since Firefox 68 and can not run our tests anymore :-( We can repro on any env using FF 68 with https://github.com/BabylonJS/Babylon.js

[sebavan]

@birtles your fix #95 seems to do it for us \o/ thanks a lot. Do you know is there are any new relrease to npm planned soon ?

[birtles]

@sebavan Thanks for confirming. I was planning on digging into the Ubuntu issues in #93 before releasing (assuming I even have the access required to submit a new release) but haven't had a chance to set up a VM for it yet.

Let me have another look at Ubuntu later this week (I'm currently traveling) and if I can't fix it easily I'll try to release another version.

[birtles]

Let me have another look at Ubuntu later this week (I'm currently traveling) and if I can't fix it easily I'll try to release another version.

Sorry, this is taking longer than expected (having trouble with the VM).

[birtles]

This error seems to be random: on some projects, the launcher works as expected (for example: https://github.com/cedx/lcov.js). On other projects, the launcher triggers the error (for example: https://github.com/cedx/enum.js).

@cedx I tried to repro this on a clean install of Ubuntu 18.04 but it looks like neither of those projects use karma-firefox-launcher any more. If you have repro steps for the Ubuntu issues please let me know.

[birtles]

So I can't repro issues on Ubuntu 18.04 with Firefox 68 and there is at least one report of this fixing things for others so I'm inclined to close this issue. (There is still the issue of the browser process not being killed but that is tracked in #101.)

If anyone else has repro steps for things failing to start in Firefox 68 please let me know.

[ericcarino]

So I can't repro issues on Ubuntu 18.04 with Firefox 68 and there is at least one report of this fixing things for others so I'm inclined to close this issue. (There is still the issue of the browser process not being killed but that is tracked in #101.)

If anyone else has repro steps for things failing to start in Firefox 68 please let me know.

Have you tried running w/ in Docker? Makes absolutely no sense, but what I've seen is everything works fine on a native install of Ubuntu 18.04. But when ran w/ in Docker, problem is reproducible 100% of the time. We downgraded to Firefox 67 to workaround.

[birtles]

Have you tried running w/ in Docker? Makes absolutely no sense, but what I've seen is everything works fine on a native install of Ubuntu 18.04. But when ran w/ in Docker, problem is reproducible 100% of the time. We downgraded to Firefox 67 to workaround.

Ok, I tried to repro this with the following steps:

> docker run -i -t ubuntu:18.04 /bin/bash
docker> apt-get update
docker> apt-get install git nodejs npm -y
docker> npm install -g yarn
docker> git clone https://github.com/karma-runner/karma-firefox-launcher.git
docker> cd karma-firefox-launcher
docker> yarn install
docker> apt-get install firefox -y
docker> apt-get install xvfb
docker> Xvfb :99 -ac -screen 0 1024x768x8 &
docker> export DISPLAY=:99
docker> npm test

but everything seemed to work:

root@8bf4bb97741c:/karma-firefox-launcher# npm test                                                                                                                                                                                           > karma-firefox-launcher@1.1.0 test /karma-firefox-launcher
> karma start --single-run

08 08 2019 01:54:58.447:INFO [karma-server]: Karma v4.2.0 server started at http://0.0.0.0:9876/
08 08 2019 01:54:58.450:INFO [launcher]: Launching browsers Firefox, FirefoxHeadless with concurrency unlimited
08 08 2019 01:54:58.456:INFO [launcher]: Starting browser Firefox
08 08 2019 01:54:58.464:INFO [launcher]: Starting browser FirefoxHeadless
08 08 2019 01:55:01.493:INFO [Firefox 68.0.0 (Ubuntu 0.0.0)]: Connected on socket 2C3JR3U8SY4fP10cAAAA with id 49225738
08 08 2019 01:55:01.518:INFO [Firefox 68.0.0 (Ubuntu 0.0.0)]: Connected on socket xpTBCmBfidnZKo99AAAB with id 14230271
.
Firefox 68.0.0 (Ubuntu 0.0.0): Executed 1 of 1 SUCCESS (0.303 secs / 0 secs)
.
Firefox 68.0.0 (Ubuntu 0.0.0): Executed 1 of 1 SUCCESS (0.163 secs / 0 secs)
TOTAL: 2 SUCCESS

[ericcarino]

@birtles Thanks for trying. I'm currently on a vacation, or else I'd give it a try again. I can when I get back, but at the same time, I'm ok if you'd like to close the issue.

[birtles]

Ok thanks @ericcarino! I'll close this for now. Please file a new issue if you can reproduce it once you get back.

[mgol]

I am hitting the same issue on macOS Sonoma even without using Docker; I submitted #328 with more details.