Node Js cluster: process fork is not working

[Zain-ul-din]

Describe the bug

I'm trying to use the cluster module of node js. I tried running my code using node it's working perfectly but when I'm trying to run the same code using npx vite-node ./file-path it works for primary cluster but when it comes to worker threads it says No files specified.

import cluster from 'cluster';
import os from 'os';

if(cluster.isPrimary) {
  console.log('invoked primary thread')
  
  for(let i =0; i< os.cpus().length; i+= 1) {
    const worker = cluster.fork();
    worker.send(i);
  }
  
  
} else if(cluster.isWorker) {
  process.on('message', (msg)=> {
    console.log('On Worker - ', msg);
  })
}

Output:

invoked primary thread
- No files specified.
vite-node/1.3.1

Usage:
  $ vite-node [...files]

Commands:
  [...files]  

For more info, run any command with the `--help` flag:
  $ vite-node --help

Options:
  -r, --root <path>    Use specified root directory 
  -c, --config <path>  Use specified config file 
  -m, --mode <mode>    Set env mode 
  -w, --watch          Restart on file changes, similar to "nodemon" 
  --script             Use vite-node as a script runner 
  --options <options>  Use specified Vite server options 
  -v, --version        Output the version number 
  -h, --help           Display help for command 
  -v, --version        Display version number 
  ... so on

Expected Output:

invoked primary thread
On Worker -  0
On Worker -  1
On Worker -  2
On Worker -  3
On Worker -  4
On Worker -  5
On Worker -  6
On Worker -  7
On Worker -  8
On Worker -  9
On Worker -  11
On Worker -  10
On Worker -  12
On Worker -  13
On Worker -  14
On Worker -  15

Reproduction

To reproduce bug,

• run yarn init -y
• create index.ts
• paste the following code in index.ts

import cluster from 'cluster';
import os from 'os';

if(cluster.isPrimary) {
  console.log('invoked primary thread')
  
  for(let i =0; i< os.cpus().length; i+= 1) {
    const worker = cluster.fork();
    worker.send(i);
  }
  
  
} else if(cluster.isWorker) {
  process.on('message', (msg)=> {
    console.log('On Worker - ', msg);
  })
}

• run npx vite-node ./index.ts

System Info

Host Name:                 DESKTOP-NOCCTH7
OS Name:                   Microsoft Windows 10 Pro
OS Version:                10.0.19045 N/A Build 19045
Node Js Version:           v20.10.0
yarn version:              1.22.19
npm version:               9.8.1

### Used Package Manager

yarn

### Validations

- [X] Follow our [Code of Conduct](https://github.com/vitest-dev/vitest/blob/main/CODE_OF_CONDUCT.md)
- [X] Read the [Contributing Guidelines](https://github.com/vitest-dev/vitest/blob/main/CONTRIBUTING.md).
- [X] Read the [docs](https://vitest.dev/guide/).
- [X] Check that there isn't [already an issue](https://github.com/vitest-dev/vitest/issues) that reports the same bug to avoid creating a duplicate.
- [X] Check that this is a concrete bug. For Q&A open a [GitHub Discussion](https://github.com/vitest-dev/vitest/discussions) or join our [Discord Chat Server](https://chat.vitest.dev).
- [X] The provided reproduction is a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) of the bug.

[hi-ogawa]

Interesting. I'm not sure what's happening yet, but at least ts-node supports this somehow, so maybe it's possible for vite-node too.

I made a reproduction on stackblitz:
https://stackblitz.com/github/hi-ogawa/reproductions/tree/main/vitest-5323-vite-node-cluster?file=index.ts

$ npx ts-node index.ts
[isPrimary] {
  args: [],
  exec: '/home/projects/rlmixvjzk.github/index.ts',
  execArgv: [
    '/home/projects/rlmixvjzk.github/node_modules/ts-node/dist/bin.js'
  ],
  silent: false
}
[isWorker]
[isWorker.msg] 0

$ npx vite-node index.ts
[isPrimary] {
  args: [],
  exec: '/home/projects/rlmixvjzk.github/node_modules/.bin/vite-node',
  execArgv: [],
  silent: false
}
No files specified.
vite-node/1.3.1

Usage:
  $ vite-node [...files]
...

[hi-ogawa]

It looks like ts-node mutates process.execArgv https://github.com/TypeStrong/ts-node/blob/ddb05ef23be92a90c3ecac5a0220435c65ebbd2a/src/bin.ts#L586-L587

Currently vite-node mutates process.argv, which is already somewhat complicated due to #3574 and #2793), so probably it requires more thinking if we want to get all scenarios right.

For the time being, I think you can use cluster.setupPrimary to manually fix up fork arguments. For example, this seems to work for your example:

https://stackblitz.com/edit/github-kfifdx?file=index-fix.ts

cluster.setupPrimary({
  args: [import.meta.filename],
})

[Zain-ul-din]

it started working but I'm not able to send messages from parent to child

import cluster from 'cluster';

if (cluster.isPrimary) {
  cluster.setupPrimary({
    args: [import.meta.filename],
  });
  console.log('[isPrimary] settings', cluster.settings);
  const worker = cluster.fork();
  worker.send('hello from primary')
  worker.on('message', (message) => {
    console.log('[isPrimary] received :', { message });
  });
} else if (cluster.isWorker) {
  const message = 'hello from worker!';
  console.log('[isWorker] sending :', { message });
  process.on('message', (msg)=> {
    console.log('child ', msg)
  })
  process.send(message);
}

[hi-ogawa]

Yeah, I noticed that in your example, but I assumed that's simply because of a race condition and you probably need to wait until worker side process.on is set up. For example, you could wait worker.send in this way:

https://stackblitz.com/edit/github-kfifdx-3xmjq2?file=index-fix.ts

[Zain-ul-din]

@hi-ogawa Thanks! I tested the same code on ts-node it's working there. maybe you need to fix this later cuz in this case, we're adding lots of code to make this happen but in ts-node we don't have to be.

[hi-ogawa]

I tested the same code on ts-node it's working there

Yeah, I understand and also plain Node works with this. But, the reason why they work might be just a lucky since there's no guarantee how long it takes from parent's worker.fork to child's process.on.
For example, if your PC and CPU is loaded with other heavy tasks, then it's possible that this race condition would manifest even for ts-node and plain Node.

I think usual practice is to put some retry mechanism to ensure initial hand-shake to counter against such race condition. Or actually waiting for a message from a child to a parent seems better since this direction is probably guaranteed to be race condition free (cf. https://stackoverflow.com/a/66190604)

Btw, keeping this issue open is fine since the workaround cluster.setupPrimary({ args: [import.meta.filename] }) looks shaky and maybe it's still better if vite-node would support this pattern out-of-the-box.