Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manage concurrency limits inside cozy-connector-libs #230

Open
edas opened this issue May 15, 2018 · 1 comment
Open

Manage concurrency limits inside cozy-connector-libs #230

edas opened this issue May 15, 2018 · 1 comment
Labels

Comments

@edas
Copy link
Contributor

edas commented May 15, 2018

I was recommended not to use Promise.all directly too much because we could hit system's limit on open file descriptors.

The user solution is to use Bluebird's Promise.map with the concurrency parameter.

However, most connectors will only use our functions like requestFactory or saveFiles.
It would probably be a good idea to deal with the concurrency internally at least for the functions we manage. Doing so, most developers won't have to even be aware of our system's limits and of the complexity induced.

For requestFactory we could add { pool: {maxSockets: Infinity} } in the default options. Attention, this requires strictSSL to be at true (it is at falsetoday).

For saveFiles we already use Bluebird and only have to add the concurrency parameter.

@doubleface
Copy link
Collaborator

doubleface commented May 15, 2018

This stack overflow post seems to be a good resource about the maxSockets option which I totally missed. And also the official nodejs documentation of course.

I suppose I had some connection error because of an old version of nodejs at that time.

I suppose we could then use Promise.all in most cases (even in saveFiles) and avoid the use of bluebird.

@doubleface doubleface added the low label Dec 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants