Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[24.0 backport] libcontainerd: work around exec start bug in c8d #45625

Merged

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented May 25, 2023


It turns out that the unnecessary serialization removed in b752462 happened to work around a bug in containerd. When many exec processes are started concurrently in the same containerd task, it takes seconds to minutes for them all to start. Add the workaround back in, only deliberately this time.

(cherry picked from commit fb7ec15)

- What I did

- How I did it
✨ mutex ✨

- How to verify it

# docker run --rm -d --name pause1 alpine sleep infinity
# time sh -c 'myjobs=""; for i in $(seq 1 50); do docker exec pause1 true & myjobs="$myjobs $!"; done; for pid in $myjobs; do wait $pid; done'

Takes ~3 seconds on my machine, compared to around a minute without the workaround.

- Description for the changelog

  • Work around an issue in containerd which causes execs to take an excessive amount of time to start

- A picture of a cute animal (not mandatory but encouraged)

It turns out that the unnecessary serialization removed in
b752462 happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit fb7ec15)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah merged commit 659604f into moby:24.0 May 25, 2023
101 checks passed
@thaJeztah thaJeztah deleted the 24.0_backport_serialize_exec_starts_workaround branch May 25, 2023 21:22
@ningmingxiao
Copy link
Contributor

ningmingxiao commented May 30, 2023

this pr may fix this issue
containerd/containerd#8593

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants