🏃 Make client.MatchingLabels faster #2529
Conversation
The 99% use-case of this is to set a selector, not to adjust an existing one. This change introduces a fastpath that does that with half the allocations and in a bit less than half the time. The reason slowpath is slow is that for each label a requirement has to be constructed that is then appended to a slice, both of which cause allocations.
k8s-ci-robot
added
the
cncf-cla: yes
k8s-ci-robot
added
approved
|
I also have to say that while yes, this got slower, I find it surprising this is noticeable relative to DeepCopying in the cache or doing an api request. Would you mind describing what exactly you do that this becomes noticeable? |
|
Awesome - I'll add a TODO to test this in our environment tomorrow/Tuesday. At a glance, I think this should help alleviate some of the hot paths we were seeing in our pprof profiles for our controller. I'd love to share them to help paint a better picture, but they're from a closed source product, and I don't want legal knocking down my door 😅 |
|
@timflannagan right I figured that, but could you share some more details, so you seem to be doing a high number of List requests with client.MatchingLabels - What is the aproximate number of labels, do you list against the cache or the api, do you set any other listOptions? |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alvaroaleman, vincepri The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@alvaroaleman Just ran this commit through a couple of our heavier environments and these changes look like it fixes the performance issues we were previously seeing with 0.15.1 tag. Thanks for jumping on this (and the quick turn around). Any chance this could get backported to the release-0.15 release branch? |
|
/cherrypick release-0.16 |
|
@samuelvl: only kubernetes-sigs org members may request cherry picks. You can still do the cherry-pick manually. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/cherry-pick release-0.15 |
|
@sbueringer: new pull request created: #2614 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
The 99% use-case of this is to set a selector, not to adjust an existing one. This change introduces a fastpath that does that with half the allocations and in a bit less than half the time.
The reason slowpath is slow is that for each label a requirement has to be constructed that is then appended to a slice, both of which cause allocations.
Fixes #2522
@timflannagan can you confirm that this fixes your issue? It brings the performance for the case that the labelselector is only set through
MatchingLabelsback to what it originally was.