🐛 crs: use separate cache for partial metadata watches on secrets to include all secrets #10633

chrischdi · 2024-05-16T09:35:48Z

What this PR does / why we need it:

This PR introduces a separate cache which is used in the clusterresourceset_controller for watching secrets.

Previously the WatchesMetadata for secrets in clusterresourcesset_controller did inherit the LabelSelector configured in main.go:

https://github.com/kubernetes-sigs/cluster-api/blob/main/main.go#L322-L329

This label selector gets passed through in controller-runtime for the informer which gets created for the watch.

Secrets for clusterresourcesets may apply for multiple clusters, so the label selector may not even exist at the secrets referred by clusterresourcesets.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #10557

/area clusterresourceset

k8s-ci-robot · 2024-05-16T09:36:20Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from chrischdi. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

exp/addons/internal/controllers/suite_test.go

exp/addons/internal/controllers/clusterresourceset_controller.go

exp/addons/internal/controllers/predicates/resource_predicates.go

sbueringer · 2024-05-17T14:59:42Z

Very nice!

exp/runtime/internal/controllers/extensionconfig_controller.go

main.go

sbueringer · 2024-05-22T15:15:15Z

@chrischdi can you please check the unit tests?

sbueringer · 2024-05-22T15:52:05Z

/test pull-cluster-api-e2e-main

sbueringer

Just a few nits. Sorry for the nitpicking, just playing around a bit with generics and trying to find the simplest implementation

Otherwise all good, also tested it and it works perfectly (inspected the caches at runtime)

exp/addons/internal/controllers/suite_test.go

exp/runtime/internal/controllers/extensionconfig_controller.go

chrischdi · 2024-05-27T07:22:45Z

/test pull-cluster-api-e2e-main

sbueringer · 2024-05-27T11:24:45Z

exp/runtime/internal/controllers/extensionconfig_controller.go

+// secretToExtensionConfigFunc returns a func which maps a secret to ExtensionConfigs with the corresponding
+// InjectCAFromSecretAnnotation to reconcile them on updates of the secrets.
+func (r *Reconciler) secretToExtensionConfigFunc(ctx context.Context, o *metav1.PartialObjectMetadata) []reconcile.Request {


Can we revert this (func name + godoc) entirely to what is on main? I think the godoc is not correct anymore (+ the func name is a bit inconsistent now with how we usually call these funcs)

sbueringer · 2024-05-27T11:25:29Z

Last nit from my side

/assign @fabriziopandini

chrischdi · 2024-05-28T05:48:52Z

/test pull-cluster-api-e2e-main

sbueringer · 2024-05-28T11:04:55Z

Thank you very much!
/lgtm

Let's get some additional reviews if possible, just in case I'm missing something
/assign @fabriziopandini @vincepri

k8s-ci-robot · 2024-05-28T11:05:01Z

LGTM label has been added.

Git tree hash: fd0b933f00763538f0332835600823f4a8a7933d

fabriziopandini

Nice change!

fabriziopandini · 2024-05-30T14:11:07Z

main.go

+	// Setup a separate cache for the metadata watches to secrets.
+	// This way the watch does not use the LabelSelector defined at the cache which
+	// would filter to secrets having the cluster label, because secrets referred
+	// by ClusterResourceSet or ExtensionConfig are not specific to a single cluster.


Suggested change

// Setup a separate cache for the metadata watches to secrets.

// This way the watch does not use the LabelSelector defined at the cache which

// would filter to secrets having the cluster label, because secrets referred

// by ClusterResourceSet or ExtensionConfig are not specific to a single cluster.

// Setup a separate cache without label selector for secrets, to be used

// when we need to watch for secrets that are not specific to a single cluster (e.g. ClusterResourceSet or ExtensionConfig controllers).

fabriziopandini · 2024-05-30T14:14:31Z

main.go

+	// This way the watch does not use the LabelSelector defined at the cache which
+	// would filter to secrets having the cluster label, because secrets referred
+	// by ClusterResourceSet or ExtensionConfig are not specific to a single cluster.
+	partialSecretCache, err := cache.New(mgr.GetConfig(), cache.Options{


q: should this be allSecretCache instead of partialSecretCache (nothing in the definition points to partial)
q: is there a way to make sure this cache is used only for Secrets (I think not, but might be we can enforce this with a DefaultTransformerFunc that always returns error)
q: should we use TransformStripManagedFields for secrets? (not necessary, but it doesn't hurt)

cc @sbueringer

the intention why I named it partialSecretCache is that we tend to only use it for PartialObjectMetadata watches/objects. Maybe I should add that information to the comment?

I like the idea of adding a DefaultTransformerFunc and implemented it.

This way we can make sure to not mis-use the cache 👍

k8s-ci-robot · 2024-05-31T09:02:37Z

New changes are detected. LGTM label has been removed.

…clude all secrets

k8s-ci-robot · 2024-05-31T09:08:54Z

@chrischdi: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-apidiff-main	`1dd1d9e`	link	false	`/test pull-cluster-api-apidiff-main`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

chrischdi · 2024-05-31T09:13:44Z

/test pull-cluster-api-e2e-main

chrischdi · 2024-05-31T15:37:42Z

Cosmetics:

/override pull-cluster-api-apidiff-main

k8s-ci-robot · 2024-05-31T15:37:44Z

@chrischdi: chrischdi unauthorized: /override is restricted to Repo administrators.

In response to this:

Cosmetics:

/override pull-cluster-api-apidiff-main

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added area/clusterresourceset Issues or PRs related to clusterresourcesets cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 16, 2024

chrischdi changed the title ~~🐛 crs: use separate cache for partial metadata watches on secrets to in…~~ 🐛 crs: use separate cache for partial metadata watches on secrets to include all secrets May 16, 2024

k8s-ci-robot requested review from JoelSpeed and richardcase May 16, 2024 09:36

chrischdi mentioned this pull request May 16, 2024

CRS associated secret updates will not trigger CRS Reconcile #10557

Open

sbueringer reviewed May 17, 2024

View reviewed changes

chrischdi force-pushed the pr-crs-watch-partial-all-secrets branch from 9fde0e2 to 4f90185 Compare May 21, 2024 08:48

chrischdi added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label May 21, 2024

sbueringer reviewed May 21, 2024

View reviewed changes

sbueringer reviewed May 23, 2024

View reviewed changes

exp/addons/internal/controllers/suite_test.go Show resolved Hide resolved

exp/runtime/internal/controllers/extensionconfig_controller.go Outdated Show resolved Hide resolved

sbueringer reviewed May 27, 2024

View reviewed changes

k8s-ci-robot assigned fabriziopandini May 27, 2024

k8s-ci-robot assigned vincepri and sbueringer May 28, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 28, 2024

fabriziopandini mentioned this pull request May 29, 2024

🌱 Test BYO certificates #10681

Open

fabriziopandini reviewed May 30, 2024

View reviewed changes

k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels May 30, 2024

k8s-ci-robot requested review from fabriziopandini, sbueringer and vincepri May 31, 2024 09:02

chrischdi added 7 commits May 31, 2024 11:03

crs: use separate cache for partial metadata watches on secrets to in…

af18830

…clude all secrets

review fixes

a8d8968

review fixes

aba61a2

exp/test: start cache

fe1ad97

review fixes

8329012

review fixes

c9f427d

review fixes

1dd1d9e

chrischdi force-pushed the pr-crs-watch-partial-all-secrets branch from 53006e4 to 1dd1d9e Compare May 31, 2024 09:05

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 crs: use separate cache for partial metadata watches on secrets to include all secrets #10633

🐛 crs: use separate cache for partial metadata watches on secrets to include all secrets #10633

chrischdi commented May 16, 2024 •

edited

k8s-ci-robot commented May 16, 2024

sbueringer commented May 17, 2024

sbueringer commented May 22, 2024

sbueringer commented May 22, 2024

sbueringer left a comment •

edited

chrischdi commented May 27, 2024

sbueringer May 27, 2024 •

edited

sbueringer commented May 27, 2024

chrischdi commented May 28, 2024

sbueringer commented May 28, 2024

k8s-ci-robot commented May 28, 2024

fabriziopandini left a comment

fabriziopandini May 30, 2024

fabriziopandini May 30, 2024

chrischdi May 31, 2024

chrischdi May 31, 2024

k8s-ci-robot commented May 31, 2024

k8s-ci-robot commented May 31, 2024 •

edited

chrischdi commented May 31, 2024

chrischdi commented May 31, 2024

k8s-ci-robot commented May 31, 2024

🐛 crs: use separate cache for partial metadata watches on secrets to include all secrets #10633

Are you sure you want to change the base?

🐛 crs: use separate cache for partial metadata watches on secrets to include all secrets #10633

Conversation

chrischdi commented May 16, 2024 • edited

k8s-ci-robot commented May 16, 2024

sbueringer commented May 17, 2024

sbueringer commented May 22, 2024

sbueringer commented May 22, 2024

sbueringer left a comment • edited

Choose a reason for hiding this comment

chrischdi commented May 27, 2024

sbueringer May 27, 2024 • edited

Choose a reason for hiding this comment

sbueringer commented May 27, 2024

chrischdi commented May 28, 2024

sbueringer commented May 28, 2024

k8s-ci-robot commented May 28, 2024

fabriziopandini left a comment

Choose a reason for hiding this comment

fabriziopandini May 30, 2024

Choose a reason for hiding this comment

fabriziopandini May 30, 2024

Choose a reason for hiding this comment

chrischdi May 31, 2024

Choose a reason for hiding this comment

chrischdi May 31, 2024

Choose a reason for hiding this comment

k8s-ci-robot commented May 31, 2024

k8s-ci-robot commented May 31, 2024 • edited

chrischdi commented May 31, 2024

chrischdi commented May 31, 2024

k8s-ci-robot commented May 31, 2024

chrischdi commented May 16, 2024 •

edited

sbueringer left a comment •

edited

sbueringer May 27, 2024 •

edited

k8s-ci-robot commented May 31, 2024 •

edited