-
Notifications
You must be signed in to change notification settings - Fork 884
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Removed AddressPool CRD is still referenced in code #2270
Comments
after 2 prs that we tried to remove it it's still up 😂 |
Thanks for fixing this @oribon! 💯 |
Hi team , We are still encountering errors in the logs after installing MetalLB version 0.14.3 on a Kubernetes cluster running version 1.28.6. Could you please confirm if this will have any impact on our setup?
Kr, |
Hi, It still seems to be referenced in some places.
> kl metallb-controller-648b76f565-4c76v -n metallb | tail -n 2
{"level":"error","ts":"2024-03-19T04:56:28Z","logger":"cert-rotation","msg":"Webhook not found. Unable to update certificate.","name":"addresspools.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","error":"CustomResourceDefinition.apiextensions.k8s.io \"addresspools.metallb.io\" not found","stacktrace":"github.com/open-policy-agent/cert-controller/pkg/rotator.(*ReconcileWH).ensureCerts\n\t/go/pkg/mod/github.com/open-policy-agent/cert-controller@v0.10.1/pkg/rotator/rotator.go:816\ngithub.com/open-policy-agent/cert-controller/pkg/rotator.(*ReconcileWH).Reconcile\n\t/go/pkg/mod/github.com/open-policy-agent/cert-controller@v0.10.1/pkg/rotator/rotator.go:785\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":"2024-03-19T04:56:28Z","logger":"cert-rotation","msg":"Ensuring CA cert","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition","name":"bgppeers.metallb.io","gvk":"apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"} Any help would be appreciated, thanks. |
0.14.4 includes these changes. Apparently there's a bug where removing a CRD doesn't terminate existing informers. I had to restart my k8s API server after the upgrade. |
I think it's kubernetes/kubernetes#79610. |
@nick-oconnor not sure I understood what you mean. Together with the crds not be used anymore, the corresponding controllers should not instantiate the informers as we removed them. The behaviour you are describing should happen only if you remove the crds but keep the old version of the controllers (on metallb side)? |
Ah yep, my comment wasn't clear. The errors were logged by the k8s controller manager. I think the issue was due to the 0.14.2 webhook config which instructed the k8s controller manager to watch the
I tried restarting the k8s controller manager which didn't help. I then restarted the API server which fixed it. I think I'm going to submit a k8s PR. That log line should include which object the request was attempting to watch. |
MetalLB Version
0.14.3
Deployment method
Charts
Main CNI
calico
Kubernetes Version
1.28.6
Cluster Distribution
kubeadm/baremetal
Describe the bug
Code still references the deprecated
addresspools.metallb.io
CRD which was removed from the helm chart in v0.14.3. Both metallb and the k8s controller manager are logging errors:I suspect this was tested on an upgraded cluster (vs new install) which still had the
addresspools.metallb.io
CRD.To Reproduce
Deploy version 0.14.3 (clean slate or remove the
addresspools.metallb.io
CRD) with cert rotation enabled (default).Expected Behavior
Cert rotation works and no errors are logged.
Additional Context
Nope!
I've read and agree with the following
I've read and agree with the following
The text was updated successfully, but these errors were encountered: