Zarf does not remove the zarf mutatingwebhookconfigurations when destroying

[carroarmato0]

Environment

Device and OS: Dell Laptop - Garuda Linux
App version: 0.33.0
Kubernetes distro being used: OpenShift 4.12
Other:

Steps to reproduce

1. Deploy Zarf using zarf init --confirm on a cluster (this will fail due to 127.0.0.1 being hardcoded to connect to the NodePort)
2. Remove zarf using zarf destroy --confirm --remove-components

Expected result

Zarf is completely removed, and also its modifications undone

Actual Result

The zarf mutatingwebhookconfigurations is not removed, causing certain functions in the cluster to not work properly due to the webhook trying to be used, but the agent has been removed.

$ oc get mutatingwebhookconfigurations
NAME                                WEBHOOKS   AGE
cdi-api-datavolume-mutate           1          139d
kubemacpool-mutator                 2          139d
machine-api                         2          140d
mutate-ns-hco.kubevirt.io-hlqkb     1          139d
network-resources-injector-config   1          139d
nmstate                             5          139d
sriov-operator-webhook-config       1          139d
virt-api-mutator                    4          139d
zarf                                4          43h

$ oc debug node/worker-1
Temporary namespace openshift-debug-8v6q2 is created for debugging node...
Temporary namespace openshift-debug-8v6q2 was removed.
Error from server (InternalError): Internal error occurred: failed calling webhook "agent-pod.zarf.dev": failed to call webhook: Post "https://agent-hook.zarf.svc:443/mutate/pod?timeout=10s": service "agent-hook" not found

{"level":"debug","ts":1714117634.344182,"logger":"controller.clone-controller","msg":"Should not reconcile this PVC","PVC":"ngp/ngp-img-idirect-velocity-fatgcp","checkPVC(AnnCloneRequest)":false,"NOT has annotation(AnnCloneOf)":true,"isBound":true,"has finalizer?":false}
{"level":"error","ts":1714117634.3442128,"logger":"controller.import-controller","msg":"Reconciler error","name":"ngp-img-idirect-velocity-fatgcp","namespace":"ngp","error":"Internal error occurred: failed calling webhook \"agent-pod.zarf.dev\": failed to call webhook: Post \"https://agent-hook.zarf.svc:443/mutate/pod?timeout=10s\": service \"agent-hook\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}

$ oc get ns | grep zarf

$ oc get mutatingwebhookconfigurations
NAME                                WEBHOOKS   AGE
cdi-api-datavolume-mutate           1          139d
kubemacpool-mutator                 2          139d
machine-api                         2          140d
mutate-ns-hco.kubevirt.io-hlqkb     1          139d
network-resources-injector-config   1          139d
nmstate                             5          139d
sriov-operator-webhook-config       1          139d
virt-api-mutator                    4          139d
zarf                                4          43h

Deleting the mutatingwebhookconfigurations zarf manually resolves the issue and the cluster works as expected.

[carroarmato0]

After retrying I could not reproduce this.

I think this might have been a left-over from previous attempts where I had mixed success and eventually for the agent starting.

Perhaps it might be useful though to ensure that Zarf also deletes the mutatingwebhookconfigurations when it's being removed?