🌱 Add compare util using go-cmp, modify webhooks & KCP controller

[sbueringer]

What this PR does / why we need it:
Needs a bit more work and some manual tests (want to see how the diffs are looking in logs)

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #8186

Example logs for KCP controller

I0516 16:01:22.008912      58 controller.go:412] "Rolling out Control Plane machines: Machine capi-quickstart-control-plane-xvq7h needs rollout: Machine KubeadmConfig ClusterConfiguration is outdated: diff: &v1beta1.ClusterConfiguration{\n    ... // 3 identical fields\n    KubernetesVersion:    \"\",\n    ControlPlaneEndpoint: \"\",\n    APIServer: v1beta1.APIServer{\n      ControlPlaneComponent: {},\n      CertSANs: []string{\n        \"localhost\",\n-       \"127.0.0.1\",\n        \"0.0.0.0\",\n        \"host.docker.internal\",\n      },\n      TimeoutForControlPlane: nil,\n    },\n    ControllerManager: {ExtraArgs: {\"enable-hostpath-provisioner\": \"true\"}},\n    Scheduler:         {},\n    ... // 5 identical fields\n  }" controller="kubeadmcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="KubeadmControlPlane" KubeadmControlPlane="default/capi-quickstart-control-plane" namespace="default" name="capi-quickstart-control-plane" reconcileID="93e4bce7-00da-44c7-9421-a714ecf996ca" Cluster="default/capi-quickstart" machinesNeedingRollout=["capi-quickstart-control-plane-xvq7h"]

Message after removing escaping

Rolling out Control Plane machines: Machine capi-quickstart-control-plane-xvq7h needs rollout: Machine KubeadmConfig ClusterConfiguration is outdated: diff: &v1beta1.ClusterConfiguration{
    ... // 3 identical fields
    KubernetesVersion:    "",
    ControlPlaneEndpoint: "",
    APIServer: v1beta1.APIServer{
      ControlPlaneComponent: {},
      CertSANs: []string{
        "localhost",
-       "127.0.0.1",
        "0.0.0.0",
        "host.docker.internal",
      },
      TimeoutForControlPlane: nil,
    },
    ControllerManager: {ExtraArgs: {"enable-hostpath-provisioner": "true"}},
    Scheduler:         {},
    ... // 5 identical fields
  }

Example for webhook

k apply -f /var/folders/v0/mny6w2dd33gfzwmwxb3wyyx40000gq/T/kubectl-edit-3932073547.yaml
[kubeadmcontrolplanetemplate.controlplane.cluster.x-k8s.io/quick-start-control-plane-2](http://kubeadmcontrolplanetemplate.controlplane.cluster.x-k8s.io/quick-start-control-plane-2) configured
[kubeadmcontrolplanetemplate.controlplane.cluster.x-k8s.io/quick-start-control-plane-3](http://kubeadmcontrolplanetemplate.controlplane.cluster.x-k8s.io/quick-start-control-plane-3) configured
The KubeadmControlPlaneTemplate "quick-start-control-plane" is invalid: spec.template.spec: Invalid value: v1beta1.KubeadmControlPlaneTemplate{TypeMeta:v1.TypeMeta{Kind:"KubeadmControlPlaneTemplate", APIVersion:"[controlplane.cluster.x-k8s.io/v1beta1](http://controlplane.cluster.x-k8s.io/v1beta1)"}, ObjectMeta:v1.ObjectMeta{Name:"quick-start-control-plane", GenerateName:"", Namespace:"default", SelfLink:"", UID:"288374d2-7a3b-4a26-9606-0627bd06db1d", ResourceVersion:"2293", Generation:2, CreationTimestamp:time.Date(2024, time.May, 16, 16, 6, 12, 0, time.Local), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"[kubectl.kubernetes.io/last-applied-configuration](http://kubectl.kubernetes.io/last-applied-configuration)":"{\"apiVersion\":\"[controlplane.cluster.x-k8s.io/v1beta1](http://controlplane.cluster.x-k8s.io/v1beta1)\",\"kind\":\"KubeadmControlPlaneTemplate\",\"metadata\":{\"annotations\":{},\"creationTimestamp\":\"2024-05-16T16:06:12Z\",\"generation\":1,\"name\":\"quick-start-control-plane\",\"namespace\":\"default\",\"resourceVersion\":\"2293\",\"uid\":\"288374d2-7a3b-4a26-9606-0627bd06db1d\"},\"spec\":{\"template\":{\"metadata\":{},\"spec\":{\"kubeadmConfigSpec\":{\"clusterConfiguration\":{\"apiServer\":{\"certSANs\":[\"localhost\",\"0.0.0.0\",\"host.docker.internal\"]},\"controllerManager\":{\"extraArgs\":{\"enable-hostpath-provisioner\":\"true\"}},\"dns\":{},\"etcd\":{},\"networking\":{},\"scheduler\":{}},\"files\":[{\"content\":\"SOMETESTCONTENT\\n\",\"path\":\"abc\"}],\"format\":\"cloud-config\",\"initConfiguration\":{\"localAPIEndpoint\":{},\"nodeRegistration\":{\"criSocket\":\"/var/run/containerd/containerd.sock\",\"imagePullPolicy\":\"IfNotPresent\",\"kubeletExtraArgs\":{\"eviction-hard\":\"nodefs.available\\u003c0%,nodefs.inodesFree\\u003c0%,imagefs.available\\u003c0%\"}}},\"joinConfiguration\":{\"discovery\":{},\"nodeRegistration\":{\"criSocket\":\"/var/run/containerd/containerd.sock\",\"imagePullPolicy\":\"IfNotPresent\",\"kubeletExtraArgs\":{\"eviction-hard\":\"nodefs.available\\u003c0%,nodefs.inodesFree\\u003c0%,imagefs.available\\u003c0%\"}}}},\"rolloutStrategy\":{\"rollingUpdate\":{\"maxSurge\":1},\"type\":\"RollingUpdate\"}}}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"[controlplane.cluster.x-k8s.io/v1beta1](http://controlplane.cluster.x-k8s.io/v1beta1)", Time:time.Date(2024, time.May, 16, 16, 7, 25, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4000a540a8), Subresource:""}}}, Spec:v1beta1.KubeadmControlPlaneTemplateSpec{Template:v1beta1.KubeadmControlPlaneTemplateResource{ObjectMeta:v1beta1.ObjectMeta{Labels:map[string]string(nil), Annotations:map[string]string(nil)}, Spec:v1beta1.KubeadmControlPlaneTemplateResourceSpec{MachineTemplate:(*v1beta1.KubeadmControlPlaneTemplateMachineTemplate)(nil), KubeadmConfigSpec:v1beta1.KubeadmConfigSpec{ClusterConfiguration:(*v1beta1.ClusterConfiguration)(0x4000d862c0), InitConfiguration:(*v1beta1.InitConfiguration)(0x40000ea620), JoinConfiguration:(*v1beta1.JoinConfiguration)(0x4001172000), Files:[]v1beta1.File{v1beta1.File{Path:"abc", Owner:"", Permissions:"", Encoding:"", Append:false, Content:"SOMETESTCONTENT\n", ContentFrom:(*v1beta1.FileSource)(nil)}}, DiskSetup:(*v1beta1.DiskSetup)(nil), Mounts:[]v1beta1.MountPoints(nil), PreKubeadmCommands:[]string(nil), PostKubeadmCommands:[]string(nil), Users:[]v1beta1.User(nil), NTP:(*v1beta1.NTP)(nil), Format:"cloud-config", Verbosity:(*int32)(nil), UseExperimentalRetryJoin:false, Ignition:(*v1beta1.IgnitionSpec)(nil)}, RolloutBefore:(*v1beta1.RolloutBefore)(nil), RolloutAfter:<nil>, RolloutStrategy:(*v1beta1.RolloutStrategy)(0x4000a54150), RemediationStrategy:(*v1beta1.RemediationStrategy)(nil)}}}}: KubeadmControlPlaneTemplate spec.template.spec field is immutable. Please create new resource instead. Diff: v1beta1.KubeadmControlPlaneTemplateResourceSpec{
    MachineTemplate: nil,
    KubeadmConfigSpec: v1beta1.KubeadmConfigSpec{
      ClusterConfiguration: &v1beta1.ClusterConfiguration{
        ... // 3 identical fields
        KubernetesVersion:    "",
        ControlPlaneEndpoint: "",
        APIServer: v1beta1.APIServer{
          ControlPlaneComponent: {},
          CertSANs: []string{
            "localhost",
-           "127.0.0.1",
            "0.0.0.0",
            "host.docker.internal",
          },
          TimeoutForControlPlane: nil,
        },
        ControllerManager: {ExtraArgs: {"enable-hostpath-provisioner": "true"}},
        Scheduler:         {},
        ... // 5 identical fields
      },
      InitConfiguration: &{NodeRegistration: {CRISocket: "/var/run/containerd/containerd.sock", KubeletExtraArgs: {"eviction-hard": "nodefs.available<0%,nodefs.inodesFree<0%,imagefs.available<0%"}, ImagePullPolicy: "IfNotPresent"}},
      JoinConfiguration: &{NodeRegistration: {CRISocket: "/var/run/containerd/containerd.sock", KubeletExtraArgs: {"eviction-hard": "nodefs.available<0%,nodefs.inodesFree<0%,imagefs.available<0%"}, ImagePullPolicy: "IfNotPresent"}},
      ... // 11 identical fields
    },
    RolloutBefore: nil,
    RolloutAfter:  nil,
    ... // 2 identical fields
  }

[sbueringer]

Not sure what the right areas are (apart from KCP)

[sbueringer]

/test pull-cluster-api-e2e-main

[sbueringer]

/test pull-cluster-api-e2e-main

[sbueringer]

/test pull-cluster-api-e2e-main

[sbueringer]

/assign @chrischdi @fabriziopandini

[chrischdi]

small nit

[sbueringer]

/test pull-cluster-api-e2e-main

[chrischdi]

/lgtm

(failing test though)

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 1cbb61b35cb32a436cb37ed94cbe31e05798a4c7

[sbueringer]

Ups, thx, fixed

[sbueringer]

/test pull-cluster-api-e2e-main

[chrischdi]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: c6142679f413cf7869f87d9fd3f3cf2730468d37

[sbueringer]

Had to rebase because of the defaulting webhook fix

/assign @fabriziopandini @chrischdi

[chrischdi]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 3dd034826d867dd90f918d23537ea8663305d1ec

[sbueringer]

/assign @fabriziopandini

[fabriziopandini]

Nice
This will greatly improve debugging rollouts in the logs
/lgtm
/approved

[fabriziopandini]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [fabriziopandini]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment