Support setting ipMode: Proxy for LB service for VPC mode

[timdengyun]

Refer to:
KEP-1860 Make Kubernetes aware of the LoadBalancer behaviour
kubernetes/kubernetes#119937: Make Kubernetes aware of the LoadBalancer behaviour
Kubernetes 1.29: New (alpha) Feature, Load Balancer IP Mode for Services

Starting from K8s 1.29.0, we can specify iPMode of load balancer status with VIP or Proxy.

This patch is to set Loadbalancer service status ipMode when the K8s version is >= 1.29.0
and VPC mode enabled:
The setting conditions:

1. service tanzu.vmware.com/ingress-ip-mode label
2. Default ipMode is Proxy if the label is not set or invalid values are set.

Testing Done:
in K8s 1.29.1 env and enable feature gate in kube-proxy, kube-apiserver,.

1. Set label: tanzu.vmware.com/ingress-ip-mode: vip for one lb service, and create the lb service, check lb status ipMode is VIP.
   status:
   loadBalancer:
   ingress:
   • ip: 192.168.64.1
     ipMode: VIP

kubectl get svc -A -o wide
NAMESPACE     NAME          TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                  AGE     SELECTOR
ns-2          tcp-svc-lb2   LoadBalancer   10.97.203.125   192.168.64.1   80:31954/TCP             80m     app=tcp

Check IPTable rules:  iptables -L -t nat
Chain KUBE-SERVICES (2 references)
target     prot opt source               destination
KUBE-SVC-63L2GNUUG6ILWM3U  tcp  --  anywhere             10.97.203.125        /* ns-2/tcp-svc-lb2:tcp cluster IP */ tcp dpt:http
KUBE-EXT-63L2GNUUG6ILWM3U  tcp  --  anywhere             192.168.64.1         /* ns-2/tcp-svc-lb2:tcp loadbalancer IP */ tcp dpt:http
There is DNAT rule KUBE-EXT-63L2GNUUG6ILWM3U generated for loadbalancer IP 192.168.64.1
and also there is cluster IP rule.

2. Set label: tanzu.vmware.com/ingress-ip-mode: proxy for one lb service, and create the lb service, check lb status ipMode is Proxy.
   status:
   loadBalancer:
   ingress:
   • ip: 192.168.244.1
     ipMode: Proxy

Check IPTable rules:  iptables -L -t nat
Chain KUBE-SERVICES (2 references)
target     prot opt source               destination
KUBE-SVC-63L2GNUUG6ILWM3U  tcp  --  anywhere             10.97.203.125        /* ns-2/tcp-svc-lb2:tcp cluster IP */ tcp dpt:http

There is no DNAT rule generated for loadbalancer IP 192.168.64.1 only cluster IP rule. 

3. Unset label tanzu.vmware.com/ingress-ip-mode or set invalid values other than vip or proxy, check lb status is ipMode: Proxy.
   There is no DNAT rule generated.

in K8s < 1.29.0 env, lb service controller is not started.

[TaoZou1]

2. VIP.

-> proxy?

per https://kubernetes.io/blog/2023/12/18/kubernetes-1-29-feature-loadbalancer-ip-mode-alpha/
This field specifies how the load balancer IP behaves and can be specified only when the .status.loadBalancer.ingress.ip field is also specified

Do we need to check that field before update IPmode?

[timdengyun]

2. VIP.

-> proxy?

per https://kubernetes.io/blog/2023/12/18/kubernetes-1-29-feature-loadbalancer-ip-mode-alpha/ This field specifies how the load balancer IP behaves and can be specified only when the .status.loadBalancer.ingress.ip field is also specified

Do we need to check that field before update IPmode?

Agreed, we can check status.loadBalancer.ingress.ip field before setting IPmode.