✨ setup-envtest: download binaries from controller-tools releases

[sbueringer]

setup-envtest today downloads archives with envtest binaries from the kubebuilder GCS bucket. Given kubernetes/k8s.io#2647 we have to ASAP move away from this.

@vincepri and I tried and discussed various options (GitHub packages, k8s staging GCS bucket & registry, k8s prod registry) and we eventually ended up with publishing envtest binaries to controller-tools releases. The reason was that this is the only supported place to host artifacts in the k8s ecosystem for us:

• GitHub packages: not allowed (see Slack thread / Usage of Github Packages / Container repository kubernetes/org#4514 (comment))
• k8s staging GCS bucket: retention of 60 days
• k8s staging image registry: it's just staging, seems wrong to host "prod artifacts" there, not possible to easily consume envtest binary archives with e.g. curl
• k8s prod image registry: requires additional effort for image promotion, not possible to easily consume envtest binary archives with e.g. curl

So our new envtest binary archive "repository" looks like this:

• Index: https://raw.githubusercontent.com/kubernetes-sigs/controller-tools/HEAD/envtest-releases.yaml
• Envtest archives for e.g. Kubernetes 1.30.0: https://github.com/kubernetes-sigs/controller-tools/releases/tag/envtest-v1.30.0

This PR now extends setup-envtest so that it can download envtest archives from the controller-tools repo. This is also the new default. This PR also deprecates the functionality to download from the kubebuilder bucket or from GCS buckets in general. We anticipate that this will be removed with v0.19 or v0.20. We only keep it around for now as a fallback option.

The UX looks like this:

• Per default setup-envtest will use the envtest binary archives from controller-tools
• To download from a custom index --index can be used
• To use the kubebuilder GCS bucket --use-deprecated-gcs can be used (for now, until either the flag is removed or the bucket goes down, whatever comes first)
• For more details, see the README.md

TBD how we backport this. My initial idea would be to backport it to release-0.17 with --use-deprecated-gcs enabled per default (I don't want to change the behavior in a patch version of setup-envtest)

I did the following manual testing:

# controller-tools
$ go run ./main.go list
(available)  v1.30.0  darwin/arm64
(available)  v1.29.4  darwin/arm64
(available)  v1.28.0  darwin/arm64

$ go run ./main.go use
Version: 1.30.0
OS/Arch: darwin/arm64
sha512: 48c1fa5555f021f8aa365faaa96b106804ce5e3022142a2a166bc91ecd5b22da79f012d20c1a4963e0236df00a214f3311cdd842de4ab19442ca1c83b3bcb859
Path: /Users/buringerst/Library/Application Support/io.kubebuilder.envtest/k8s/1.30.0-darwin-arm64

$ ls -la "/Users/buringerst/Library/Application Support/io.kubebuilder.envtest/k8s/1.30.0-darwin-arm64"
total 372048
dr-xr-xr-x@ 5 buringerst  staff        160 May  6 19:16 .
drwxr-xr-x@ 3 buringerst  staff         96 May  6 19:16 ..
-r-xr-xr-x@ 1 buringerst  staff   26294882 May  6 19:16 etcd
-r-xr-xr-x@ 1 buringerst  staff  112793586 May  6 19:16 kube-apiserver
-r-xr-xr-x@ 1 buringerst  staff   51395554 May  6 19:16 kubectl

$ go run ./main.go cleanup
removed 1.30.0 (darwin/arm64)

# GCS
$ go run ./main.go list --use-gcs
(available)  v1.30.0  darwin/arm64
(available)  v1.29.3  darwin/arm64
(available)  v1.29.1  darwin/arm64
...

$ go run ./main.go use --use-gcs
Version: 1.30.0
OS/Arch: darwin/arm64
md5: Y8pPj/mvjlDyp8XIiCylDQ==
Path: /Users/buringerst/Library/Application Support/io.kubebuilder.envtest/k8s/1.30.0-darwin-arm64

$ ls -la "/Users/buringerst/Library/Application Support/io.kubebuilder.envtest/k8s/1.30.0-darwin-arm64"
total 371512
dr-xr-xr-x@ 5 buringerst  staff        160 May  6 19:18 .
drwxr-xr-x@ 3 buringerst  staff         96 May  6 19:18 ..
-r-xr-xr-x@ 1 buringerst  staff   26019842 May  6 19:18 etcd
-r-xr-xr-x@ 1 buringerst  staff  112793586 May  6 19:18 kube-apiserver
-r-xr-xr-x@ 1 buringerst  staff   51395554 May  6 19:18 kubectl

$ go run ./main.go cleanup
removed 1.30.0 (darwin/arm64)

Additional infos:

Old

• location: https://storage.googleapis.com/kubebuilder-tools
• archive file names: kubebuilder-tools-1.30.0-darwin-amd64.tar.gz
• relative path in the archive: ./kubebuilder/bin

New

• location: https://raw.githubusercontent.com/kubernetes-sigs/controller-tools/master/envtest-releases.yaml
• archive file names: envtest-v1.30.0-darwin-amd64.tar.gz
• relative path in the archive: ./controller-tools/envtest

Testing this via this PR in CAPI: kubernetes-sigs/cluster-api#10569

[sbueringer]

/assign @vincepri @alvaroaleman

/cc @chrischdi @fabriziopandini (fyi)

[sbueringer]

cc @camilamacedo86

[sbueringer]

/hold

Want to create a few more envtest binary archives via controller-tools releases after this PR is approved and before it is merged

[camilamacedo86]

Thank you a lot !!!
It seems great

[sbueringer]

Sorry for the push force. Forgot to push a few small improvements before

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alvaroaleman, sbueringer, vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [alvaroaleman,sbueringer,vincepri]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sbueringer]

Fyi. Going to create a few more releases with setup-envtest binaries in controller-tools early next week and then I'll remove the hold

[sbueringer]

/hold cancel

Created envtest-binaries for the following Kubernetes versions over in controller-tools:

• v1.30.0
• v1.29.4
• v1.29.3
• v1.29.1
• v1.29.0
• v1.28.3
• v1.28.0
• v1.27.1
• v1.26.1
• v1.26.0
• v1.25.0
• v1.24.2
• v1.24.1