Proposal: Adding Kubernetes Target to spin deploy Command

[Mossaka]

Hey Fermyon friends! I would like to propose an extention to spin deploy command to target Kubernetes clusters. Currently, Spin only supports deploying Spin applications to Fermyon Cloud platform. Adding the ability to deploy to K8s clusters will provide greater flexibility for developers who may want to test their applications in a local cluster or host them on a different managed cloud platform, such as Azure Kubernetes Service (AKS).

This proposal will leverage on the capability of spin oci to package to push a Spin application to an OCI registry. The spin deploy k8s command will look for a default k8s context, create a default k8s deployment with 3 replicas that pull the Spin application from the OCI registry. Additionally, it will look for a registered runtime class named wasmtime-spin-v1 spin shim via the containerd-wasm-shims project built on top of runwasi. Finally it will deploy the Spin workload to the target K8s cluster.

This is an initial idea and far from a complete draft. I would love to hear your thoughts and opinions on this proposal and welcome any feedback or suggestions.

[itowlson]

This would be really interesting to have, but my first reaction is to suggest doing it as a plugin. We are thinking about moving Fermyon Cloud deployment to a plugin as well, so this would fit with the long-term architecture we plan for deployment commands. It would mean the UI would end up as something like spin k8s deploy rather than spin deploy k8s, but again that is consistent how we want to do things for Fermyon Cloud. Would you be interested in exploring that approach with us?

[Mossaka]

Yes, I am interested in exploring this approach! I just saw the proposals in #1117 and #1118 . I totally support that the k8s target should be aligned with the long-term architecture of Spin, and spin k8s deploy sounds great to me!

[Mossaka]

Is writing a SIP a good starting point for this work?

[radu-matei]

cc @bacongobbler, who is interested in this as well.

[itowlson]

@Mossaka We've not previously done SIPs for plugins - it's kind of free for authors to do their own thing. But that's certainly a great way to refine scope, UX and design, and get feedback from other folks who are interested. I'd say go for it. Are you okay with driving that?

[bacongobbler]

A generic spin deploy relies on us agreeing to implement a generic spin login experience. I'm unfamiliar with how spin k8s deploy would ask the user to "log in", push the Spin app to an OCI/Bindle endpoint, and inform Kubernetes how to pull that app from the registry.

We would also need to agree what the behavior of a "deploy" means for both the Fermyon Cloud and for Kubernetes. For example, we plan to introduce custom domains to the Fermyon Cloud. That has potential to leak those abstractions to spin deploy as the user may want to change their application's domain name at deploy time. As we introduce new features to Fermyon Cloud, how would those translate to the containerd-wasm-shim on kubernetes?

As others have stated, my first impression on this proposal would be to start both spin cloud deploy and spin k8s deploy as separate plugins, with the existing spin deploy command aliased to spin cloud deploy for compatibility with earlier versions of Spin.

When we decide to implement Spin 2, we can make a more coordinated effort to a generic spin login and spin deploy experience if our goals align.

What are your thoughts on that, @Mossaka?

[Mossaka]

I'm unfamiliar with how spin k8s deploy would ask the user to "log in", push the Spin app to an OCI/Bindle endpoint, and inform Kubernetes how to pull that app from the registry.

To get started, could spin k8s deploy assumes the user has ran docker login?

As we introduce new features to Fermyon Cloud, how would those translate to the containerd-wasm-shim on kubernetes?

I am not sure becasue I need more background on how will these features be implemented concretly. But ideally features that are specific to Fermyon Cloud like "custom domains" should be decoupled from features to containerd-wasm-shim. Are you suggesting that this is difficult to achieve?

my first impression on this proposal would be to start both spin cloud deploy and spin k8s deploy as separate plugins, with the existing spin deploy command aliased to spin cloud deploy for compatibility with earlier versions of Spin.

Yes, I agree. I am mostly interested in the experience of deploying to K8s so I will be focusing on spin k8s deploy instead.

When we decide to implement Spin 2, we can make a more coordinated effort to a generic spin login and spin deploy experience if our goals align.

Oh interesting! Happy to! I am not seeing this as a blocker to the spin k8s deploy proposal to the current Spin implementation, right?

[itowlson]

@Mossaka Agree, I don't think future convergence is a blocker. I would love to get something working; then if and when we tackle convergence, we will at least have a working reference point to guide that design.

[Mossaka]

https://github.com/chrismatteson/spin-plugin-k8s

It looks like this repo has added a k8s plugin to spin. Very nice!

[melissaklein24]

@Mossaka are you ok with closing this issue?

[itowlson]

No dissent so closing