Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E test happy path #2477

Open
Tracked by #2512
phillebaba opened this issue May 6, 2024 · 1 comment
Open
Tracked by #2512

E2E test happy path #2477

phillebaba opened this issue May 6, 2024 · 1 comment
Labels
tech-debt 💳 Debt that the team has charged and needs to repay

Comments

@phillebaba
Copy link
Collaborator

Describe what should be investigated or refactored

The current e2e tests are large and require a certain amount of knowledge of Zarf and its code base to run. After spending some time getting the tests to run as a new Zarf user I have identified some friction points which could easily be addressed.

  1. The dependencies documentation does not specify that Docker and buildx are required dependencies. While it could be assumed that most people will have Docker installed, they may not have buildx installed.

1. GoLang >= 1.21.x
2. Make
3. Any clean K8s cluster (local or remote) or Linux with `root` if you want to use the Zarf-installed K3s cluster

  1. The tests assume, unless run in appliance mode that access to a Kubernetes cluster is properly configured. A check exists but it does not seem to do anything as it does not check the error. This results in errors much later, which wastes time and becomes harder to decipher.

zarf/src/test/common.go

Lines 57 to 62 in 0fa2173

func (e2e *ZarfE2ETest) SetupWithCluster(t *testing.T) {
if !e2e.RunClusterTests {
t.Skip("")
}
_ = exec.CmdWithPrint("sh", "-c", fmt.Sprintf("%s tools kubectl describe nodes | grep -A 99 Non-terminated", e2e.ZarfBinPath))
}

  1. It would be nice to have a minimal test run documented to make it easy to verify that the e2e tests can run, as it would take a long time for all the tests to run completely.

  2. A lot of these tests seem to have little impact on each other, even those that are running against a cluster. While some test cases are marked to run in parallel some are not. This should probably be explored further to speed up test runs.

  3. It could be good to move the test images from Docker Hub to GHCR to reduce the risk of rate limiting occurring when running the e2e locally multiple times in a day.

@phillebaba phillebaba added the tech-debt 💳 Debt that the team has charged and needs to repay label May 6, 2024
@satscottfoster
Copy link

How do I vote for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech-debt 💳 Debt that the team has charged and needs to repay
Projects
Status: No status
Development

No branches or pull requests

2 participants