Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and simplify storage tests #2147

Merged
merged 10 commits into from
Oct 24, 2023
Merged

Fix and simplify storage tests #2147

merged 10 commits into from
Oct 24, 2023

Conversation

mtrmac
Copy link
Collaborator

@mtrmac mtrmac commented Oct 14, 2023

For CRI-O (and, maybe, Podman/libimate), I want to introduce a new reference lookup method, and that needs tests against a locally-created image.

It turns out those tests required root, and we never run them as root. So a large part of storage transport tests have been unused for 4 years now.

This fixes that problem:
Fixes #1729 . And at least on a Mac (?), the tests don’t actually require any privileges, so run them unconditionally.

This PR also consolidates the existing code to create/test images to significantly reduce duplication, to benefit from testify, and to make it easier to add more test cases, now that I have been reading the code.

@vrothberg PTAL. Cc: @saschagrunert

@mtrmac
Copy link
Collaborator Author

mtrmac commented Oct 14, 2023

And at least on a Mac (?), the tests don’t actually require any privileges, so run them unconditionally.

Well, so much for that part. To be continued.

Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM

@rhatdan
Copy link
Member

rhatdan commented Oct 16, 2023

LGTM

@mtrmac mtrmac added the kind/bug A defect in an existing functionality (or a PR fixing it) label Oct 16, 2023
@TomSweeneyRedHat
Copy link
Member

Changes LGTM, but there are a number of "Operation not Permited" errors.

@mtrmac mtrmac marked this pull request as draft October 19, 2023 12:18
@mtrmac
Copy link
Collaborator Author

mtrmac commented Oct 19, 2023

I think containers/storage#1735 might get us a bit closer, but ultimately I don’t think c/storage can run entirely unprivileged (without adding some more options throughout the codebase).

So I currently plan to settle with making the tests be macOS-only (where ~everything is fake, due to containers/storage#811 ). They won’t run on CI, but at least frequently on my machine.

containers#731 introduced
a WaitGroup which is waited but never signaled, and a nonsensical
error reporting mechanism (where errors are pushed but don't terminate
future operations).

Instead, rely on PipeWriter.CloseWithError() to propagate failures
from the goroutine.

And move all of the body of the goroutine into a separate function,
so that we can use defer() to ensure CloseWithError() is _always_
called.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
defer cleanup immediately after creating objects, use shorter error
checking.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The old ones are too fake and nowadays rejected when reading.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that we can update it in one place.

Should not change (test) behavior, other than the precise
skip message.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
c/storage on macOS runs in a special mode which does not
require any privileges.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Return a struct so that we can maintain an array of them.
Create a helper to call PutBlob and format config data.

Should not change (test) behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add createUncommittedImageDest / createImage which
calls the ImageDestination APIs as required, so that test bodies
can focus on the primary purpose.

Should not change (test) behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
TestWriteRead could probably share more with createImage,
but this just might do.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
nil is defined to mean the same thing; if anything, nil is the
value more likely to trigger crashes.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
@mtrmac
Copy link
Collaborator Author

mtrmac commented Oct 24, 2023

So I currently plan to settle with making the tests be macOS-only (where ~everything is fake, due to containers/storage#811 ). They won’t run on CI, but at least frequently on my machine.

I have modified the PR to work that way. containers/storage#1735 would be cleaner but is not strictly required, so I have dropped the c/storage update requirement here.

@mtrmac mtrmac marked this pull request as ready for review October 24, 2023 19:30
@mtrmac
Copy link
Collaborator Author

mtrmac commented Oct 24, 2023

Ready for review/merging again.

@rhatdan
Copy link
Member

rhatdan commented Oct 24, 2023

LGTM

@rhatdan rhatdan merged commit 02ddb38 into containers:main Oct 24, 2023
9 checks passed
@mtrmac mtrmac deleted the storage-tests branch October 25, 2023 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A defect in an existing functionality (or a PR fixing it)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storage tests are mostly broken when run as root
5 participants