Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dockerfiles for "unknown" Linux #679

Closed
wants to merge 1 commit into from

Conversation

omajid
Copy link
Member

@omajid omajid commented Aug 30, 2022

This is a fake Linux distribution, which has a name that's not part of the .NET Runtime Identifier catalog/graph.

The goal is to be able to test .NET behaviour when it builds/runs on an Linux distribution that's completely standard and otherwise supported by .NET, except with an unknown name.

I am not sure of the name for this Dockerfile. @tmds suggested "banana linux" previously; is that more descriptive?

I have based this on CentOS Stream 9, but I open to switching this to some other platform instead.

Relates to dotnet/source-build#297 dotnet/runtime#22143 dotnet/runtime#48507 and probably a ton of others.

@tmds
Copy link
Member

tmds commented Sep 1, 2022

If we make the /etc/os-release file writable in existing images, any can serve for an unknown distro tests.

Copy link
Member

@mthalman mthalman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer defining a new variant of an existing image type instead of a whole new "unknown" directory. There's no need to build a whole other image when all we want different is the os-release file. You can have a Dockerfile based on the image produced by https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/main/src/centos/stream9/Dockerfile and then overwrite /etc/os-release. Just name the variant "unknown" and include README that you have.

This is a fake Linux distribution, which has a name that's not part of
the .NET Runtime Identifier catalog/graph.

The goal is to be able to test .NET behaviour when it builds/runs on an
Linux distribution that's completely standard and otherwise supported by
.NET, except with an unknown name.
@omajid
Copy link
Member Author

omajid commented Sep 1, 2022

Thanks for the feedback!

I have turned this into a variant of an existing distro. I picked Debian 11 because CentOS Stream 9 still needs some work.

@tmds
Copy link
Member

tmds commented Sep 2, 2022

If we make the /etc/os-release file writable in existing images, any can serve for an unknown distro tests.

This is what I prefer. It allows to set the rid as part of the test, which gives it more visibility.
It also allows to test cases like: unknown version of a known distro.
And it gives more flexibility as certain parts of the test can use a known rid.

@mthalman
Copy link
Member

mthalman commented Sep 2, 2022

If we make the /etc/os-release file writable in existing images, any can serve for an unknown distro tests.

This is what I prefer. It allows to set the rid as part of the test, which gives it more visibility. It also allows to test cases like: unknown version of a known distro. And it gives more flexibility as certain parts of the test can use a known rid.

That approach seems reasonable to me. As long as the container is run as root, the file can be overwritten. The Helix container is the only run that doesn't run as root by default. I'm not aware of how these images are intended to be consumed. Is it possible to run them and include custom commands to overwrite the os-release file? If so, that seems like a better, more general purpose solution.

@tmds
Copy link
Member

tmds commented Sep 2, 2022

As long as the container is run as root, the file can be overwritten.

I was thinking:

# chmod a+w /etc/os-release

And, optionally include a small script that allows to easily set the fields .NET cares about.

@omajid
Copy link
Member Author

omajid commented Sep 2, 2022

If we make the /etc/os-release file writable in existing images, any can serve for an unknown distro tests.

That's true, but I am thinking of integrating this into existing CI runs and that would make that part harder.

With the current approach, we produce container images, just add it to the list of CI jobs and CI can just build/test on this.

If we make the files writable, I will have to conditionalize and patch the CI jobs themselves to identify when the RID need to be patched, then do the patching.. for all CI jobs in relevant repos?

This current approach seems like the easier way forward; am I just being lazy?

@tmds
Copy link
Member

tmds commented Sep 5, 2022

I've added a build for an unknown rid in dotnet/runtime#74504.

Besides the rid, I had to add another parameter that tells the runtime build what the unknown rid inherits: glibc-based linux (linux) or musl-based linux (linux-musl).

          platform:
            targetRID: banana.24-x64
            runtimeOS: linux

Only patching /etc/os-release is not enough.

The jobs are already heavily parameterized and scripted, I don't think we're making things more complex.

We'd like source-build legs of repos to get parameters passed to them same way as top-level source-build passes them. For the runtime repo, the rid is passed in the TargetRid property. It's not read from /etc/os-release.

Maybe for the banana.24 build, we'd like the /etc/os-release rid to be orange.18 to ensure we're not mixing bananas and oranges.

@mthalman
Copy link
Member

mthalman commented Sep 8, 2022

@omajid - Based on the comment from @tmds above, is any work needed in this PR or can it be closed?

@omajid
Copy link
Member Author

omajid commented Sep 8, 2022

It seems like I should update all the container images to include a world-writable /etc/os-release so we can test any image against the banana/orange distro that dotnet/runtime#74504 would enable?

@tmds
Copy link
Member

tmds commented Sep 8, 2022

We need at least a glibc based and a musl based image where we can control the rid.

@mthalman
Copy link
Member

mthalman commented May 3, 2023

@omajid - Any further changes on this based on the feedback?

@omajid
Copy link
Member Author

omajid commented May 4, 2023

@mthalman I will have to rework this PR entirely based on the feedback. Shall I close it or move it to draft?

@mthalman
Copy link
Member

mthalman commented May 4, 2023

@mthalman I will have to rework this PR entirely based on the feedback. Shall I close it or move it to draft?

I guess that depends on how soon you think you'll get to this. If it's going to be soon, draft is fine. Otherwise, I think closing would be best.

@omajid omajid closed this May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants