More robust identification of the build OS in Microsoft.AspNetCore.App.Runtime.csproj #37400

omajid · 2021-10-08T17:05:13Z

aspnetcore/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj

Lines 101 to 110 in a0b950f

    
               <!-- 
        
                 Determine the crossgen2 package path property name. Special case linux-musl-arm and linux-musl-arm64 because they 
        
                 are built on an Ubuntu container with cross compilation tools. linux-musl-x64 is built in an alpine container. 
        
                 Special case the crossgen2 package reference on Windows to avoid the x86 package when building in Visual Studio. 
        
               --> 
        
               <BuildOsName>$(TargetOsName)</BuildOsName> 
        
               <BuildOsName Condition="'$(TargetOsName)' == 'linux-musl' and '$(TargetArchitecture)'!='x64'">linux</BuildOsName> 
        
               <Crossgen2BuildArchitecture Condition=" '$(BuildOsName)' == 'win' ">x64</Crossgen2BuildArchitecture> 
        
               <Crossgen2BuildArchitecture Condition=" '$(Crossgen2BuildArchitecture)' == '' ">$(BuildArchitecture)</Crossgen2BuildArchitecture> 
        
               <Crossgen2PackageRootVariableName>PkgMicrosoft_NETCore_App_Crossgen2_$(BuildOsName)-$(Crossgen2BuildArchitecture)</Crossgen2PackageRootVariableName>

This hardcodes some assumptions that are valid in the CI legs, but may not be in other systems.

For example, if this project is being built on Linux arm64 hardware using an OS that uses musl - in other words, linux-musl-arm64, it will try and use the linux-arm64 crossgen2 instead.

Another example, if this code is being built on Windows on arm64 (or really, any architecture that's not x64), it will use the wrong crossgen, win-x64, which will most likely fail to run on an Arm 64 machine.

See also this comment posted by @dougbu in #37374 (comment)

@dougbu

The text was updated successfully, but these errors were encountered:

dougbu · 2021-10-08T18:49:49Z

@davidwrighton @rainersigwald the requirements here are very similar to the requests in dotnet/msbuild#2468 and dotnet/msbuild#5065. We should download the crossgen2 package for the current OS / processor and ignore the requested $(TargetArchitecture) in that logic.

$([MSBuild]::IsOSPlatform('FreeBSD')), $([MSBuild]::IsOSPlatform('Linux')), $([MSBuild]::IsOSPlatform('OSX')), and $([MSBuild]::IsOSPlatform('Windows')) would get us partway there.

Differentiating between arm, arm64, x64 and x86 on Windows probably just means checking $(PROCESSOR_ARCHITECTURE). May be a bit more complicated on Linux and OSX because there are a few different uname -m values for similar-enough architectures.

The special case for using x64 on Windows should be used only when $(PROCESSOR_ARCHITECTURE) is X86 and $(BuildingInsideVisualStudio) is true.

Differentiating between Linux MUSL and generic Linux probably requires

checking whether /lib/ld-musl-x86_64.so.1 exists (it does on Alpine, so why aren't we using a linux-musl package there❔),
<Exec />ing apk info musl and checking for errors, or
<Exec />ing ldd /bin/ls and greping for musl.

Which do you prefer❔

I'm probably missing a few things and would appreciate your guidance❕

omajid · 2022-03-15T18:20:50Z

<Exec />ing ldd /bin/ls and greping for musl.

FWIW, the approach runtime takes is: ldd --version 2>&1 | grep -q musl. That's the approach used in dotnet/installer#13410

omajid · 2022-04-13T17:32:38Z

I have been trying to build main branch on Alpine x64 (musl), and running into issues. This change to eng/build.sh let me get started:

diff --git a/eng/build.sh b/eng/build.sh
index a6b6ba99be..9a4bc957c8 100755
--- a/eng/build.sh
+++ b/eng/build.sh
@@ -37,7 +37,11 @@ runtime_source_feed_key=''
 if [ "$(uname)" = "Darwin" ]; then
     target_os_name='osx'
 else
-    target_os_name='linux'
+    if (ldd --version 2>&1 || true) | grep -q musl; then
+        target_os_name='linux-musl'
+    else
+        target_os_name='linux'
+    fi
 fi
 
 msbuild_args=()

But I am running into errors because Grpc.Tools isn't available for linux-musl-x64: grpc/grpc#18428, grpc/grpc#24188.

/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003: The specified task executable "/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc" could not be run. System.ComponentModel.Win32Exception (2): An error occurred trying to start process '/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc' with working directory '/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite'. No such file or directory [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]

@ayakael did you run into this? How did you work around it?

ayakael · 2022-04-17T01:03:41Z

I have been trying to build main branch on Alpine x64 (musl), and running into issues. This change to eng/build.sh let me get started:

diff --git a/eng/build.sh b/eng/build.sh
index a6b6ba99be..9a4bc957c8 100755
--- a/eng/build.sh
+++ b/eng/build.sh
@@ -37,7 +37,11 @@ runtime_source_feed_key=''
 if [ "$(uname)" = "Darwin" ]; then
     target_os_name='osx'
 else
-    target_os_name='linux'
+    if (ldd --version 2>&1 || true) | grep -q musl; then
+        target_os_name='linux-musl'
+    else
+        target_os_name='linux'
+    fi
 fi
 
 msbuild_args=()

But I am running into errors because Grpc.Tools isn't available for linux-musl-x64: grpc/grpc#18428, grpc/grpc#24188.

/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003: The specified task executable "/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc" could not be run. System.ComponentModel.Win32Exception (2): An error occurred trying to start process '/home/dotnet/.nuget/packages/grpc.tools/2.43.0/tools/linux_x64/protoc' with working directory '/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite'. No such file or directory [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj] 
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]
/home/dotnet/.nuget/packages/grpc.tools/2.43.0/build/_protobuf/Google.Protobuf.Tools.targets(280,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/home/dotnet/aspnetcore/src/Grpc/Interop/test/testassets/InteropWebsite/InteropWebsite.csproj]

@ayakael did you run into this? How did you work around it?

I have not encoutnered this issue, investigating why you have it but source-build aspnetcore doesn't

ayakael · 2022-04-17T01:09:01Z

@omajid Oh right, I go around this issue by building aspnetcore with --os-name=linux-musl. Thus, target_os_name is overriden. Aspnetcore should be able to detect linux-musl without source-build doing this check. See https://github.com/dotnet/installer/pull/13410/files#diff-1773f0861dc9197a3da55e87fbac6ca93eadfc48389e198085b1457422895b92

dougbu mentioned this issue Oct 8, 2021

Fix build on arm64 machines #37374

Merged

mkArtakMSFT added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Oct 8, 2021

omajid mentioned this issue Jan 27, 2022

Aspnetcore doesn't look for linux-musl nupkgs when building on linux-musl platform #39822

Open

ayakael mentioned this issue Mar 14, 2022

[release/6.0.1xx] Fix build against musl dotnet/installer#13410

Closed

dougbu self-assigned this Apr 11, 2022

solrevdev mentioned this issue Jun 10, 2022

Docker + Alpine Linux - AspNetCore.SassCompiler.targets(11,5): error : Error running sass compiler: koenvzeijl/AspNetCore.SassCompiler#93

Closed

dougbu removed their assignment Mar 7, 2023

wtgodbe added this to the Backlog milestone Feb 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More robust identification of the build OS in Microsoft.AspNetCore.App.Runtime.csproj #37400

More robust identification of the build OS in Microsoft.AspNetCore.App.Runtime.csproj #37400

omajid commented Oct 8, 2021 •

edited

dougbu commented Oct 8, 2021

omajid commented Mar 15, 2022

omajid commented Apr 13, 2022 •

edited

ayakael commented Apr 17, 2022 •

edited

ayakael commented Apr 17, 2022 •

edited

More robust identification of the build OS in Microsoft.AspNetCore.App.Runtime.csproj #37400

More robust identification of the build OS in Microsoft.AspNetCore.App.Runtime.csproj #37400

Comments

omajid commented Oct 8, 2021 • edited

dougbu commented Oct 8, 2021

omajid commented Mar 15, 2022

omajid commented Apr 13, 2022 • edited

ayakael commented Apr 17, 2022 • edited

ayakael commented Apr 17, 2022 • edited

omajid commented Oct 8, 2021 •

edited

omajid commented Apr 13, 2022 •

edited

ayakael commented Apr 17, 2022 •

edited

ayakael commented Apr 17, 2022 •

edited