ILLink_links_simple_app_without_analysis_warnings_and_it_runs failing in 8.0.1xx

[marcpopMSFT]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=675180
Build error leg or test failing: Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
Pull request: #40874

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.ArgumentOutOfRangeException : Index was out of range. Must be non-negative and less than the size of the collection",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=675180
Error message validated: [System.ArgumentOutOfRangeException : Index was out of range. Must be non-negative and less than the size of the collection]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 5/14/2024 8:31:19 PM UTC

Report

Build	Definition	Test	Pull Request
682168	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
682120	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#41036
681959	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
681753	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40934
680845	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#41003
681126	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40981
681102	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40976
681038	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#41009
2455368	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39771
2455129	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39751
2454904	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39737
2454776	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39727
2454769	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39724
2454731	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39712
679791	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40981
679645	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
678298	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40934
679290	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
2453469	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39604
2453266	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39604
678560	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40947
678501	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40942
2453172	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39650
678065	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
2452301	dotnet-sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#39604
676938	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
676009	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40884
675796	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
675695	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs
675180	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40874
674516	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40854
675178	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs	#40873
675166	dotnet/sdk	Microsoft.NET.Build.Tests.GivenThatWeWantToFloatWarningLevels.It_maps_analysis_properties_to_globalconfig	#40872
674481	dotnet/sdk	Microsoft.NET.Publish.Tests.GivenThatWeWantToRunILLink.ILLink_links_simple_app_without_analysis_warnings_and_it_runs

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	15	34

[dotnet-policy-service]

@dotnet/illink-contrib a new issue has been filed in the ILLink area, please triage

[marcpopMSFT]

@agocke seems like there's an issue in 8.0.1xx for this test

[sbomer]

It looks like the test is only failing on the Windows fullframework build. I tried the following:

> .\build.cmd
> .\eng\dogfood.cmd
> & "C:\Users\svbomer\src\sdk\.dotnet\dotnet.exe" exec --depsfile "C:\Users\svbomer\src\sdk\artifacts\bin\Tests\Microsoft.NET.Publish.Tests\Debug\Microsoft.NET.Publish.Tests.deps.json" --runtimeconfig "C:\Users\svbomer\src\sdk\artifacts\bin\Tests\Microsoft.NET.Publish.Tests\Debug\Microsoft.NET.Publish.Tests.runtimeconfig.json"  "C:\.tools\.nuget\packages\xunit.runner.console/2.4.2/tools/netcoreapp2.0/xunit.console.dll" "C:\Users\svbomer\src\sdk\artifacts\bin\Tests\Microsoft.NET.Publish.Tests\Debug\Microsoft.NET.Publish.Tests.dll" -noautoreporters -xml "C:\Users\svbomer\src\sdk\artifacts\TestResults\Debug\Microsoft.NET.Publish.Tests_net8.0_x64.xml" -html "C:\Users\svbomer\src\sdk\artifacts\TestResults\Debug\Microsoft.NET.Publish.Tests_net8.0_x64.html" -method "*.ILLink_links_simple_app_without_analysis_warnings_and_it_runs"

And the test passes for me locally. Does this look like the right way to run the test locally in the Windows fullframework setup?

[sbomer]

The failure is happening at:

sdk/src/Tests/Microsoft.NET.Publish.Tests/GivenThatWeWantToRunILLink.cs

Lines 2247 to 2250 in d7cce5a

	var illinkTargetsCommand = new GetValuesCommand(testAsset, "ILLinkTargetsPath", targetFramework: targetFramework);
	illinkTargetsCommand.Properties.Add("PublishTrimmed", "true");
	illinkTargetsCommand.Execute().Should().Pass();
	var illinkTargetsPath = illinkTargetsCommand.GetValues()[0];

which suggests that maybe ILLinkTargetsPath is not set for some reason, or maybe GetValues isn't working correctly. Unfortunately without a repro, I can't do much except add extra logging for when this fails in ci.

[agocke]

Based on the Build Analysis info, this started happening 1 day ago. We should look at the PRs. It is likely that something broke this test.

[dsplaisted]

I looked at this a bit. ILLinkTargetsPath is not set.

@sbomer To repro locally (hopefully I'm not forgetting anything):

• Get code for this PR
• Follow steps to run tests in Visual Studio
  • build.cmd
  • .\artifacts\sdk-build-env.bat
  • devenv sdk.sln
• Apply fix from Fix getting path to full framework MSBuild #40934
• Set TestCommandLine.UseFullFrameworkMSBuild to true to run tests using full MSBuild
• Run test using test explorer in Visual Studio

That should repro the failure. To investigate more, you can (in another prompt) run eng\dogfood.cmd (see here). Then you can check the test output log to find the folder it created the test project in and the commands it run. You can rerun those yourselves and add -bl to create a binlog, or modify the project or commands as necessary.

[nagilson]

Thanks everyone for your investigation. This is blocking the rest of the teams from flowing their code. I think this should be addressed but as a warning we are going to disable the test to get other things moving along @sbomer

[sbomer]

Thanks for the instructions @dsplaisted, that let me repro the failure.

The test is running GetValuesCommand with PublishTrimmed set to true in the generated <project>.WriteValuesToFile.g.targets. In the failure case, ImportProjectExtensionProps is false (due to https://github.com/dotnet/msbuild/blob/147ecadd19ae031d5a511ad55908cff9bcdc17c5/src/Tasks/Microsoft.Common.props#L68 imported from full framework MSBuild props) during restore, so this file isn't imported, we don't set PublishTrimmed, Microsoft.NET.ILLink.Tasks isn't restored, and then ILLinkTargetsPath never gets set.

I was wondering why this wasn't failing in main on .NET Core, and found that the GetValuesCommand behavior was fixed as part of #39088 to not rely on ImportProjectExtensionProps. So we could fix this by backporting the GetValuesCommand fix.

But a simpler fix might be to just set PublishTrimmed in the project file for this testcase. I'll make a PR.

[agocke]

Should be fixed by #41135

[nagilson]

Thank you!