Could not write lines to file CoverletSourceRootsMapping - in use by another process

[ta264]

Our build process with coverlet > 3.0 tends to fail with an error like this:

C:\Users\VssAdministrator\.nuget\packages\coverlet.collector\3.0.4-preview.25\build\netstandard1.0\coverlet.collector.targets(48,5): error MSB3491: Could not write lines to file "..\..\_tests\net5.0\linux-musl-arm64\CoverletSourceRootsMapping". The process cannot access the file 'D:\a\1\s\_tests\net5.0\linux-musl-arm64\CoverletSourceRootsMapping' because it is being used by another process. [D:\a\1\s\src\NzbDrone.Host.Test\Radarr.Host.Test.csproj]
  Radarr.Windows.Test -> D:\a\1\s\_tests\net5.0\linux-musl-x64\publish\

I suspect this is because we have a centralized output directory (not one per project).

  <PropertyGroup>
    <!-- Output to _output and _tests respectively -->
    <OutputPath Condition="'$(RadarrProject)'=='true'">$(RadarrRootDir)_output\</OutputPath>
    <OutputPath Condition="'$(RadarrOutputType)'=='Test'">$(RadarrRootDir)_tests\</OutputPath>
  </PropertyGroup>

Can this be worked around at all?

[MarcoRossignoli]

Unfortunately at the moment CoverletSourceRootsMapping uses $OutputPath https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.collector/build/netstandard1.0/coverlet.collector.targets#L46

[LevYas]

I have the same issue. Looks like as a workaround we can build tests into a subdirectory of that centralized directory.

[MrOwen]

@LevYas can you expand on your workaround? I'm using --output flag with the dotnet cli so I don't specify the output at the project level but the solution I'm building contains multiple projects. Does that mean a different build output needs to be specified in each project file in the solution and I can't use the cli --output flag? Seems like an arduous workaround that's error prone if you expect to put this all into an automated build.

If this is a race condition (I expect it is as I run into about 75% of the time), can it be solved with some retry logic? I grabbed the following from the same dotnet publish step and there is some retry logic going on so I'm wondering if the same can be done for the CoverletSourceRootsMapping file:

Copying file from "C:\Users\ContainerAdministrator\.nuget\packages\coverlet.collector\3.0.3\build\netstandard1.0\System.Linq.Expressions.dll" to "C:\__w\165\b\SampleAPI\System.Linq.Expressions.dll".
9:6>C:\Users\ContainerAdministrator\.nuget\packages\coverlet.collector\3.0.3\build\netstandard1.0\coverlet.collector.targets(16,5): warning MSB3026: Could not copy "C:\Users\ContainerAdministrator\.nuget\packages\coverlet.collector\3.0.3\build\netstandard1.0\System.Linq.Expressions.dll" to "C:\__w\165\b\SampleAPI\System.Linq.Expressions.dll". Beginning retry 1 in 1000ms. The process cannot access the file 'C:\__w\165\b\SampleAPI\System.Linq.Expressions.dll' because it is being used by another process.  [C:\__w\165\s\project\API\Sample.Tests\Sample.Tests.csproj]

[LevYas]

@MrOwen Sure. I have output setup per project for all my projects, and I use CI/CD. Here's how it looks like for test project:

<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
<AppendRuntimeIdentifierToOutputPath>false</AppendRuntimeIdentifierToOutputPath
<OutputPath>..\..\..\Bin\$(Configuration)\Tests\Unit</OutputPath>

[MrOwen]

Ah, that's what I was afraid of. I'm working on a project with 60 test projects... I actually ended up upgrading Microsoft.NET.Test.Sdk, MSTest.TestAdapter, and MSTest.TestFramework to their latest versions since we were way behind and all issues seem to have gone away. Tried looking in the changelogs for each of these to see if some sort of race-condition was fixed but didn't spot any specifically. Either way, for future peeps stumbling on this problem, there's a few fixes in this thread now. Thanks again @LevYas!

[echalone]

We had the same problem now. Our "workaround" for now was to only build with a single process instead of multiple. Not ideal though as the build will take longer once the solution grows.

[zplan]

Same problem here.
Build with single process with a solution >50 projects is not an option.

[MarcoRossignoli]

For now a possible solutions could be update coverlet to write the mapping in a folder like $(OutputPath)\CoverletSourceMappings\CoverletSourceRootsMapping_{Guid}` and merge/use when needed.

[cboulva99]

Hello!

Any development on this subject? It seems since we migrated to .net6, the bug occurs almost always. We did not have the problem before that.

[maxpavlov]

Hello!

Any development on this subject? It seems since we migrated to .net6, the bug occurs almost always. We did not have the problem before that.

Yes, we observe the same - 85% chance of error, prior to net6 we haven't seen this error.

[DamianSuess]

Any word on the progress, or if the community should aim to assist on this fix?
For efficiency purposes, we keep our tests in an Output\Tests\ folder in our projects for accessibility, reduction of multiple DLLs, and easy-to-clean purposes.

However, we continuously get the error, "The process cannot access the file '...\CoverletSourceRootsMapping' because it is being used by another process. This is problematic for our build machines.

Thank you.

[MarcoRossignoli]

It's on high priority, but didn't have time to work on it yet, bit busy times.

Can someone provide a repro to speedups a bit the analysis?

Coverlet is open source if someone wants to take this one I'm glad to review and help.
In case let me know that I'll assign the issue to avoid to accidentally waste time.

[japj]

We have the same issue, basically the scenario is:

• a solution with multiple test projects
• output to the same folder
• build the solution (with msbuild /m to enable msbuild concurrency)

Since this is a "build timing" bug related to writing to the same file, it is not 100% reproducable.

However, I think a good fix would probably be to append or prefix the MSBuildProjectName to the CoverletSourceRootsMapping file, e.g.

_sourceRootMappingFilePath: "$([MSBuild]::EnsureTrailingSlash('$(OutputPath)'))CoverletSourceRootsMapping -$(MSBuildProjectName)"

This will ensure that the CoverletSourceRootsMapping file is unique per MSBuildProject (i.e. per Assembly)

[DamianSuess]

I'm all for @japj's suggestion, it's cleaner than a GUID suffix. If we created a branch for this, are there sufficient existing unit tests against this to support such a change?

[MarcoRossignoli]

I don't think that is enough only update with the guid some code needs review

https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Helpers/SourceRootTranslator.cs
https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Reporters/CoberturaReporter.cs#L71

Tests are here https://github.com/coverlet-coverage/coverlet/blob/master/test/coverlet.integration.tests/DeterministicBuild.cs

[DamianSuess]

Thank you, Marco, for providing your insight & experience to help keep this moving 👍

[MarcoRossignoli]

I've started to work on this one the idea is to create the file mapping using some sort of hash for the "current" test container to avoid collision in case of solutions with more containers in the same output folder.
Something like

Let's see it it can work...there're some place to change:

• targets msbuild/collectors
• SourceRootTranslator
• Tests

cc: @daveMueller

[MarcoRossignoli]

Hi all,

I've published a test package here https://f.feedz.io/marcorossignoli/coverletunofficial/nuget/index.json the versions are

coverlet.collector 3.1.3-preview.7.gb1603b798b
coverlet.msbuild 3.1.3-preview.7.gb1603b798b

I need help, can someone give it a try and let me know if it's working?

[bretthysuik]

@MarcoRossignoli: I can confirm that coverlet.collector 3.1.3-preview.7.gb1603b798b fixed the problem on my end.

[MarcoRossignoli]

thanks @bretthysuik can you confirm that in you output folder you see a list of CoverletSourceRootsMapping_dlltestname?

Can someone else give a try to coverlet.msbuild 3.1.3-preview.7.gb1603b798b?

[bretthysuik]

Yes, a CoverletSourceRootsMapping_dlltestname.Tests file was created for every test project in the solution.

[walljumper]

Works for me as well.

[MarcoRossignoli]

Can someone cover the coverlet.msbuild use case pls?

[bretthysuik]

Can someone cover the coverlet.msbuild use case pls?

coverlet.msbuild 3.1.3-preview.7.gb1603b798b looks to be working as well. A coverage.json file is created for each test project in the solution, and no errors occurred related to CoverletSourceRootsMapping.

[MarcoRossignoli]

Thanks for the testing.

[LucidoKeith]

Is there a target release date for the 3.1.3 version?

[MarcoRossignoli]

Not yet, I'm having issue with this fix in CI...I need to investigate a bit more.

[C302]

Any update on the official 3.1.3 version?

[MarcoRossignoli]

This fix is not yet merged...I'm having some hassle on tests and I'm a bit busy, I'll try to work on it asap.

[ravi-chandra-gl]

Any update on the 3.1.3 version?

[Kizuto3]

Any updates on the issue?

[MarcoRossignoli]

unfortunately not at the moment I didn't have much time to work on it in last months the PR started here #1320 but there're some issue with testing in CI that doesn't repro on my local so more investigation is needed, if someone wants to take over it could be a huge help cc: @daveMueller

[MarcoRossignoli]

@daveMueller did the fix..pls someone tomorrow can try the dogfood? https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/ConsumeNightlyBuild.md