Missing atlbase.h file in windows-2022 runner

[kanak-clumio]

Description

We have a CI pipeline that stopped working from 16hrs ago. It is in a private repository and there is no code change in it for weeks.

This is the error we are getting -

fatal error C1083: Cannot open include file: 'atlbase.h': No such file or directory

It uses windows-2022 runner

worked yesterday: https://github.com/actions/runner-images/blob/win22/20240421.1/images/windows/Windows2022-Readme.md
broken today: https://github.com/actions/runner-images/blob/win22/20240514.3/images/windows/Windows2022-Readme.md

Since there was no recent changes to our repository there must be some changes to the windows-2022 runner that made altbase.h no longer available when compiling
I see the package name in win22/20240514.3 is Microsoft.VisualStudio.Component.VC.ATL which is different from the previous release.

We also filed an issue with microsoft for this - microsoft/setup-msbuild#127

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 11
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

Image: windows-2022
Version: 20240514.3.0
Included Software: https://github.com/actions/runner-images/blob/win22/20240514.3/images/windows/Windows2022-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20240514.3

Is it regression?

Yes, it was working in version 20240421.1.0

Expected behavior

We should not see 'atlbase.h' not found error.

Actual behavior

Seeing error that 'atlbase.h' file is not found.

Repro steps

Run a workflow like this and try to build any C++ Visual Studio project which imports this header file -

name: CI
on:
  push:
    branches: [ 'main', 'releases/**' ]
  pull_request:
    branches: [ '**' ]
  workflow_dispatch:
  # Allow manual trigger for debugging the workflow.
jobs:
  CI-checks:
    runs-on: windows-2022
    steps:
      - name: Disable git autocrlf
        run: git config --global core.autocrlf false

      - uses: actions/checkout@v4

      # The msbuild version is dictated by the Windows image provided by GitHub.
      - uses: microsoft/setup-msbuild@v2
      - uses: darenm/Setup-VSTest@v1.2
      - name: Build and test with Sonar Wrapper
        shell: cmd
        run: test-all.cmd

[erik-bershel]

Hey @kanak-clumio!

Take a look here, please: #9701
We excluded multiple Visual Studio 2022 VC components from the image due to a bug in VS which blocks other images users. I may recommend you to add necessary components/version in runtime using snippet:

    steps:
      - name: Delete components
        run: |
                Set-Location "C:\Program Files (x86)\Microsoft Visual Studio\Installer\"
                $InstallPath = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
                $componentsToRemove= @(
                    "Microsoft.VisualStudio.Component.VC.ATL"
                )
                [string]$workloadArgs = $componentsToRemove | ForEach-Object {" --remove " +  $_}
                $Arguments = ('/c', "vs_installer.exe", 'modify', '--installPath', "`"$InstallPath`"",$workloadArgs, '--quiet', '--norestart', '--nocache')
                # should be run twice
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
      
      - name: Install components
        run: |
                Set-Location "C:\Program Files (x86)\Microsoft Visual Studio\Installer\"
                $InstallPath = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
                $componentsToRemove= @(
                  "Microsoft.VisualStudio.Component.VC.v141.ATL"
                  "Microsoft.VisualStudio.Component.VC.14.29.16.11.ATL"
                )
                [string]$workloadArgs = $componentsToRemove | ForEach-Object {" --add " +  $_}
                $Arguments = ('/c', "vs_installer.exe", 'modify', '--installPath', "`"$InstallPath`"",$workloadArgs, '--quiet', '--norestart', '--nocache')
                # should be run twice
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden

[erik-bershel]

@kanak-clumio
I'll wait for your response. One version of the ATL package must contain a header file:

                  "Microsoft.VisualStudio.Component.VC.v141.ATL"
                  "Microsoft.VisualStudio.Component.VC.14.29.16.11.ATL"

I recommend reporting the bug to the Visual Studio Developer Community to understand why it was excluded in the latest version of Component.VC.ATL.

[sodul]

@erik-bershel uninstalling and re-installing the ATL component seems to fix the compilation issue but it add 5-6m to the runs.
We found that the missing header is actually present on disk at:

/c/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.39.33519/atlmfc/include/atlbase.h

So we believe that it was just not installed properly and not added to the import paths of the compiler. If there is a faster way to make the existing atlbase.h discoverable that might be good enough.

[erik-bershel]

Hey @sodul!

Glad to hear that we localised bug. Appreciate your effort! Would be very nice if you report this bag to the Visual Studio so they can fix it for other users.

Same time I found a discussion on SO: https://stackoverflow.com/questions/3898287/c-include-atlbase-h-is-not-found

There were mentioned two options:

After that add additional VC++ directories in the project properties:

Include directories: C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.14.26428\atlmfc\include
Library directories: C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.14.26428\atlmfc\lib\x86
The VC++ compiler should now be able to find the ATL source and library files.

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC>mklink /d atlmfc "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.11.25503\atlmfc"
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.11.25503\atlmfc\lib>mklink /d amd64 "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.11.25503\atlmfc\lib\x64

I am sure that the second one will definitely work if you correct a path.

[erik-bershel]

This action might be also a solution (a workaround to be honest) of the issue: https://github.com/marketplace/actions/setup-msvc-developer-command-prompt

[erik-bershel]

Hey @sodul!

This #9876 should solve your issue I think.

[kanak-clumio]

Thanks @erik-bershel for the help!

I've made a change to add the workaround you suggested of deleting and adding components so that our CI goes through. This increases the build time by 7 min but we can manage it till your above change is available in the next windows-2022 runner version.

[Metadorius]

Had the same issue today when we merged another PR in our project. https://github.com/Phobos-developers/Phobos/actions/runs/9141719975/job/25136537634

Is it possible to somehow revert to the previous runner version where it was working?

[erik-bershel]

Is it possible to somehow revert to the previous runner version where it was working?

Nope. Only to wait for the next release or to update ATL library in runtime.

[provegard]

Do you plan on making a release with the fix in the next few days? I'm considering if I should add the workaround to our pipeline, but if a new release is in the works, I might as well wait.

[erik-bershel]

Hey @provegard!
We are expecting release on the very next week, but without any strict frames and promises - some blocker might happen.

[provegard]

Thanks @erik-bershel, good to know!

[hudsy13]

Hello @erik-bershel any update on when a new 2022 runner will be released. I am encountering the same issue with the missing ATL and deciding on whether to add the code or wait for the next runner.

Thanks

[erik-bershel]

Hey @hudsy13!
Suddenly we got some release blocker. But we are keeping hope to release Windows images ob this week.

[bhanusridhar]

Hi @erik-bershel,
We are having our pipelines running on windows-latest image. With latest image version 20240514.3.0, we are facing issue in VSBuild task as "Error RC1015: cannot open include file 'afxres.h'".

With latest version of windows-2019 image, VSBuild task is working fine.

Will this issue also be fixed as part of the deployment.

Please feel to correct me if I am writing down in the wrong window.

[erik-bershel]

Hey @bhanusridhar!
It seems to be a part of v142 MFC VC component which was previously removed. I'll take a look on what might be done here to eliminate this. Till possible solution it's better to install it in runtime.

[bhanusridhar]

@erik-bershel As it was working on windows-2019 image, we have currently mapped our pipeline to run on windows-2019 image without making any additional changes of including any installation steps.

Hoping to see it work in windows-2022 image as well.
Thanks in advance!

[erik-bershel]

Hey @bhanusridhar!
I'm a bit surprised. Header file afxres.h supposed to be a part of Microsoft.VisualStudio.Component.VC.ATLMFC which is already a part of our installation.

And it seems to be existing on disk:

Run Get-ChildItem -Path "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\" -Recurse -Filter "afxres.h" | Select-Object -ExpandProperty FullName
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.39.33519\atlmfc\include\afxres.h

May I ask you run once next code before build to check if the issue persists with the current version of library only?

      - name: Delete components
        run: |
                Set-Location "C:\Program Files (x86)\Microsoft Visual Studio\Installer\"
                $InstallPath = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
                $componentsToRemove= @(
                    "Microsoft.VisualStudio.Component.VC.ATLMFC"
                    "Microsoft.VisualStudio.Component.VC.ATLMFC.Spectre"                   
                )
                [string]$workloadArgs = $componentsToRemove | ForEach-Object {" --remove " +  $_}
                $Arguments = ('/c', "vs_installer.exe", 'modify', '--installPath', "`"$InstallPath`"",$workloadArgs, '--quiet', '--norestart', '--nocache')
                # should be run twice
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
      
      - name: Install components
        run: |
                Set-Location "C:\Program Files (x86)\Microsoft Visual Studio\Installer\"
                $InstallPath = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
                $componentsToRemove= @(
                  "Microsoft.VisualStudio.Component.VC.14.29.16.11.MFC"
                  "Microsoft.VisualStudio.Component.VC.14.29.16.11.MFC.Spectre"
                  "Microsoft.VisualStudio.Component.VC.14.29.16.11.ATL"
                  "Microsoft.VisualStudio.Component.VC.14.29.16.11.ATL.Spectre"
                )
                [string]$workloadArgs = $componentsToRemove | ForEach-Object {" --add " +  $_}
                $Arguments = ('/c', "vs_installer.exe", 'modify', '--installPath', "`"$InstallPath`"",$workloadArgs, '--quiet', '--norestart', '--nocache')
                # should be run twice
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
                $process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden

[bhanusridhar]

Hi @erik-bershel. I have tested the above.
By including the above scripts before build, Build solution passed now.

So, I guess issue persists with the current version of library only.

[bhanusridhar]

Any update here please @erik-bershel

[erik-bershel]

Hey @bhanusridhar!
Image rollout is in progress. Might be fixed with the update - it has major build tools update. In other case should be raised to the Microsoft Support via Developer Community: https://developercommunity.microsoft.com/VisualStudio

[hudsy13]

Hi @erik-bershel thanks for the updates. The image rollout that is currently in progress is that to correct the atlbase.h missing issue or one of the others that have been raised in this thread?

I just attempted to run our pipeline again (image '20240603.1.0') and it's still throwing an error, so I was looking for clarification.

Thanks

[erik-bershel]

Hey @hudsy13!
Which issue do we speak about? I'm a bit confused here with all other issues. 😄
For the very first mentioned altbase.h it should be resolved in the upcoming release (except the small possibility that the latest build tools update was broken). For the afxres.h it's only workaround available - this one not linked somehow and it seems like a VS component bug.

[hudsy13]

@erik-bershel lol....yeah a bit of thread-jacking going on. I am talking about the original issue of the thread the altbase.h. Thanks for the update, any idea what release number it would be so that I can be on the lookout?

[erik-bershel]

Lets see.

Issue firstly happen to be when we removed v142 and other components. atlbase.h is supposed to be a part of "Microsoft.VisualStudio.Component.VC.ATL" which was directly installed by https://github.com/actions/runner-images/pull/9876/files. I was expecting 20240603 release to solve the initial issue.

Judging by your previous comment, the update did not solve the problem. In this case, it is a bug of the component itself. As a temporary workaround, you can downgrade it in runtime or contact the Visual Studio developers - unfortunately, we are not yet able to install two versions of components at the same time due to a bug.

To check it before reporting I may recommend you to run code from this comment: #9873 (comment). In case it works for you - it's for sure a bug of VS2022.

[sodul]

@erik-bershel We tried to rollback the ATL re-install logic a few minutes ago and unfortunately we are getting the fatal error C1083: Cannot open include file: 'atlbase.h': No such file or directory error.

Here is the output from the setup step:

Current runner version: '2.317.0'
Operating System
  Microsoft Windows Server 2022
  10.0.20348
  Datacenter
Runner Image
  Image: windows-2022
  Version: 20240603.1.0
  Included Software: https://github.com/actions/runner-images/blob/win22/20240603.1/images/windows/Windows2022-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20240603.1
Runner Image Provisioner
  2.0.370.1
GITHUB_TOKEN Permissions
Secret source: Actions
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'actions/checkout@v4' (SHA:a5ac7e51b41094c92402da3b24376905380afc29)
Download action repository 'microsoft/setup-msbuild@v2' (SHA:6fb02220983dee41ce7ae257b6f4d8f9bf5ed4ce)
Download action repository 'darenm/Setup-VSTest@v1.2' (SHA:d9a5dffa3f11d9c27ec42eb69515d3aaeaad9ef8)
Download action repository 'actions/setup-python@v5' (SHA:82c7e631bb3cdc910f68e0081d67478d79c6982d)
Download action repository 'actions/upload-artifact@v4' (SHA:65462800fd760344b1a7b4382951275a0abb4808)
Complete job name: CI-checks

The workaround to uninstall and re-install ATL might still work but our CI started failing which we need to investigate.

[bhanusridhar]

Hey @bhanusridhar! Image rollout is in progress. Might be fixed with the update - it has major build tools update. In other case should be raised to the Microsoft Support via Developer Community: https://developercommunity.microsoft.com/VisualStudio

Seems 20240603 image didn't fixed. May be I should approach Developer community. Thanks for the help in providing workaround. Anyhow, we are using windows-2019.

[erik-bershel]

Hey @sodul and @bhanusridhar!
Major VC components update happened last week. It gave rise to many incompatibility problems and, as a result, it is now very difficult to understand anything. I would prefer to continue to deal with the problem on our side, but a little later. Presumably over the next couple of weeks the main problems with the windows-2022 image should be fixed, after which we will again be able to experiment with a set of libraries. I still hope that we can make a set in such a way that it can be added to the image without compromising other builds.
But I will also be very grateful to you for sending a report to the developers - perhaps they know the problem and they will be able to offer an optimal solution faster.

[sodul]

@erik-bershel where should we send a report? I'm happy to send one but I'm not sure which developers to ping.

[erik-bershel]

@sodul I recommend reporting the bug to the Visual Studio Developer Community