macos-13-arm64 agents disconnecting frequently

[rohanp-91]

Description

We are seeing a lot of instances of agents getting disconnected for the macos-13-arm64 agents:

##[error]
,##[error]We stopped hearing from agent Azure Pipelines 173. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error. For more information, see: https://go.microsoft.com/fwlink/?linkid=846610
Pool: Azure Pipelines
Image: macOS-13-arm64
Agent: Azure Pipelines 173
Started: Today at 6:49 PM
Duration: 38m 27s

Example run: https://msasg.visualstudio.com/Bing_Ads/_build/results?buildId=45247131&view=logs&j=c2497230-5798-5958-1bf3-10d84887536f

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• macOS 11
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

Agent name: 'Azure Pipelines 173'
Agent machine name: 'Mac-1712713774296'
Current agent version: '3.236.1'
Operating System
macOS
13.6.6
22G630
Runner Image
Image: macos-13-arm64
Version: 20240405.1
Included Software: https://github.com/actions/runner-images/blob/macos-13-arm64/20240405.1/images/macos/macos-13-arm64-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-13-arm64%2F20240405.1
Runner Image Provisioner
2.0.361.1+33a726b54233caea13315dba3887b0e7c5a1d482
Current image version: '20240405.1'

Is it regression?

Not sure

Expected behavior

Agent disconnection should not be so frequent.

Actual behavior

Agent disconnection is happening very frequently.

Repro steps

N/A

[erik-bershel]

Hey @rohanp-91!
Your problem is either a duplicate of an existing one #7754 or is related to the consuming of all available resources. Most likely the second, judging by the text of the error. In this case, I can't recommend anything other than process optimisation and limiting the resource consumption of your processes. Unfortunately, we do not deal with such problems - although we understand how great your frustration is, we cannot increase the amount of available hardware resources. There were a thread related to the similar issues on other builds: #7188.