Runner scale set blocked after upgrade

[tyndalla]

Checks

• I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
• I am using charts that are officially provided

Controller Version

0.9.1

Deployment Method

Helm

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions).
• I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

1. Deploy ARC 0.9.1 via Helm
2. Deploy auto scaling runner set v0.9.1 via Helm
3. Upgrade via Helm to change maxRunners value
4. Uninstall auto scaling runner set via Helm

Describe the bug

After following the steps to upgrade the runner set, the runner is blocked. To try and resolve, the runner was uninstalled via Helm. The autoscalingrunnerset was not removed.

To try and resolve, the runner was uninstalled via Helm. The autoscalingrunnerset was not removed.

In the controller logs, this was the last message received:

2024-05-03T11:44:15Z INFO AutoscalingRunnerSet Waiting for dependent resources to be deleted {"autoscalingrunnerset": {"name":"aah-legacy-runner-set","namespace":"github"}}

In the repository, the auto scaling runner set is still listed.

Describe the expected behavior

The auto scaling runner set runner set should upgrade and be deleted cleanly.

Additional Context

githubConfigSecret:
  github_token: <REDACTED>

containerMode:
  type: "kubernetes"
  kubernetesModeWorkVolumeClaim:
    accessModes: ["ReadWriteOnce"]
    storageClassName: "microk8s-hostpath"
    resources:
      requests:
        storage: 8Gi

template:
  spec:
    containers:
    - name: runner
      image: ghcr.io/actions/actions-runner:latest
      command: ["/home/runner/run.sh"]
      env:
        - name: ACTIONS_RUNNER_REQUIRE_JOB_CONTAINER
          value: "false"
    imagePullSecrets:
    - name: ghcr-secret

minRunners: 1
maxRunners: 3

Controller Logs

https://gist.github.com/tyndalla/f99cd9458731dc213fc9be57f9d38b61

Runner Pod Logs

The runner scale set is not running due to the problem above.

[github-actions]

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.