Add new hosted MacOS runners to supported platforms

[chkimes]

keyword: darwin-aarch64

[MSP-Greg]

'new hosted MacOS runners'

When are the new platforms expected?

[eregon]

Are those runner images already available? We need them to run https://github.com/ruby/ruby-builder and build the binaries for all versions on it, before adding support here.

FWIW, I found actions/runner-images#7396

[chkimes]

I don't have an official date to give, however we will release the new images once there is enough capacity available and the images have been fully tested. This action is one of the issues we discovered during internal testing.

[eregon]

That's good news, I've heard lots of people are interested to have darwin-aarch64 as a GH-hosted runners.

This action is one of the issues we discovered during internal testing.

I think you will need to skip it during internal testing then because we can only add support in this action once the Rubies are built on these images and it's tested in CI, and that needs the image to be public somehow (but it can be beta or use a different name, etc). Also if those new images use rubies built from https://github.com/ruby/ruby-builder then again we need a public image to build those rubies.

[dfed]

I put up #501 before seeing this one. If this PR supersedes mine, we'll likely want to update the README.md and .github/workflows/test.yml here.

Note that per #500 macOS 13 is now supported by Github Actions

[eregon]

We merged macOS 13 amd64 support (that was trivial).
For macOS 13 on aarch64, we need GitHub to publish that image first and build all Ruby versions there.

[dentarg]

They are here now: https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/ (actions/runner-images#8439) but they aren't free so not sure (how) this action can support them?

[henninghall]

They are here now: https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/ (actions/runner-images#8439) but they aren't free so not sure (how) this action can support them?

Great news! 🎉 Ended up here after receiving following error message when trying the new runner. @eregon let us know if we can help out in any way to get this merged 🚀

Error: The current runner (macos-13-arm64) was detected as self-hosted because the platform does not match a GitHub-hosted runner image (or that image is deprecated and no longer supported).

[dentarg]

@henninghall You are not using the correct runner key: https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/#how-to-use-the-runner

Also, merging this will not solve the issue fully, this action needs builds of Ruby built by ruby-builder repo (see above), but as far as I know, these new runners are not available to https://github.com/ruby ("Larger runners are only available for organizations and enterprises using the GitHub Team or GitHub Enterprise Cloud plans.") so no builds for arm64 can be produced.

[eregon]

Yes, we need free macos arm64 runners for these 5 repos, otherwise it's not possible to support macos arm64 in this action.

• setup-ruby
• ruby-builder
• ruby-dev-builder,
• jruby-dev-builder,
• truffleruby-dev-builder

@chkimes @Steve-Glass Any chance GitHub could make some of these macos arm64 runners available to the ruby organization for that purpose? Or maybe there is a plan to make non-large macos arm64 runners freely available at some point?

[henninghall]

Alright I see, thanks for the clarification!

@henninghall You are not using the correct runner key: https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/#how-to-use-the-runner

Actually using macos-latest-xlarge, not sure why the error message says macos-13-arm64.

[Steve-Glass]

Yes, we need free macos arm64 runners for these 5 repos, otherwise it's not possible to support macos arm64 in this action.

• setup-ruby
• ruby-builder
• ruby-dev-builder,
• jruby-dev-builder,
• truffleruby-dev-builder

@chkimes @Steve-Glass Any chance GitHub could make some of these macos arm64 runners available to the ruby organization for that purpose? Or maybe there is a plan to make non-large macos arm64 runners freely available at some point?

👋 @eregon Thank you for reaching out to us! We've already escalated your request and will collaborate with you the path moving forward. I expect being able to provide an update on next steps by tomorrow afternoon (EST). Please let us know if you have any questions in the interim!

[Steve-Glass]

👋 @eregon We have approval to provide setup-ruby with M1 access. The next step for our team is to determine how we can make this happen from a billing/engineering perspective. I will follow up with you before the EoW with additional details.

[magnussp]

👋 @eregon We have approval to provide setup-ruby with M1 access. The next step for our team is to determine how we can make this happen from a billing/engineering perspective. I will follow up with you before the EoW with additional details.

@Steve-Glass any update for the community on this? We would love this solved since this would mean that our org could use the new arm64 large runners with fastlane. Thx for supporting the community!

[Steve-Glass]

any update for the community on this? We would love this solved since this would mean that our org could use the new arm64 large runners with fastlane. Thx for supporting the community!

Are you referring to when M1s will be available for free to all public repositories? Or when access will be provided to setup-ruby?

[MSP-Greg]

As mentioned above, the main issue with setup-ruby is having pre-built Ruby M1 builds available.

You should be able to run setup-ruby if you use one of the pre-installed Rubies. I believe macos-13 currently has 3.0.6 and 3.1.4 available.

[duncaninsiris]

@MSP-Greg
3.1.4 doesn't appear to work (and I don't have time to try 3.0.4) on macos-latest-xlarge:

Is there a list of compiled m1 compatible builds anywhere?

[eregon]

@duncaninsiris Could you run ls /Users/runner/hostedtoolcache/Ruby (and maybe ls /Users/runner/hostedtoolcache/ruby to be extra sure)? That should list which rubies are pre-installed.
It might be none because Ruby builds in the image might come from https://github.com/ruby/ruby-builder as well IIRC.

Or when access will be provided to setup-ruby?

@Steve-Glass Do you have any ETA for that? (thanks for the email last week)

[MSP-Greg]

Is there a list of compiled m1 compatible builds anywhere?

That's part of what's discussed here. The GitHub ruby organization does not have access to M1 runners. The Ruby builds that are pre-installed on the Ubuntu and Windows runner images are the same versions used by setup-ruby.

I believe the code to install macOS builds is at

https://github.com/actions/runner-images/blob/main/images/macos/provision/core/ruby.sh.

Like the Ubuntu images, it also downloads builds from ruby/ruby-builder. Since there are none...

[magnussp]

any update for the community on this? We would love this solved since this would mean that our org could use the new arm64 large runners with fastlane. Thx for supporting the community!

Are you referring to when M1s will be available for free to all public repositories? Or when access will be provided to setup-ruby?

When access will be provided to setup-ruby 🙂

[Steve-Glass]

@Steve-Glass Do you have any ETA for that? (thanks for the email last week)

No problem! Our team has started working on the solution we discussed via email. I don't expected it to take long and access should be provided by EoW. If we discover an issue during testing, then we'll circle back via email and discuss next steps.

[duncaninsiris]

@duncaninsiris Could you run ls /Users/runner/hostedtoolcache/Ruby (and maybe ls /Users/runner/hostedtoolcache/ruby to be extra sure)? That should list which rubies are pre-installed. It might be none because Ruby builds in the image might come from https://github.com/ruby/ruby-builder as well IIRC.

Thanks for the information @MSP-Greg and @eregon I did confirm that nothing is available. It helps me to understand a little more so thank you.

[Steve-Glass]

@eregon - Email with instructions has been sent and setup-ruby should have access now. If you run into issues, please don't hesitate to follow up on the email thread with @chkimes and I. Thanks for your patience!

[MSP-Greg]

I just want to clarify to everyone that the main issue is not setup-ruby. setup-ruby downloads/installs/configures Ruby builds if they are not present on the runner (the pre-installed Ruby builds). Importantly, setup-ruby does not create ruby build packages.

So, there are several repos that actually create the builds, as @eregon listed in the above message. All of these repos need access so that they can create the build packages that setup-ruby installs.

This also creates a cascade issue, in that all the code repos (like ruby/ruby) would certainly benefit from access, as they contain the code base that is used for the daily head/master/main builds...

[chkimes]

@MSP-Greg access was granted to the entire ruby org, not just this repo.

[eregon]

I'm working on this.
Initial testing shows that:

• 2.6, 2.7, 3.0, 3.2, 3.3, truffleruby, truffleruby+graalvm work.
• Anything 2.5 or older fails to build on macos-arm64 Ventura as can be seen at https://github.com/ruby/ruby-builder/actions/workflows/build.yml Those are old, so that's simply how it is. (2.5, 2.6.6)
• Weirdly 3.1.x fails to build too: https://github.com/ruby/ruby-builder/actions/runs/6485390387/job/17611360976. Unsure what can be done about it.
• jruby fails to build too: https://github.com/ruby/ruby-builder/actions/runs/6485449378/job/17611539289 because there is no Java available currently in https://github.com/actions/runner-images/blob/main/images/macos/macos-13-arm64-Readme.md. There is https://github.com/actions/setup-java but it wouldn't help for users of setup-ruby. So that will need to wait Java is part of that image.
• *-head will be done later.

[eregon]

This is released now as https://github.com/ruby/setup-ruby/releases/tag/v1.156.0 (and v1 is bumped).
*-head is done, except for truffleruby-head which is currently running.

• The same as above applies.

[eregon]

Re the 3.1.4 failure I found ruby/rbs#877 and rbenv/ruby-build#1961 (comment)
This seems the actual error:

dyld[47952]: terminating because inserted dylib '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' could not be loaded: tried: '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (no such file), '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')), './libruby.3.1.dylib' (no such file), '/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need ''))

[eregon]

I managed to build Ruby 3.1.x on macos-arm64 by using --disable-shared as mentioned in rbenv/ruby-build#1961 (comment): https://github.com/ruby/ruby-builder/actions/runs/6494667764/job/17638054965
It seems better to have those builds with --disable-shared than not at all. I think nobody ever successfully built Ruby 3.1.x on macos-arm64 with --enable-shared and that XCode version, that probably needs a fix in CRuby 3.1's branch: https://bugs.ruby-lang.org/issues/19920

[radeva]

@eregon You can try FlyCI's M1 and M2 runners. They are on average 2x faster and 2x cheaper than GitHub's AND we have a free tier for OSS projects (see below).

Install Instructions

1. Install FlyCI app and
2. Easily replace one line of code and start using FlyCI runners:

jobs:
 ci:
-    runs-on: macos-latest
+    runs-on: flyci-macos-large-latest-m1
   steps:
   - name: 👀 Checkout repo
     uses: actions/checkout@v4

500 mins/month Free for Public Repos

Since your repo is public, FlyCI offers 500 mins/month of free M1 runner usage with the flyci-macos-large-latest-m1 runner for public projects.

Don't hesitate to contact us in case the free tier doesn't suit your needs or you experience any issues with the runners. Our team is here to support you!

Best Regards,
Veselina Radeva
Product Manager at FlyCI