Need to augment wheel tag computation code to account for nogil builds

[dnicolodi]

Starting with Python 3.13 there will be a nogil variant of the interpreter that will need to be get an interpreter tag in the wheel tag string. See https://discuss.python.org/t/python-abis-and-pep-703/34018

[dnicolodi]

See also pypa/packaging#727 pypa/packaging#728 pypa/packaging#747

[rgommers]

gh-621 addresses part of this. As of today, it's not yet very easy to test with a free-threaded CPython build and make the test suite pass. The hiccups are:

• pip does not yet have support. Two issues: it incorrectly installs incompatible abi3 wheels, and it refuses to install wheels with the cp313t/cp313td ABI tag. The former doesn't affect meson-python, the latter was a little painful to work around (see below for how).
• cython does not yet have full support. The latest release (3.0.10) fails to build one of our test packages (the complex one), however the branch from Update module setup code for free-threaded CPython cython/cython#6137 does work.
• CPython needs to be built from source (./configure --with-pydebug --disable-gil)

In order to make pip install wheels with the t tag, the _fail_if_link_is_unsupported_wheel function in pip/_internal/resolution/resolvelib/factory.py needs to be patched - simply replacing raise UnsupportedWheel(msg) with return is enough.

Then the venv fixture in our test suite needs patching, because it installs a fresh copy of pip in the venv's it creates. This hacky patch does that (replace path to site-packages as needed):

diff --git a/tests/conftest.py b/tests/conftest.py
index 80303a14f..68df9e2ef 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -101,7 +101,14 @@ class VEnv(EnvBuilder):
 @pytest.fixture()
 def venv(tmp_path_factory):
     path = pathlib.Path(tmp_path_factory.mktemp('mesonpy-test-venv'))
-    return VEnv(path)
+    fixture = VEnv(path)
+
+    # Apply fix to `pip` installed into venv to avoid unknown tag issue
+    sitepkg_dir = '/home/rgommers/mambaforge/envs/nogil/lib/python3.13/site-packages'
+    sitepkg_venv = str(path) + '/lib/python3.13/site-packages'
+    _file = '/pip/_internal/resolution/resolvelib/factory.py'
+    subprocess.run(['cp', sitepkg_dir + _file, sitepkg_venv + _file])
+    return fixture
 
 
 def generate_package_fixture(package):

[dnicolodi]

gh-621 addresses part of this

@rgommers With #621 merged, what is left to do to support nogil Python?

[rgommers]

Everything works, I only left this open to remind myself to add a CI job once the setup-python action starts offering a free-threaded CPython.

Happy to close this though and keep track of that task elsewhere - it's not urgent, and we're unlikely to regress on full support.

[dnicolodi]

Do you have a sense for when we may see nogil builds available via actions/setup-python? The alpha releases of Python 3.13 are available, but no nogil builds.

[rgommers]

Tracked in actions/setup-python#771. No timeline, but I think the desire is to do that for the 3.13.0b1. So hopefully in May.

[dnicolodi]

Oh, you mentioned the actions/setup-python bug in #621. I noticed it when I read the PR but then I completely forgot about it, sorry. Our test matrix is becoming very large. I'm wondering if there is something that would be reasonable to drop.

[rgommers]

We should probably be able to drop some Meson versions? If we test oldest/newest releases plus master branch, that gets us pretty much what we need. Unless we have a specific reason to test every minor version. We can also combine Meson and Python versions, rather than having separate jobs for "check each Python version with latest meson" and then "check each meson version".

[henryiii]

You can use python3.13t in manylinux2014+ starting today.

[rgommers]

Awesome. I wasn't expecting that just yet, but it looks like it was enabled by including pip 24.1b1 in the image (see pypa/manylinux#1564).

[dnicolodi]

I'm giving it a try just now, see #627

[dnicolodi]

Tests for free-threading Python 3.13 are up and running. Closing this issue.