[BUG] Lost distutils.ccompiler.compiler_class in latest release

[effigies]

setuptools version

setuptools==76.0.0

Python version

Python 3.9

OS

Fedora 41

Additional environment information

No response

Description

As of this morning, my tests have begun failing with:

   /home/runner/work/pybids/pybids/.tox/py39-min/lib/python3.9/site-packages/numpy/distutils/__init__.py:24: in <module>
      from . import ccompiler
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
      import os
      import re
      import sys
      import shlex
      import time
      import subprocess
      from copy import copy
      from distutils import ccompiler
  >   from distutils.ccompiler import (
          compiler_class, gen_lib_options, get_default_compiler, new_compiler,
          CCompiler
      )
  E   ImportError: cannot import name 'compiler_class' from 'distutils.ccompiler' (/home/runner/work/pybids/pybids/.tox/py39-min/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py)

Note that this import is removed from the latest numpy; this only occurs because I test my declared minimum versions using a lowest-direct resolution. This means that if I say I support numpy >=1.22, then I get numpy 1.22, but if setuptools is an indirect dependency, it is allowed to be the latest. (It's infeasible to use a recursive lowest for long-standing packages, since that will currently find dependencies with no minimum stated version or with sdists that predate requires-python and attempt to install python 2 packages.)

If necessary, I can pin an older version of setuptools in my tox.ini, but if it's possible to re-add this import, it would save doing this across many repositories.

I initially posted this as pypa/distutils#336, not realizing I wasn't posting on setuptools. Feel free to close if that issue is enough.

Expected behavior

I would prefer not to break backwards compatibility with distutils until Python 3.11 is EOL. Many projects did not fully fix this until 3.12.

How to Reproduce

uv venv -p 3.9 /tmp/py39
source /tmp/py39/bin/activate
pip install setuptools numpy==1.22
python -c 'import numpy.testing'

Output

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/py39/lib/python3.9/site-packages/numpy/testing/__init__.py", line 12, in <module>
    from ._private import extbuild, decorators as dec
  File "/tmp/py39/lib/python3.9/site-packages/numpy/testing/_private/extbuild.py", line 11, in <module>
    from numpy.distutils.ccompiler import new_compiler
  File "/tmp/py39/lib/python3.9/site-packages/numpy/distutils/__init__.py", line 24, in <module>
    from . import ccompiler
  File "/tmp/py39/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 9, in <module>
    from distutils.ccompiler import (
ImportError: cannot import name 'compiler_class' from 'distutils.ccompiler' (/tmp/py39/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py)

[maurya7026]

Any update when the PR will be merged and ready to use?

[atos-ghassen-bchir]

If you are looking for a working version, the 75.8.2 seems to be the latest version which does not include the issue.

[ManiacDC]

This isn't quite fixed, numpy also imports _default_compilers from ccompiler.py when using numba AOT compilation. I will open a new PR.

[abravalheri]

_default_compilers is a private symbol, so ideally it would be best if numpy/numba do not rely on it...

[ManiacDC]

@abravalheri totally agree. This is in a really old version of numpy (1.23.5)/numba, but still supported by python 3.10. Newer versions may not have this issue.

[ManiacDC]

Actually, looks like they still reference this in main:
https://github.com/numpy/numpy/blob/main/numpy/distutils/ccompiler.py

[ManiacDC]

I've opened a numpy bug for them to fix current versions:
numpy/numpy#28551

[effigies]

The numpy folks have done their job here. numpy.distutils was deprecated in 1.23, so the fix should be for numba to stop depending on it. If numba already has and you're just testing on an old version of numba, I think the fix is either to bump your minimum numba or cap setuptools in the short term.

[ManiacDC]

@effigies looks like they stopped relying on that in numba 0.57.0. I'm on 0.56.4. I'll see if I'm able to upgrade.

[ManiacDC]

However, they aren't removing that code until Python 3.12. I would think it would be best that distutils should try to refrain from breaking changes until then as well.

[ManiacDC]

I was able to update our code to numba 0.57.0 to workaround the issue. I'll leave this PR open and let the distutils team decide how they want to handle it. Thanks @effigies .

[jaraco]

The workaround for this issue should be out now. As I posted elsewhere, I've exposed these attributes temporarily to ease the transition, but I'm not committing to keeping them around for several years.