Transient failures for Windows AMD64 builds when searching for pyproject.toml

[alexlancaster]

Description

I am getting transient failures on Windows AMD64 builds. It only seems to affect Windows builds, but the exact version of Python seems to change, the error is related to not finding the pyproject.toml file:

ipx run --python "C:\hostedtoolcache\windows\Python\3.12.1\x64\python.exe" --spec "D:\a\_actions\pypa\cibuildwheel\v2.16.4" cibuildwheel "." --output-dir '"wheelhouse"' --config-file '"{package}/pyproject.toml"' --only '"cp310-win_amd64"'
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    CIBW_TEST_COMMAND: pytest -v {package}/tests 
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Scripts\cibuildwheel.exe\__main__.py", line 7, in <module>
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Lib\site-packages\cibuildwheel\__main__.py", line 144, in main
    build_in_directory(args)
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Lib\site-packages\cibuildwheel\__main__.py", line 269, in build_in_directory
    options = compute_options(platform=platform, command_line_arguments=args, env=os.environ)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Lib\site-packages\cibuildwheel\options.py", line 749, in compute_options
    options = Options(platform=platform, command_line_arguments=command_line_arguments, env=env)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Lib\site-packages\cibuildwheel\options.py", line 401, in __init__
    self.reader = OptionsReader(
                  ^^^^^^^^^^^^^^
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Lib\site-packages\cibuildwheel\options.py", line 215, in __init__
    config_options, config_platform_options = self._load_file(config_file_path)
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files (x86)\pipx\.cache\f7cd3d446587c0e\Lib\site-packages\cibuildwheel\options.py", line 286, in _load_file
    with filename.open("rb") as f:
         ^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.12.1\x64\Lib\pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 22] Invalid argument: '"D:\\a\\pypop\\pypop\\pyproject.toml"'
creating virtual environment...
creating virtual environment...
determining package name from 'D:\\a\\_actions\\pypa\\cibuildwheel\\v2.16.4'...
installing cibuildwheel from spec 'D:\\a\\_actions\\pypa\\cibuildwheel\\v2.16.4'...

Once the first set of wheels fails, i can normally get them to regenerate on a second attempt. The exact versions of Python change from time to time, but it's always on Windows.

Build log

https://github.com/alexlancaster/pypop/actions/runs/7717817812/job/21037765716#step:4:23

CI config

https://github.com/alexlancaster/pypop/blob/last_zenodo_check/.github/workflows/build_wheels.yml

[alexlancaster]

These errors only started after I upgraded from 2.16.2 to 2.16.4

[alexlancaster]

Notice that my second attempt for the same job cp310-win_amd64 gets past the same error: https://github.com/alexlancaster/pypop/actions/runs/7717817812/job/21040035683

[henryiii]

This is fixed in #1741. GitHub is rolling out a new powershell, it doesn't matter what cibuildwheel version you have.

[henryiii]

(Notice how it's including the " in the filename, that's a powershell quoting change)

[alexlancaster]

(Notice how it's including the " in the filename, that's a powershell quoting change)

Ah - I figured it might be something github-wide, rather than cibuildwheel specific. that would explain why it happens on some runners and not others.

[alexlancaster]

OK, I'll close this after the next release of cibuildwheel is out and I can test it. (I see also #1740).

[henryiii]

Release is happening (basically) now. Also supports the new M1 runners released today. :)

[henryiii]

It’s out, let us know if it’s fixed! :)

[alexlancaster]

It’s out, let us know if it’s fixed! :)

Thanks - I forced a dependabot check to get the new action, and it's building the PR now: alexlancaster/pypop#189

[alexlancaster]

Builds seem to work cleanly now. Closing.