-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use scikit-build-core instead of scikit-build #95
Conversation
d5b625e
to
720ce5c
Compare
- clang-format_version.cmake -> clang-format_version.txt - contains `llvm_version.wheel_version`, e.g. `18.0.2.1` - build logic is now contained in pyproject.toml + CMakeLists.txt - pyproject.toml - version parsed from clang-format_version.txt (wheel_version is omitted if zero) - CMakeLists.txt - version is taken from pyproject.toml - clang-format-diff and git-clang-format are now installed with valid python module names - specify Release config as build argument of clang-format to ensure we get a release build with msvc - include Release sub-folder in clang-format binary location when building with msvc - set `LLVM_TARGETS_TO_BUILD` empty to reduce llvm compile time by only building default host target instead of the default all targets - remove code to call bundled python scripts, directly point to script entry points in pyproject.toml - remove setup.py and MANIFEST.in - disable fail-fast and use latest runners in CI - resolves #80
720ce5c
to
fa91c60
Compare
@dokempf the CI now seems to work (apart from the test-pypi upload but I guess that is not related to these changes?): https://github.com/ssciwr/clang-format-wheel/actions/runs/8361576686 |
The pypi issue was a bug in this pr, now fixed: https://github.com/ssciwr/clang-format-wheel/actions/runs/8371384624 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, I mostly have some questions.
.github/workflows/release.yml
Outdated
@@ -29,30 +29,31 @@ jobs: | |||
runs-on: ${{ matrix.os }} | |||
|
|||
strategy: | |||
fail-fast: false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the reason for not failing fast? If the fast builds fail, we can save quite a bit of resources by stopping the slow ones.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was just for debugging convenience when different builds were failing for different reasons so I wanted to see them all (try to) finish - I'll revert this
@@ -18,10 +18,14 @@ ExternalProject_add(build-clang-format | |||
USES_TERMINAL_DOWNLOAD 1 | |||
USES_TERMINAL_CONFIGURE 1 | |||
USES_TERMINAL_BUILD 1 | |||
CMAKE_ARGS -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DLLVM_ENABLE_ZLIB=OFF -DLLVM_ENABLE_ZSTD=OFF -DLLVM_ENABLE_PROJECTS=clang | |||
BUILD_COMMAND ${CMAKE_COMMAND} --build . --target clang-format | |||
CMAKE_ARGS -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DLLVM_ENABLE_ZLIB=OFF -DLLVM_ENABLE_ZSTD=OFF -DLLVM_ENABLE_PROJECTS=clang -DLLVM_TARGETS_TO_BUILD= |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you give some insight into the additional -DLLVM_TARGETS_TO_BUILD=
? What does it do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is list of CPU targets the clang we are building should be able to generate binaries for, e.g. x86, arm, etc
By default this is all supported targets, setting it to empty means it will only use the host architecture, so there are (slightly) fewer files to compile.
if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC") | ||
set(config-subfolder "Release") | ||
endif() | ||
set(clang-format-executable ${CMAKE_BINARY_DIR}/llvm/${config-subfolder}/bin/clang-format${CMAKE_EXECUTABLE_SUFFIX}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have any idea why this worked before? Is it a scikit-build
vs. -core
difference in default CMake arguments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking into this again I think the difference is that scikit-build was using Ninja on windows, and scikit-build-core is using MSVC. I'll try to fix this as ninja is faster and then we don't need this config-subfolder
cruft.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This ended up being a bit of a rabbit hole:
- scikit-build-core uses the default compiler in your environment (unlike scikit-build which has its own logic for choosing the compiler and generator and ignores your defaults/environment)
- it also does
cmake --help
and parses this to find the default generator - on windows by default this works fine with visual studio generator + mvsc compiler
- if you set ninja as the generator via a cmake arg this doesn't alter the
cmake --help
default generator logic and you get an error due to inconsistent setting of the generator - if you set ninja as the generator via an env var this also modifies the
cmake --help
default generator and it works, but now the default compiler (at least on GHA windows CI) changes to mingw gcc for whatever reason - so you also need to set up MSVC as the default compiler on windows CI
tldr: ninja is now used on windows ci, but from source builds using the visual studio generator should also work
${CMAKE_BINARY_DIR}/llvm-project/clang/tools/clang-format/git-clang-format | ||
DESTINATION clang_format/data/bin | ||
RENAME git_clang_format.py | ||
DESTINATION clang_format |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting approach, but I think it solves all the issues we had in #36.
… config-subfolder logic
…enerator config-subfolder logic" This reverts commit dd88159.
…rms in CI - note: setting this in the scikit-build-core cmake args is not sufficient - scikit-build-core uses the default cmake generator on windows, which is modified by the CMAKE_GENERATOR env var - fix config-subfolder logic to only apply when Visual Studio is used as generator - ensure MSVC is the active default compiler on windows CI - scikit-build-core uses whatever the current default is (while scikit-build always uses MSVC)
llvm_version.wheel_version
, e.g.18.0.2.1
LLVM_TARGETS_TO_BUILD
empty to reduce llvm compile time by only building default host target instead of the default all targets