[R] Support gnu libtool?

[jonkeane]

Describe the enhancement requested

We currently are failing on most macOS systems on CRAN. It appears that the CRAN runners have GNU libtool installed instead of the xcode version so we need to avoid '-no_warning_for_no_symbols'.

Note: the mac builder CRAN machine does not exhibit this behavior. So we will need to test this on our own.

[ 19%] Bundling /var/folders/k4/0jwzxmln0nb8y6rkzprptb640000gq/T/Rtmp4C6n7h/file63de2525b3fd/release/libarrow_bundled_dependencies.a
Scanning dependencies of target arrow_dependencies
Usage: /Library/Frameworks/R.framework/Resources/bin/libtool [OPTION]... [MODE-ARG]...
Try 'libtool --help' for more information.
libtool:   error: unrecognised option: '-no_warning_for_no_symbols'
make[2]: *** [src/arrow/CMakeFiles/arrow_bundled_dependencies_merge] Error 1
make[1]: *** [src/arrow/CMakeFiles/arrow_bundled_dependencies_merge.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 19%] Built target arrow_dependencies
make: *** [all] Error 2
**** Complete build log may still be present at /var/folders/k4/0jwzxmln0nb8y6rkzprptb640000gq/T//Rtmp4C6n7h/file63de4e4bba18.log 
*** Failed to find Arrow C++ libraries.
------------------------- NOTE ---------------------------
There was an issue preparing the Arrow C++ libraries.
See https://arrow.apache.org/docs/r/articles/install.html
----------------------------------------------------------

Component(s)

R

[kou]

Does this work?

diff --git a/cpp/cmake_modules/BuildUtils.cmake b/cpp/cmake_modules/BuildUtils.cmake
index 083ac2fe9a..c8da0d43d4 100644
--- a/cpp/cmake_modules/BuildUtils.cmake
+++ b/cpp/cmake_modules/BuildUtils.cmake
@@ -96,10 +96,7 @@ function(arrow_create_merged_static_lib output_target)
     list(APPEND all_library_paths $<TARGET_FILE:${lib}>)
   endforeach()
 
-  if(APPLE)
-    set(BUNDLE_COMMAND "libtool" "-no_warning_for_no_symbols" "-static" "-o"
-                       ${output_lib_path} ${all_library_paths})
-  elseif(CMAKE_CXX_COMPILER_ID MATCHES "^(Clang|GNU|Intel|IntelLLVM)$")
+  if(CMAKE_CXX_COMPILER_ID MATCHES "^(Clang|GNU|Intel|IntelLLVM)$")
     set(ar_script_path ${CMAKE_BINARY_DIR}/${ARG_NAME}.ar)
 
     file(WRITE ${ar_script_path}.in "CREATE ${output_lib_path}\n")
@@ -120,7 +117,9 @@ function(arrow_create_merged_static_lib output_target)
     endif()
 
     set(BUNDLE_COMMAND ${ar_tool} -M < ${ar_script_path})
-
+  elseif(APPLE)
+    set(BUNDLE_COMMAND "libtool" "-no_warning_for_no_symbols" "-static" "-o"
+                       ${output_lib_path} ${all_library_paths})
   elseif(MSVC)
     if(CMAKE_LIBTOOL)
       set(BUNDLE_TOOL ${CMAKE_LIBTOOL})

[kou]

BTW, can we see all build log?

[jonkeane]

Oh, yes, I can try that as well — when I tried that locally I saw errors with arguments being passed to ar / ranlib (where the apple version doesn't accept the same as the non-apple version). But I'll try with the reversed order – it's also possible I messed something else up when I was doing that.

[jonkeane]

Ok I've tried swapping those if/else blocks (locally) and on macOS we still hit the APPLE only block (the compiler ID is "AppleClang", so that's not totally surprising)

[jonkeane]

I also tried adding AppleClang to the list of enumerated compiler ids and here is that AR command mismatch I got:

[ 11%] Performing install step for 'lz4_ep'
-- lz4_ep install command succeeded.  See also /var/folders/b2/8w36qffn5rgbkzd2bf8tcgbh0000gn/T/RtmpUYYlct/file361952aacb3a/lz4_ep-prefix/src/lz4_ep-stamp/lz4_ep-install-*.log
[ 12%] Completed 'lz4_ep'
[ 12%] Built target lz4_ep
-- re2_ep build command succeeded.  See also /var/folders/b2/8w36qffn5rgbkzd2bf8tcgbh0000gn/T/RtmpUYYlct/file361952aacb3a/re2_ep-prefix/src/re2_ep-stamp/re2_ep-build-*.log
[ 12%] Performing install step for 're2_ep'
-- re2_ep install command succeeded.  See also /var/folders/b2/8w36qffn5rgbkzd2bf8tcgbh0000gn/T/RtmpUYYlct/file361952aacb3a/re2_ep-prefix/src/re2_ep-stamp/re2_ep-install-*.log
[ 12%] Completed 're2_ep'
[ 12%] Built target re2_ep
[ 12%] Built target toolchain
[ 13%] Bundling /var/folders/b2/8w36qffn5rgbkzd2bf8tcgbh0000gn/T/RtmpUYYlct/file361952aacb3a/release/libarrow_bundled_dependencies.a
usage:  ar -d [-TLsv] archive file ...
	ar -m [-TLsv] archive file ...
	ar -m [-abiTLsv] position archive file ...
	ar -p [-TLsv] archive [file ...]
	ar -q [-cTLsv] archive file ...
	ar -r [-cuTLsv] archive file ...
	ar -r [-abciuTLsv] position archive file ...
	ar -t [-TLsv] archive [file ...]
	ar -x [-ouTLsv] archive [file ...]
make[2]: *** [src/arrow/CMakeFiles/arrow_bundled_dependencies_merge] Error 1
make[1]: *** [src/arrow/CMakeFiles/arrow_bundled_dependencies_merge.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 13%] Built target arrow_dependencies

[kou]

Ah, sorry. I should have explained the idea of the suggested patch.

In general, Apache Arrow C++ build uses Apple's C/C++ toolchain. So CMAKE_CXX_COMPILER_ID is AppleClang. In this case, if(CMAKE_CXX_COMPILER_ID MATCHES "^(Clang|GNU|Intel|IntelLLVM)$") isn't matched. if(APPLE) (Apple's libtool) is used.

If macOS + CRAN build does NOT use Apple's C/C++ toolchain (uses C/C++ toolchain provided by R.framework that also includes GNU libtool, gcc, g++, GNU ar and so on), CMAKE_CXX_COMPILER_ID may be GNU not AppleClang. In this case, if(CMAKE_CXX_COMPILER_ID MATCHES "^(Clang|GNU|Intel|IntelLLVM)$") (GNU ar) is used.

[jonkeane]

Ah, right got it — in this case I don't think that CRAN is using a full toolchain in R.framework.

The most direct evidence is in the CRAN logs:

-- The C compiler identification is AppleClang 14.0.0.14000029
-- The CXX compiler identification is AppleClang 14.0.0.14000029

The bin dir of that framework also doesn't have too much there (though does have libtool!):

ls /Library/Frameworks/R.framework/Resources/bin 
BATCH		Rcmd		SHLIB		exec		qpdf
COMPILE		Rd2pdf		Stangle		fc-cache	rtags
INSTALL		Rdconv		Sweave		javareconf
LINK		Rdiff		build		libtool
R		Rprof		check		mkinstalldirs
REMOVE		Rscript		config		pager

[kou]

Thanks!

The bin dir of that framework also doesn't have too much there (though does have libtool!):

Oh, why... OK. We need to choose your approach (#40259).

[jonkeane]

Oh, why... OK.

😂

[jonkeane]

Issue resolved by pull request 40259
#40259