Hi,
> * On the latter, I see that we're using snappy and lz4 from the system
> (cmake finds them in the Arrow C++ build) but when the arrow.so is built
> for the R package, there is -llz4 in the libs but no -lsnappy. The line
> where the link libs are assembled from the C++ build via pkg-config is
> here: https://github.com/apache/arrow/blob/master/r/configure#L185, maybe
> the --static is relevant? Kou, do you know?
We can't use --libs-only-l and --libs-only-other at the same
time. If we use both of them at the same time,
--libs-only-other is ignored. The following outputs were
executed on CentOS 7 with the official arrow-devel RPM
package:
$ pkg-config --libs-only-l --static arrow
-larrow -larrow_bundled_dependencies -lbrotlidec -lbrotlienc -lz -llz4 -lzstd
-lbrotlicommon
$ pkg-config --libs-only-other --static arrow
/usr/lib64/libsnappy.so /usr/lib64/libbz2.so -pthread
$ pkg-config --libs-only-l --libs-only-other --static arrow
-larrow -larrow_bundled_dependencies -lbrotlidec -lbrotlienc -lz -llz4 -lzstd
-lbrotlicommon
How about changing the line in r/configure to the following?
PKG_LIBS="$PKG_LIBS `PKG_CONFIG_PATH=${LIB_DIR}/pkgconfig pkg-config
--libs-only-l --static --silence-errors ${PKG_CONFIG_NAME}`"
PKG_LIBS="$PKG_LIBS `PKG_CONFIG_PATH=${LIB_DIR}/pkgconfig pkg-config
--libs-only-other --static --silence-errors ${PKG_CONFIG_NAME}`"
Thanks,
--
kou
In <caocv4hjd4wz3ubcxz6krrpretjkjuyo7ktuv7ydngnzumyn...@mail.gmail.com>
"Re: [R] Install arrow package: arrow.so undefined symbol" on Tue, 3 May 2022
14:05:09 -0400,
Neal Richardson <neal.p.richard...@gmail.com> wrote:
> Hmm, I see a couple of things:
>
> * There are two errors at the end, one about pthread_cancel and one about
> undefined symbols for snappy. I can't tell if the pthread issue is fatal.
> * On the latter, I see that we're using snappy and lz4 from the system
> (cmake finds them in the Arrow C++ build) but when the arrow.so is built
> for the R package, there is -llz4 in the libs but no -lsnappy. The line
> where the link libs are assembled from the C++ build via pkg-config is
> here: https://github.com/apache/arrow/blob/master/r/configure#L185, maybe
> the --static is relevant? Kou, do you know?
> * In terms of working around this, you have a few options, which are
> described in more detail here:
> https://arrow.apache.org/docs/r/articles/install.html
> * Set NOT_CRAN=true and get a fully-functioning C++ binary downloaded
> * Install a binary from RStudio Package Manager, if that's an option
> * conda may also be an option
> * If none of those work and you're building from source, you can set
> EXTRA_CMAKE_FLAGS="-DSNAPPY_SOURCE=BUNDLED" to skip the system version of
> snappy and build it in the Arrow build
> * You may also want to use a newer devtoolset since you're building
> with gcc 4.8, and some features aren't supported with that compiler.
>
> I've trimmed the installation logs to the relevant bits in the quoted reply
> that follows, in case that's useful.
>
> Neal
>
>
> On Tue, May 3, 2022 at 1:08 PM Rares Vernica <rvern...@gmail.com> wrote:
>
>> Hi Dragos,
>>
>> It still fails after setting the environment variable. Here is the log.
>>
>> Cheers,
>> Rares
>>
>> ...
>> > install.packages("arrow")
>> Installing package into '/usr/lib64/R/library'
>> ...
>> *** Building libarrow from source
>> For a faster, more complete installation, set the environment variable
>> NOT_CRAN=true before installing
>> See install vignette for details:
>> https://cran.r-project.org/web/packages/arrow/vignettes/install.html
>> *** Building with MAKEFLAGS= -j2
>> **** arrow with SOURCE_DIR='tools/cpp'
>> BUILD_DIR='/tmp/RtmpObd2i3/file5310bef585' DEST_DIR='libarrow/arrow-7.0.0'
>> CMAKE='/usr/bin/cmake3' EXTRA_CMAKE_FLAGS='' CC='gcc -m64 -std=gnu99'
>> CXX='g++ -m64 -std=gnu++11' LDFLAGS='-Wl,-z,relro' ARROW_S3='OFF'
>> ARROW_MIMALLOC='OFF'
>> ...
>> + /usr/bin/cmake3 -DARROW_BOOST_USE_SHARED=OFF -DARROW_BUILD_TESTS=OFF
>> -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON -DARROW_COMPUTE=ON
>> -DARROW_CSV=ON -DARROW_DATASET=ON -DARROW_DEPENDENCY_SOURCE=AUTO
>> -DAWSSDK_SOURCE= -DARROW_FILESYSTEM=ON -DARROW_JEMALLOC=OFF
>> -DARROW_MIMALLOC=OFF -DARROW_JSON=ON -DARROW_PARQUET=ON -DARROW_S3=OFF
>> -DARROW_WITH_BROTLI=OFF -DARROW_WITH_BZ2=OFF -DARROW_WITH_LZ4=ON
>> -DARROW_WITH_RE2=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_UTF8PROC=ON
>> -DARROW_WITH_ZLIB=OFF -DARROW_WITH_ZSTD=OFF
>> -DARROW_VERBOSE_THIRDPARTY_BUILD=OFF -DCMAKE_BUILD_TYPE=Release
>> -DCMAKE_INSTALL_LIBDIR=lib
>>
>> -DCMAKE_INSTALL_PREFIX=/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0
>> -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON
>> -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_UNITY_BUILD=OFF -G
>> 'Unix Makefiles' /tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/tools/cpp
>> -- Building using CMake version: 3.17.5
>> -- The C compiler identification is GNU 4.8.5
>> -- The CXX compiler identification is GNU 4.8.5
>> ...
>> -- Looking for pthread.h
>> -- Looking for pthread.h - found
>> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
>> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
>> -- Check if compiler accepts -pthread
>> -- Check if compiler accepts -pthread - yes
>> -- Found Threads: TRUE
>> ...
>> -- Found Snappy: /usr/lib64/libsnappy.so
>> -- Providing CMake module for Snappy
>> ...
>> -- Checking for module 'liblz4'
>> -- Found liblz4, version 1.8.3
>> -- Found Lz4: /usr/lib64/liblz4.so
>> -- Providing CMake module for Lz4
>> ...
>> -- Install configuration: "RELEASE"
>> -- Installing:
>>
>> /tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib/cmake/arrow/FindSnappy.cmake
>> -- Installing:
>>
>> /tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib/cmake/arrow/FindLz4.cmake
>> ...
>>
>> PKG_CFLAGS=-I/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/include
>> -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET
>> -DARROW_R_WITH_JSON
>>
>> PKG_LIBS=-L/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib
>> -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies
>> -llz4 -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet
>> ...
>> g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro -o arrow.so
>> RTasks.o altrep.o array.o array_to_vector.o arraydata.o arrowExports.o
>> bridge.o buffer.o chunkedarray.o compression.o compute-exec.o compute.o
>> config.o csv.o dataset.o datatype.o expression.o feather.o field.o
>> filesystem.o imports.o io.o json.o memorypool.o message.o parquet.o
>> r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o scalar.o
>> schema.o symbols.o table.o threadpool.o type_infer.o
>> -L/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib
>> -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies
>> -llz4 -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet
>> -L/usr/lib64/R/lib -lR
>> installing to /usr/lib64/R/library/00LOCK-arrow/00new/arrow/libs
>> ** R
>> ...
>> ** testing if installed package can be loaded from temporary location
>> libgcc_s.so.1 must be installed for pthread_cancel to work
>> sh: line 1: 3916 Aborted (core dumped)
>> '/usr/lib64/R/bin/R' --no-save --slave 2>&1 <
>> '/tmp/Rtmp8rFBf9/file2f3cba608e'
>> Error: package or namespace load failed for 'arrow' in dyn.load(file,
>> DLLpath = DLLpath, ...):
>> unable to load shared object
>> '/usr/lib64/R/library/00LOCK-arrow/00new/arrow/libs/arrow.so':
>> /usr/lib64/R/library/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined
>> symbol: _ZN6snappy11RawCompressEPKcmPcPm
>> Error: loading failed
>> Execution halted
>> ERROR: loading failed
>> * removing '/usr/lib64/R/library/arrow'
>>
