I created a PR to apply this change: https://github.com/apache/arrow/pull/13151
On Tue, May 3, 2022 at 5:08 PM Sutou Kouhei <k...@clear-code.com> wrote: > Hi, > > > * On the latter, I see that we're using snappy and lz4 from the system > > (cmake finds them in the Arrow C++ build) but when the arrow.so is built > > for the R package, there is -llz4 in the libs but no -lsnappy. The line > > where the link libs are assembled from the C++ build via pkg-config is > > here: https://github.com/apache/arrow/blob/master/r/configure#L185, > maybe > > the --static is relevant? Kou, do you know? > > We can't use --libs-only-l and --libs-only-other at the same > time. If we use both of them at the same time, > --libs-only-other is ignored. The following outputs were > executed on CentOS 7 with the official arrow-devel RPM > package: > > $ pkg-config --libs-only-l --static arrow > -larrow -larrow_bundled_dependencies -lbrotlidec -lbrotlienc -lz -llz4 > -lzstd -lbrotlicommon > $ pkg-config --libs-only-other --static arrow > /usr/lib64/libsnappy.so /usr/lib64/libbz2.so -pthread > $ pkg-config --libs-only-l --libs-only-other --static arrow > -larrow -larrow_bundled_dependencies -lbrotlidec -lbrotlienc -lz -llz4 > -lzstd -lbrotlicommon > > How about changing the line in r/configure to the following? > > PKG_LIBS="$PKG_LIBS `PKG_CONFIG_PATH=${LIB_DIR}/pkgconfig pkg-config > --libs-only-l --static --silence-errors ${PKG_CONFIG_NAME}`" > PKG_LIBS="$PKG_LIBS `PKG_CONFIG_PATH=${LIB_DIR}/pkgconfig pkg-config > --libs-only-other --static --silence-errors ${PKG_CONFIG_NAME}`" > > > Thanks, > -- > kou > > In <caocv4hjd4wz3ubcxz6krrpretjkjuyo7ktuv7ydngnzumyn...@mail.gmail.com> > "Re: [R] Install arrow package: arrow.so undefined symbol" on Tue, 3 May > 2022 14:05:09 -0400, > Neal Richardson <neal.p.richard...@gmail.com> wrote: > > > Hmm, I see a couple of things: > > > > * There are two errors at the end, one about pthread_cancel and one about > > undefined symbols for snappy. I can't tell if the pthread issue is fatal. > > * On the latter, I see that we're using snappy and lz4 from the system > > (cmake finds them in the Arrow C++ build) but when the arrow.so is built > > for the R package, there is -llz4 in the libs but no -lsnappy. The line > > where the link libs are assembled from the C++ build via pkg-config is > > here: https://github.com/apache/arrow/blob/master/r/configure#L185, > maybe > > the --static is relevant? Kou, do you know? > > * In terms of working around this, you have a few options, which are > > described in more detail here: > > https://arrow.apache.org/docs/r/articles/install.html > > * Set NOT_CRAN=true and get a fully-functioning C++ binary downloaded > > * Install a binary from RStudio Package Manager, if that's an option > > * conda may also be an option > > * If none of those work and you're building from source, you can set > > EXTRA_CMAKE_FLAGS="-DSNAPPY_SOURCE=BUNDLED" to skip the system version of > > snappy and build it in the Arrow build > > * You may also want to use a newer devtoolset since you're building > > with gcc 4.8, and some features aren't supported with that compiler. > > > > I've trimmed the installation logs to the relevant bits in the quoted > reply > > that follows, in case that's useful. > > > > Neal > > > > > > On Tue, May 3, 2022 at 1:08 PM Rares Vernica <rvern...@gmail.com> wrote: > > > >> Hi Dragos, > >> > >> It still fails after setting the environment variable. Here is the log. > >> > >> Cheers, > >> Rares > >> > >> ... > >> > install.packages("arrow") > >> Installing package into '/usr/lib64/R/library' > >> ... > >> *** Building libarrow from source > >> For a faster, more complete installation, set the environment > variable > >> NOT_CRAN=true before installing > >> See install vignette for details: > >> > https://cran.r-project.org/web/packages/arrow/vignettes/install.html > >> *** Building with MAKEFLAGS= -j2 > >> **** arrow with SOURCE_DIR='tools/cpp' > >> BUILD_DIR='/tmp/RtmpObd2i3/file5310bef585' > DEST_DIR='libarrow/arrow-7.0.0' > >> CMAKE='/usr/bin/cmake3' EXTRA_CMAKE_FLAGS='' CC='gcc -m64 -std=gnu99' > >> CXX='g++ -m64 -std=gnu++11' LDFLAGS='-Wl,-z,relro' ARROW_S3='OFF' > >> ARROW_MIMALLOC='OFF' > >> ... > >> + /usr/bin/cmake3 -DARROW_BOOST_USE_SHARED=OFF -DARROW_BUILD_TESTS=OFF > >> -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON -DARROW_COMPUTE=ON > >> -DARROW_CSV=ON -DARROW_DATASET=ON -DARROW_DEPENDENCY_SOURCE=AUTO > >> -DAWSSDK_SOURCE= -DARROW_FILESYSTEM=ON -DARROW_JEMALLOC=OFF > >> -DARROW_MIMALLOC=OFF -DARROW_JSON=ON -DARROW_PARQUET=ON -DARROW_S3=OFF > >> -DARROW_WITH_BROTLI=OFF -DARROW_WITH_BZ2=OFF -DARROW_WITH_LZ4=ON > >> -DARROW_WITH_RE2=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_UTF8PROC=ON > >> -DARROW_WITH_ZLIB=OFF -DARROW_WITH_ZSTD=OFF > >> -DARROW_VERBOSE_THIRDPARTY_BUILD=OFF -DCMAKE_BUILD_TYPE=Release > >> -DCMAKE_INSTALL_LIBDIR=lib > >> > >> > -DCMAKE_INSTALL_PREFIX=/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0 > >> -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON > >> -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_UNITY_BUILD=OFF -G > >> 'Unix Makefiles' /tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/tools/cpp > >> -- Building using CMake version: 3.17.5 > >> -- The C compiler identification is GNU 4.8.5 > >> -- The CXX compiler identification is GNU 4.8.5 > >> ... > >> -- Looking for pthread.h > >> -- Looking for pthread.h - found > >> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD > >> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed > >> -- Check if compiler accepts -pthread > >> -- Check if compiler accepts -pthread - yes > >> -- Found Threads: TRUE > >> ... > >> -- Found Snappy: /usr/lib64/libsnappy.so > >> -- Providing CMake module for Snappy > >> ... > >> -- Checking for module 'liblz4' > >> -- Found liblz4, version 1.8.3 > >> -- Found Lz4: /usr/lib64/liblz4.so > >> -- Providing CMake module for Lz4 > >> ... > >> -- Install configuration: "RELEASE" > >> -- Installing: > >> > >> > /tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib/cmake/arrow/FindSnappy.cmake > >> -- Installing: > >> > >> > /tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib/cmake/arrow/FindLz4.cmake > >> ... > >> > >> > PKG_CFLAGS=-I/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/include > >> -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET > >> -DARROW_R_WITH_JSON > >> > >> > PKG_LIBS=-L/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib > >> -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies > >> -llz4 -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet > >> ... > >> g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro -o > arrow.so > >> RTasks.o altrep.o array.o array_to_vector.o arraydata.o arrowExports.o > >> bridge.o buffer.o chunkedarray.o compression.o compute-exec.o compute.o > >> config.o csv.o dataset.o datatype.o expression.o feather.o field.o > >> filesystem.o imports.o io.o json.o memorypool.o message.o parquet.o > >> r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o > scalar.o > >> schema.o symbols.o table.o threadpool.o type_infer.o > >> -L/tmp/Rtmp8rFBf9/R.INSTALL2f7e48ba40/arrow/libarrow/arrow-7.0.0/lib > >> -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies > >> -llz4 -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet > >> -L/usr/lib64/R/lib -lR > >> installing to /usr/lib64/R/library/00LOCK-arrow/00new/arrow/libs > >> ** R > >> ... > >> ** testing if installed package can be loaded from temporary location > >> libgcc_s.so.1 must be installed for pthread_cancel to work > >> sh: line 1: 3916 Aborted (core dumped) > >> '/usr/lib64/R/bin/R' --no-save --slave 2>&1 < > >> '/tmp/Rtmp8rFBf9/file2f3cba608e' > >> Error: package or namespace load failed for 'arrow' in dyn.load(file, > >> DLLpath = DLLpath, ...): > >> unable to load shared object > >> '/usr/lib64/R/library/00LOCK-arrow/00new/arrow/libs/arrow.so': > >> /usr/lib64/R/library/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined > >> symbol: _ZN6snappy11RawCompressEPKcmPcPm > >> Error: loading failed > >> Execution halted > >> ERROR: loading failed > >> * removing '/usr/lib64/R/library/arrow' > >> >
