Thanks Uwe for seeing through this massive improvement to our C++ build system! This should help package managers build the project in precisely the way that's needed to make the Arrow C++ libraries available in more places.
A couple notes: * I personally keep my C++ toolchain and conda environments separate; if you also do this you can see how to configure to pick up your toolchain directory even though $CONDA_PREFIX is set: -DARROW_DEPENDENCY_SOURCE=SYSTEM -DARROW_PACKAGE_PREFIX=$CPP_TOOLCHAIN (see my environment configuration at https://github.com/wesm/dev-toolchain/blob/master/toolchain/arrow-toolchain.sh#L142) * One of the cool things about this is that you can control dependency resolution in a more granular way: -D$PACKAGE_SOURCE=BUNDLED -> Use ExternalProject to build -D$PACKAGE_SOURCE=SYSTEM -> Look in system paths for the package -D$PACKAGE_ROOT=$SOMEWHERE -> Look in $SOMEWHERE for the package So in the event that you need to override one of your system dependencies, you have the flexibility to do that. - Wes Thanks Wes On Fri, Mar 15, 2019 at 9:48 AM Uwe L. Korn <m...@uwekorn.com> wrote: > > Hello fellow Arrow Devs, > > we have merged the CMake refactor yesterday > https://github.com/apache/arrow/pull/3688 and this means that the build > system behaves a bit different. The main differences are: > > * If you're in a conda environment, we automatically detect this using the > environment variable $CONDA_PREFIX and expect that all dependencies (except > jemalloc and ORC) are installed via conda. > * Otherwise, we will look in the standard system paths for a dependency. If > it isn't found, we use CMake's ExternalProject mechanism to build it. > * The *_HOME variables are not longer use and are replaced by *_ROOT > variables to use CMake's standard detection features. Be aware that > dependencies are no longer written in all caps but their preferred casing as > seen in > https://github.com/apache/arrow/blob/0d302125abb4b514dba210f496c574a77ce4cd1d/cpp/cmake_modules/ThirdpartyToolchain.cmake#L41-L59 > * You can manually select the way we detect dependencies via > ARROW_DEPENDENCY_SOURCE > https://github.com/apache/arrow/blob/0d302125abb4b514dba210f496c574a77ce4cd1d/cpp/CMakeLists.txt#L189-L207 > The hope is that you as a developer should not normally need to change this > and as packager for distributions, you can use > `ARROW_DEPENDENCY_SOURCE=SYSTEM` to ensure that ExternalProject is not used > but only packages from the package manager. If your system is in a > non-default prefix, you can indicate this by setting ARROW_PACKAGE_PREFIX. > > Also, please clear your existing CMake directories and do a fresh built to > avoid any problems. As well when you're using conda packages, please update > them all using `conda update --all` as I have errors in the packaging > directly on conda-forge instead of doing workarounds in our CMake code. A > helpful information is here that conda-forge now provides a `compilers` > package that provides the whole build toolchain. > > Uwe
