Hi, On Wed, 30 Jun 2021 at 21:10, Phil Beadling <p...@beadling.co.uk> wrote:
> When trying to use the parquet module in pyarrow - it's not finding the > internal module. This is true for latest 4.0.1 and also the previous > version (shown below). > > This is despite the fact that in the underlying apache-arrow package > parquet support is explicitly turned on as far as I can see: > https://github.com/guix-mirror/guix/blob/5ed105a8bb1a812975496dc3a091596355a0234c/gnu/packages/databases.scm#L3687 > > #:configure-flags > (list "-DARROW_PYTHON=ON" > "-DARROW_GLOG=ON" > ;; Parquet options > "-DARROW_PARQUET=ON" > "-DPARQUET_BUILD_EXECUTABLES=ON"..... The package 'apache-arrow' is built with the Parquet support but not the package 'python-pyarrow'. To add the support of Parquet to the Python binding, the environment variable PYTHON_WITH_PARQUET should be set to 1, IIRC the doc. Tweaking the 'python-pyarrow' with: --8<---------------cut here---------------start------------->8--- (add-after 'unpack 'set-env (lambda _ (setenv "PYARROW_WITH_PARQUET" "1") #t))))) --8<---------------cut here---------------end--------------->8--- then running "./pre-inst-env guix build python-pyarrow --no-grafts" I get: --8<---------------cut here---------------start------------->8--- -- Found Arrow: /gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include (found version "4.0.1") -- Arrow version: 4.0.1 (CMake package configuration: Arrow) -- Arrow SO and ABI version: 400 -- Arrow full SO version: 400.1.0 -- Found the Arrow core shared library: /gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib/lib/libarrow.so -- Found the Arrow core import library: -- Found the Arrow core static library: -- Could NOT find ArrowPython (missing: ArrowPython_DIR) -- Checking for module 'arrow-python' -- Found arrow-python, version 4.0.1 -- Found ArrowPython: /gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include (found version "4.0.1") -- Found the Arrow Python by pkg-config: arrow-python -- Found the Arrow Python shared library: /gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib/lib/libarrow_python.so -- Found the Arrow Python import library: -- Found the Arrow Python static library: ARROW_PYTHON_static_lib-NOTFOUND -- Could NOT find Parquet (missing: Parquet_DIR) -- Checking for module 'parquet' -- Found parquet, version 4.0.1 -- Found Parquet: /gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib//gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include (found version "4.0.1") -- Parquet version: 4.0.1 (pkg-config: parquet) -- Found the Parquet shared library: /gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib/lib/libparquet.so -- Found the Parquet import library: -- Found the Parquet static library: PARQUET_static_lib-NOTFOUND -- Configuring done CMake Error in CMakeLists.txt: Imported target "parquet_shared" includes non-existent path "/gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib//gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. CMake Error in CMakeLists.txt: Imported target "parquet_shared" includes non-existent path "/gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib//gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include" in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include: * The path was deleted, renamed, or moved to another location. * An install or uninstall procedure did not complete successfully. * The installation package was faulty and references files it does not provide. --8<---------------cut here---------------end--------------->8--- Without this environment variable PYTHON_WITH_PYARROW set to 1, the configuration is not looking after Parquet, for instance: --8<---------------cut here---------------start------------->8--- -- Found Arrow: /gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include (found version "4.0.1") -- Arrow version: 4.0.1 (CMake package configuration: Arrow) -- Arrow SO and ABI version: 400 -- Arrow full SO version: 400.1.0 -- Found the Arrow core shared library: /gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib/lib/libarrow.so -- Found the Arrow core import library: -- Found the Arrow core static library: -- Could NOT find ArrowPython (missing: ArrowPython_DIR) -- Checking for module 'arrow-python' -- Found arrow-python, version 4.0.1 -- Found ArrowPython: /gnu/store/bx4lprdcq5zw372f2rcc27c412chximr-apache-arrow-4.0.1-include/share/include (found version "4.0.1") -- Found the Arrow Python by pkg-config: arrow-python -- Found the Arrow Python shared library: /gnu/store/cagr2n03155mdx1mim8cxva1v22qhh0y-apache-arrow-4.0.1-lib/lib/libarrow_python.so -- Found the Arrow Python import library: -- Found the Arrow Python static library: ARROW_PYTHON_static_lib-NOTFOUND -- Configuring done -- Generating done -- Build files have been written to: /tmp/guix-build-python-pyarrow-4.0.1.drv-0/source/python/build/temp.linux-x86_64-3.8 -- Finished cmake for pyarrow -- Running cmake --build for pyarrow cmake --build . --config release -- make[1]: Entering directory '/tmp/guix-build-python-pyarrow-4.0.1.drv-0/source/python/build/temp.linux-x86_64-3.8' --8<---------------cut here---------------end--------------->8--- Well, setting the environment variable is not enough. :-) Do you want to give a try for fixing it? All the best, simon
