hi Tom, Looks like it could be https://bugs.python.org/issue32973, but I'm not sure. I wasn't able to reproduce locally. The Python version 3.7.1 running in CI is also potentially suspicious.
This class of error seems to have a lot of bug reports based on Google searches Message isn't picklable so we should probably fix that regardless https://issues.apache.org/jira/browse/ARROW-7826 - Wes On Mon, Feb 10, 2020 at 12:17 PM Tom Augspurger <tom.augspurge...@gmail.com> wrote: > > Hi all, > > I'm seeing a strange issue when importing pyarrow on the intake CI. I get an > exception saying > > AttributeError: type object 'pyarrow.lib.Message' has no attribute > '__reduce_cython__' > > The full traceback is: > > ______________________________ test_arrow_import > _______________________________ > > def test_arrow_import(): > > > import pyarrow > > intake/cli/server/tests/test_server.py:32: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ > > ../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49: > in <module> > > from pyarrow.lib import cpu_count, set_cpu_count > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ > > > ??? > > E AttributeError: type object 'pyarrow.lib.Message' has no attribute > '__reduce_cython__' > > pyarrow/ipc.pxi:21: AttributeError > > _________________ TestServerV1Source.test_read_part_compressed > _________________ > > > I'm unable to reproduce this locally, and was wondering if anyone else has > seen something similar. > Pyarrow was installed using pip / a wheel ( > https://travis-ci.org/intake/intake/jobs/648523104#L311). > > A common cause of this error message is building with too old of a Cython. > While checking this, I noticed > that some of the files are generated with Cython 0.29.8, while others were > generated with 0.29.14. > I have no idea if this is a problem in general of if it's causing this > specific issue. > > ``` > _hdfs.cpp:1:/* Generated by Cython 0.29.14 */ > include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */ > include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */ > _plasma.cpp:1:/* Generated by Cython 0.29.14 */ > _fs.cpp:1:/* Generated by Cython 0.29.14 */ > lib_api.h:1:/* Generated by Cython 0.29.14 */ > gandiva.cpp:1:/* Generated by Cython 0.29.14 */ > _json.cpp:1:/* Generated by Cython 0.29.14 */ > _parquet.cpp:1:/* Generated by Cython 0.29.14 */ > _csv.cpp:1:/* Generated by Cython 0.29.14 */ > _compute.cpp:1:/* Generated by Cython 0.29.14 */ > _dataset.cpp:1:/* Generated by Cython 0.29.14 */ > _flight.cpp:1:/* Generated by Cython 0.29.14 */ > lib.cpp:1:/* Generated by Cython 0.29.14 */ > ``` > > See the https://travis-ci.org/intake/intake/jobs/648523104 for the full log. > > > Thanks for any pointers!
