Re: [build system] IMPORTANT: taking centos workers offline for pyarrow upgrade

2018-08-08 Thread shane knapp
test builds started (on ubuntu). if these pass, i will feel comfortable performing the same installation/upgrade steps on the centos workers. upgrade/installation commands: conda install python==3.5 conda install pyarrow=0.10.* -c conda-forge -n py3k pip install sphinx builds: https://amplab.

Re: [build system] IMPORTANT: taking centos workers offline for pyarrow upgrade

2018-08-08 Thread shane knapp
i updated my staging/testing ubuntu server to python 3.5, successfully installed pyarrow 0.10.0 via conda forge. however i'm getting a ton of python package dep failures. i will see if we can get by w/o needing to wipe and recreate every anaconda installation. please hold. On Wed, Aug 8, 2018 a

Re: [build system] IMPORTANT: taking centos workers offline for pyarrow upgrade

2018-08-08 Thread shane knapp
well... i've been running in to problems (aka dependency hell), and just hit a show-stopper: UnsatisfiableError: The following specifications were found to be in conflict: - pyarrow 0.10.* -> arrow-cpp 0.10.0.* -> python >=2.7,<2.8.0a0 - python 3.4* Use "conda info " to see the dependencies f

[build system] IMPORTANT: taking centos workers offline for pyarrow upgrade

2018-08-08 Thread shane knapp
pyarrow 0.10.0 has been released, and this is important to be tested against for the 2.4 release (esp due to memory leak problems, etc) https://issues.apache.org/jira/browse/SPARK-23874 https://github.com/apache/spark/pull/21939 https://issues.apache.org/jira/browse/ARROW-1973 i will be putting t