(big CC list so the people involved have visibility) we're currently using a (very old) installation of anaconda python to manage the python3 build deps, but it turns out that our (very old) versions of numpy and pandas are starting to hold us back.
reference: https://github.com/apache/spark/pull/18459 i'm proposing an upgrade to bring numpy and pandas up to 1.11.3 and 0.19.2 respectively... however, as with all things "dependency hell" related, a lot more will be upgraded in the process: https://github.com/apache/spark/pull/18459#issuecomment-313468679 the upgrade is easy to do, and to facilitate "holy shit i broke the world" rollbacks, the original anaconda installation has been backed up and i can revert in moments. i'm reasonably sure that this upgrade won't horribly break things, but it's hard to know for sure w/o a dedicated staging environment (also on the books for the end-of-summer new jenkins rollout). if people seem ok w/this plan (update, watch builds for failures, rollback immediately if necessary), i will do this first thing tomorrow morning (~7am PDT). thoughts? comments? concerns? shane --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
