hi folks, So, I want to bring light to the problems we are having delivering binary artifacts after Arrow releases.
We have some amount of packaging automation implemented in https://github.com/apache/arrow-dist using Travis CI and Appveyor to upload packages to Bintray, a packaging hosting service. Unfortunately, we discovered a bunch of problems with these packaging scripts after the release vote closed on Monday, and now 4 days later, we still have been unable to post binaries to https://pypi.python.org/pypi/pyarrow This is no one's fault, but it highlights structural problems with our development process: * Why does producing packages after a release require error-prone manual labor? * Why are we only finding out about packaging problem after a release vote closes? * Why is setting up nightly binary builds a brittle and bespoke process? I hope all agree that: * Packaging should not be a hardship or require a lot of manual labor * Packaging problems on the master branch should be made known within ~24 hours, so they can be remedied immediately * It should be straightforward to produce binary artifacts for all supported platforms and programming languages Eventually, we should include some binary artifacts in our release votes, but we are pretty far away from suitable automation to make this possible. I don't know any easy solutions, but Apache Arrow has grown widely used enough that I think it's worth our taking the time to plan and execute some solutions to these problems, which I expect to pay dividends in our community's productivity over time. Thanks, Wes
