I know in Spark we’ve benefited by having some of the different language devs act as RMs and each time that language dev has ended up improving a bunch of how their components packaging has been done. Not to suggest we should just do what other projects do, but maybe an idea to consider?
On Fri, Mar 23, 2018 at 12:59 PM Wes McKinney <wesmck...@gmail.com> wrote: > hi folks, > > So, I want to bring light to the problems we are having delivering > binary artifacts after Arrow releases. > > We have some amount of packaging automation implemented in > https://github.com/apache/arrow-dist using Travis CI and Appveyor to > upload packages to Bintray, a packaging hosting service. > > Unfortunately, we discovered a bunch of problems with these packaging > scripts after the release vote closed on Monday, and now 4 days later, > we still have been unable to post binaries to > https://pypi.python.org/pypi/pyarrow > > This is no one's fault, but it highlights structural problems with our > development process: > > * Why does producing packages after a release require error-prone manual > labor? > > * Why are we only finding out about packaging problem after a release > vote closes? > > * Why is setting up nightly binary builds a brittle and bespoke process? > > I hope all agree that: > > * Packaging should not be a hardship or require a lot of manual labor > > * Packaging problems on the master branch should be made known within > ~24 hours, so they can be remedied immediately > > * It should be straightforward to produce binary artifacts for all > supported platforms and programming languages > > Eventually, we should include some binary artifacts in our release > votes, but we are pretty far away from suitable automation to make > this possible. > > I don't know any easy solutions, but Apache Arrow has grown widely > used enough that I think it's worth our taking the time to plan and > execute some solutions to these problems, which I expect to pay > dividends in our community's productivity over time. > > Thanks, > Wes > -- Twitter: https://twitter.com/holdenkarau
