I just went over a lot of open PRs and sadly I wasn't able to reduce the number of open ones significantly. Some of them make slow progress and it might be worthwhile to jump in in a week, for now I would rather wait and let the initial authors finish them to get more involved in the project. Currently the CI issues are a main bottleneck for all of us, besides the long-running Python tests, we also spent a lot of time on the environment setup. Typically this is a thing that can really be improved with a docker setup, sadly Travis takes quite some time to pull the current image we use for the manylinux1 build. I'll first have a look at improving it and if the download times get better, we might want to move some things in there (sadly CircleCI and Apache projects still don't work together).
Also I think a confusing thing is that we have separate documentations between Python and C++. This is also a thing I'm going to work on once I have some time. The two implementation are bound very thight together and a lot that applies to one language also applies to the other one. Uwe On Thu, Feb 1, 2018, at 6:09 PM, Wes McKinney wrote: > hi folks, > > We've had a rough couple of weeks in our PR queue due to various CI > issues causing a high incidence of build failures: > > * Package dependency upgrades (Thrift -- this has been fixed) > * Failures due possibly to VM setting changes in Travis CI (memory > thrashing / VM timeouts, see ARROW-2062, ARROW-2071) > * apt flakiness (this is still ongoing, see ARROW-2021) > > Meanwhile, at the moment, we have 37 open PRs > (https://github.com/apache/arrow/pulls). Some of these are stale and > need to either be reviewed, updated, or closed. We have many other PRs > that need to be rebased (builds should mostly pass now if rebased on > master) and/or reviewed. I've been doing the best I can do keep up > with the PR queue (and others have been reviewing and merging PRs, > too), but it's currently not enough to keep up, and there's a lot of > development work for the 0.9.0 milestone that I'd like to also be > doing. > > The project is growing fast -- both in users and new developers. Just > on a single install path for the Python libraries, Arrow is being > installed _over 1000 times per day_ > (https://anaconda.org/conda-forge/pyarrow) -- when you add up all the > install paths it is likely to be much more than that. > > Reviews and help maintaining PRs from the community, but especially > from other committers and PMC members, would be especially useful > right now to get the project operating smoothly with a steady stream > of high quality patches making their way into master. > > If there's anything else we can do to improve developer and community > productivity in Arrow right now, I'm open to ideas. > > Thanks, > Wes
