What is the main barrier to getting CircleCI to work with Apache projects? On Thu, Feb 1, 2018 at 2:03 PM Uwe L. Korn <uw...@xhochy.com> wrote:
> I just went over a lot of open PRs and sadly I wasn't able to reduce the > number of open ones significantly. Some of them make slow progress and it > might be worthwhile to jump in in a week, for now I would rather wait and > let the initial authors finish them to get more involved in the project. > Currently the CI issues are a main bottleneck for all of us, besides the > long-running Python tests, we also spent a lot of time on the environment > setup. Typically this is a thing that can really be improved with a docker > setup, sadly Travis takes quite some time to pull the current image we use > for the manylinux1 build. I'll first have a look at improving it and if the > download times get better, we might want to move some things in there > (sadly CircleCI and Apache projects still don't work together). > > Also I think a confusing thing is that we have separate documentations > between Python and C++. This is also a thing I'm going to work on once I > have some time. The two implementation are bound very thight together and a > lot that applies to one language also applies to the other one. > > Uwe > > On Thu, Feb 1, 2018, at 6:09 PM, Wes McKinney wrote: > > hi folks, > > > > We've had a rough couple of weeks in our PR queue due to various CI > > issues causing a high incidence of build failures: > > > > * Package dependency upgrades (Thrift -- this has been fixed) > > * Failures due possibly to VM setting changes in Travis CI (memory > > thrashing / VM timeouts, see ARROW-2062, ARROW-2071) > > * apt flakiness (this is still ongoing, see ARROW-2021) > > > > Meanwhile, at the moment, we have 37 open PRs > > (https://github.com/apache/arrow/pulls). Some of these are stale and > > need to either be reviewed, updated, or closed. We have many other PRs > > that need to be rebased (builds should mostly pass now if rebased on > > master) and/or reviewed. I've been doing the best I can do keep up > > with the PR queue (and others have been reviewing and merging PRs, > > too), but it's currently not enough to keep up, and there's a lot of > > development work for the 0.9.0 milestone that I'd like to also be > > doing. > > > > The project is growing fast -- both in users and new developers. Just > > on a single install path for the Python libraries, Arrow is being > > installed _over 1000 times per day_ > > (https://anaconda.org/conda-forge/pyarrow) -- when you add up all the > > install paths it is likely to be much more than that. > > > > Reviews and help maintaining PRs from the community, but especially > > from other committers and PMC members, would be especially useful > > right now to get the project operating smoothly with a steady stream > > of high quality patches making their way into master. > > > > If there's anything else we can do to improve developer and community > > productivity in Arrow right now, I'm open to ideas. > > > > Thanks, > > Wes >
