+1 On Thu, Oct 10, 2019, 2:12 PM Jacques Nadeau <jacq...@apache.org> wrote:
> Proposed report update below. LMK your thoughts. > > ## Description: > The mission of Apache Arrow is the creation and maintenance of software > related to columnar in-memory processing and data interchange > > ## Issues: > > * We are struggling with Continuous Integration scalability as the project > has > definitely outgrown what Travis CI and Appveyor can do for us. Some > contributors have shown reluctance to submit patches they aren't sure > about > because they don't want to pile on the build queue. We are exploring > alternative solutions such as Buildbot, Buildkite, and GitHub Actions to > provide a path to migrate away from Travis CI / Appveyor. In our request > to > Infrastructure INFRA-19217, some of us were alarmed to find that an CI/CD > service like Buildkite may not be able to be connected to the @apache > GitHub > account on account of requiring admin access to repository webhooks, but > no > ability to modify source code. There are workarounds (building custom > OAuth > bots) that could enable us to use Buildkite, but it would require extra > development and result in a less refined experience for community > members. > > > > ## Membership Data: > * Apache Arrow was founded 2016-01-19 (4 years ago) > * There are currently 48 committers and 28 PMC members in this project. > * The Committer-to-PMC ratio is roughly 3:2. > > Community changes, past quarter: > - Micah Kornfield was added to the PMC on 2019-08-21 > - Sebastien Binet was added to the PMC on 2019-08-21 > - Ben Kietzman was added as committer on 2019-09-07 > - David Li was added as committer on 2019-08-30 > - Kenta Murata was added as committer on 2019-09-05 > - Neal Richardson was added as committer on 2019-09-05 > - Praveen Kumar was added as committer on 2019-07-14 > > ## Project Activity: > > * The project has just made a 0.15.0 release. > * We are discussing ways to make the Arrow libraries as accessible as > possible > to downstream projects for minimal use cases while allowing the > development > of more comprehensive "standard libraries" with larger dependency stacks > in > the project > * We plan to make a 1.0.0 release as our next major release, at which time > we > will declare that the Arrow binary protocol is stable with forward and > backward compatibility guarantees > > ## Community Health: > > * The community is continuing to grow at a great rate. We see good adoption > among many other projects and fast growth of key metrics. > * Many contributors are struggling with the slowness of pre-commit CI. > Arrow > has a large number of different platforms and components and a complex > build > matrix. As new commits come in, they frequently take a long time to > complete. The community is trying several ways to solve this. There is > bubbling frustration in the community around the GitHub repo rules for > using > third party services. This is especially challenging when there are free > solutions to relieve the community pressure but the community is unable > to > access these resources. This frustration is greatest among people who > work > on many non-asf OSS projects which don't have such restrictive rules > around GitHub. Some examples of ways the community has tried to resolve > these have included: > * Try to use CircleCI, rejected in INFRA-15964 > * Try to use Azure Pipelines, rejected in INFRA-17030 > * Try to resolves Issues with Travis CI capacity: INFRA-18533 & > https://s.apache.org/ci-capacity (no resolution beyond "find > donations") > * The creation of new infrastructure design (in progress but a huge > amount of > thankless work) > * While the community has seen great growth in contribution (more than 300 > unique contributors at this point), the vast majority are casual > contributors. The daily active committers (the workhorses of the project > that bear the load committing the constant PRs, more than 5000 closed at > this point) have been growing slower than adoption. This is despite the > fact > that the community has been very aggressive at being inclusive of new > committers (with likelihood to have more than 50 in the next week). The > community is still continuing to try to brainstorm ways to improve this. >
