I think we need to more direct in listing issues for the board. What have we done? What do we want them to do?
In general, any large org is going to be slow to add new deep integrations into GitHub. I don't think we should expect Apache to be any different (it took several years before we could merge things through github for example). If I were on the INFRA side, I think I would look and see how many different people are asking for BuildKite before considering integration. It seems like we only opened the JIRA 6 days ago and no other projects have requested access to this? I'm not clear why this is a board issue. What do we think the board can do for us that we can't solve ourselves and need them to solve? Remember, a board solution to a problem is typically very removed from what matters to individuals on a project. On Tue, Oct 8, 2019 at 7:03 AM Wes McKinney <wesmck...@gmail.com> wrote: > New draft > > ## Description: > The mission of Apache Arrow is the creation and maintenance of software > related > to columnar in-memory processing and data interchange > > ## Issues: > > * We are struggling with Continuous Integration scalability as the project > has > definitely outgrown what Travis CI and Appveyor can do for us. Some > contributors have shown reluctance to submit patches they aren't sure > about > because they don't want to pile on the build queue. We are exploring > alternative solutions such as Buildbot, Buildkite, and GitHub Actions to > provide a path to migrate away from Travis CI / Appveyor. In our request > to > Infrastructure INFRA-19217, some of us were alarmed to find that an CI/CD > service like Buildkite may not be able to be connected to the @apache > GitHub > account on account of requiring admin access to repository webhooks, but > no > ability to modify source code. There are workarounds (building custom > OAuth > bots) that could enable us to use Buildkite, but it would require extra > development and result in a less refined experience for community > members. > > ## Membership Data: > * Apache Arrow was founded 2016-01-19 (4 years ago) > * There are currently 48 committers and 28 PMC members in this project. > * The Committer-to-PMC ratio is roughly 3:2. > > Community changes, past quarter: > - Micah Kornfield was added to the PMC on 2019-08-21 > - Sebastien Binet was added to the PMC on 2019-08-21 > - Ben Kietzman was added as committer on 2019-09-07 > - David Li was added as committer on 2019-08-30 > - Kenta Murata was added as committer on 2019-09-05 > - Neal Richardson was added as committer on 2019-09-05 > - Praveen Kumar was added as committer on 2019-07-14 > > ## Project Activity: > > * The project has just made a 0.15.0 release. > * We are discussing ways to make the Arrow libraries as accessible as > possible > to downstream projects for minimal use cases while allowing the > development > of more comprehensive "standard libraries" with larger dependency stacks > in > the project > * We plan to make a 1.0.0 release as our next major release, at which time > we > will declare that the Arrow binary protocol is stable with forward and > backward compatibility guarantees > > ## Community Health: > > * The community is overall healthy, with the aforementioned concerns > around CI > scalability. New contributors frequently take notice of the long build > queue > times when submitting pull requests. > > On Tue, Oct 8, 2019 at 8:58 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > > Yes, I agree with raising the issue to the board. > > > > On Tue, Oct 8, 2019 at 8:31 AM Antoine Pitrou <anto...@python.org> > wrote: > > > > > > > > > I agree. Especially given that the constraints imposed by Infra don't > > > help solving the problem. > > > > > > Regards > > > > > > Antoine. > > > > > > > > > Le 08/10/2019 à 15:02, Uwe L. Korn a écrit : > > > > I'm not sure what qualifies for "board attention" but it seems that > CI is a critical problem in Apache projects, not just Arrow. Should we > raise that? > > > > > > > > Uwe > > > > > > > > On Tue, Oct 8, 2019, at 12:00 AM, Wes McKinney wrote: > > > >> Here is a start for our Q3 board report > > > >> > > > >> ## Description: > > > >> The mission of Apache Arrow is the creation and maintenance of > software related > > > >> to columnar in-memory processing and data interchange > > > >> > > > >> ## Issues: > > > >> There are no issues requiring board attention at this time > > > >> > > > >> ## Membership Data: > > > >> * Apache Arrow was founded 2016-01-19 (4 years ago) > > > >> * There are currently 48 committers and 28 PMC members in this > project. > > > >> * The Committer-to-PMC ratio is roughly 3:2. > > > >> > > > >> Community changes, past quarter: > > > >> - Micah Kornfield was added to the PMC on 2019-08-21 > > > >> - Sebastien Binet was added to the PMC on 2019-08-21 > > > >> - Ben Kietzman was added as committer on 2019-09-07 > > > >> - David Li was added as committer on 2019-08-30 > > > >> - Kenta Murata was added as committer on 2019-09-05 > > > >> - Neal Richardson was added as committer on 2019-09-05 > > > >> - Praveen Kumar was added as committer on 2019-07-14 > > > >> > > > >> ## Project Activity: > > > >> > > > >> * The project has just made a 0.15.0 release. > > > >> * We are discussing ways to make the Arrow libraries as accessible > as possible > > > >> to downstream projects for minimal use cases while allowing the > development > > > >> of more comprehensive "standard libraries" with larger dependency > stacks in > > > >> the project > > > >> * We plan to make a 1.0.0 release as our next major release, at > which time we > > > >> will declare that the Arrow binary protocol is stable with > forward and > > > >> backward compatibility guarantees > > > >> * We are struggling with Continuous Integration scalability as the > project has > > > >> definitely outgrown what Travis CI and Appveyor can do for us. We > are > > > >> exploring alternative solutions such as Buildbot, Buildkite (see > > > >> INFRA-19217), and GitHub Actions to provide a path to migrate > away from > > > >> Travis CI / Appveyor > > > >> > > > >> ## Community Health: > > > >> > > > >> * The community is overall healthy, with the aforementioned > concerns around CI > > > >> scalability. New contributors frequently take notice of the long > build queue > > > >> times when submitting pull requests. > > > >> >
