I've justed created a parent JIRA for Docker-ifying all of the Linux builds in Travis CI
https://issues.apache.org/jira/browse/ARROW-5801 I did Java here since it was one of the easier ones https://github.com/apache/arrow/pull/4761 Expensive Docker images can be pushed to @ursalab on Docker Hub (https://cloud.docker.com/u/ursalab/repository/list) -- I will be happy to give any Arrow committer access to this Docker Hub organization to help maintain the images in the short term. Some of the others will require more work. We should think about how to refactor the build scripts for macOS in a way that is decoupled from Travis CI environment variables and custom build image details so they can more easily be run on arbitrary macOS build workers. On Sun, Jun 30, 2019 at 11:26 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > GitLab is currently more mature but on the other hand we're already on > > GitHub. We should probably evaluate both options if we go this way. > > We have to keep the code repository on GitHub because all Apache > projects are on GitHub now. How projects manage patches and CI is up > to each project, though. Other projects I'm familiar with like Apache > Kudu and Apache Impala use Gerrit and Jenkins for their code review > and CI, respectively. > > If we can use GitLab CI and get it to make status reports into our PR > queue on GitHub, that would be nice. Buildbot is another option > (though they are not mutually exclusive). My colleagues and I plan go > to continue investing in our Buildbot infrastructure though not > necessarily to the exclusion of other things. > > Perhaps we can set up a proof of concept on GitLab to see what things > look like. I've set up a repository mirror at > > https://gitlab.com/ursa-labs/arrow > > where we can experiment > > On Sat, Jun 29, 2019 at 10:02 PM Jed Brown <j...@jedbrown.org> wrote: > > > > Sutou Kouhei <k...@clear-code.com> writes: > > > > > How about creating a mirror repository on > > > https://gitlab.com/ only to run CI jobs? > > > > > > This is an idea that is described in > > > https://issues.apache.org/jira/browse/ARROW-5673 . > > > > > > GitLab CI can attach external workers. So we can increase CI > > > capacity by adding our new workers. GitLab also provides > > > Docker registry. It means that we can cache built Docker > > > images for our CI. It will reduce CI time. > > > > I have some experience with GitLab-CI. The gitlab-runner is great and > > easy to deploy. GitLab-CI integrates structured artifacts (like which > > tests pass and their output [1] and changes to continuous metrics like > > performance [2]) very nicely into GitLab Merge Requests, but when you > > connect to an external repository (GitHub, etc.), it only reports > > pass/fail and you can't access the structured artifacts [3], only the > > console logs and compressed archives of artifacts if you use that > > feature. > > > > If you're happy with clicking through to console logs in case a pipeline > > fails (the Travis model), then GitLab-CI is easy to use and will serve > > your purposes. If you really want the structured features, then I'd > > encourage you to mention that in [3]. > > > > [1] https://docs.gitlab.com/ee/ci/junit_test_reports.html#how-it-works > > [2] > > https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html#how-it-works > > [3] https://gitlab.com/gitlab-org/gitlab-ce/issues/60158
