On Thu, Mar 5, 2020 at 4:27 PM Wes McKinney <wesmck...@gmail.com> wrote: > > On Thu, Mar 5, 2020 at 5:01 AM Krisztián Szűcs > <szucs.kriszt...@gmail.com> wrote: > > > > On Thu, Mar 5, 2020 at 10:59 AM Antoine Pitrou <anto...@python.org> wrote: > > > > > > > > > It sounds like this would be a good reason to use BuildKite, which AFAIU > > > can automatically provision and operate cloud resources for us? > > That's true for buildbot as well [1] including openstack and mesos support > > which > > would be nice for physical machines. > > I think we should try to reduce the number of services we have to > configure and sysadmin, it has a non-zero cost in time and money. Agree. > > > With the ursa machines down we ended up with two missing services: > > - nightly trigger and report: I've already ported the cron jobs triggering > > the > > builds and reporting to github actions [2] > > - the comment bot: it is implemented in ursabot including the github event > > listener, so buildkite wouldn't provide a solution here because we need > > to maintain a web service for it. > > > > The solution for the comment bot could be to factor out the implementation > > from ursabot and run it from a github actions build triggered using the > > issue_comment event [3] (buildkite doesn't seem to support it currently > > [4]). Luckily it is going to work. Since github has added support for additional action trigger we can act on issue and pull request review comments. I've tested it on my fork [1]. This means that we can implement the comment bot without maintaining any service. I'm working on it.
[1]: https://github.com/kszucs/arrow/runs/488092876?check_suite_focus=true#step:5:8 > > > > We can also host the buildbot buildmaster in the cloud, at least until we > > decomission all of its services. > > > > [1] > > https://docs.buildbot.net/latest/manual/configuration/workers.html#supported-latent-workers > > [2] https://github.com/ursa-labs/crossbow/tree/master/.github/workflows > > [3] > > https://help.github.com/en/actions/reference/events-that-trigger-workflows#issue-comment-event-issue_comment > > [4] https://github.com/buildkite/feedback/issues/288 > > > > > > > > > Le 04/03/2020 à 16:21, Wes McKinney a écrit : > > > > hi folks, > > > > > > > > The tornado the night before last in Nashville, Tennessee temporarily > > > > disabled the physical hardware that I have been running there where > > > > we've been running "Ursabot" builds and where we've been experimenting > > > > with other self-hosted CI solutions like GitHub Actions Self-Hosted > > > > Runners and Buildkite. > > > > > > > > While dedicated physical hardware can be useful to reduce cloud > > > > computing costs, I think this natural disaster should help inform our > > > > approach to this problem: > > > > > > > > * In the event that physical hosted infrastructure becomes > > > > unavailable, we eventually should have the capability to spin up > > > > machines in the cloud with the desired properties (GCE provides both > > > > Linux and Windows VMs, for example) > > > > * Adding new machines to our CI process ideally should not require a > > > > human in-the-loop (GHA presently requires a human -- in particular > > > > someone from ASF Infra -- in the loop to add workers, so this IMHO > > > > should be taken into consideration) > > > > > > > > Any other thoughts about this topic would be welcome. > > > > > > > > Thanks > > > > Wes > > > > > > > > On Thu, Feb 20, 2020 at 9:27 AM Krisztián Szűcs > > > > <szucs.kriszt...@gmail.com> wrote: > > > >> > > > >> On Thu, Feb 20, 2020 at 3:53 PM Wes McKinney <wesmck...@gmail.com> > > > >> wrote: > > > >>> > > > >>> On Thu, Feb 20, 2020 at 8:40 AM Krisztián Szűcs > > > >>> <szucs.kriszt...@gmail.com> wrote: > > > >>>> > > > >>>> On Thu, Feb 20, 2020 at 12:14 PM Wes McKinney <wesmck...@gmail.com> > > > >>>> wrote: > > > >>>>> > > > >>>>> hi Ganesh, > > > >>>>> > > > >>>>> Thanks for writing. > > > >>>>> > > > >>>>> I've been working on setting up Buildkite (BK) as a way for third > > > >>>>> parties for attach machines to run builds on, with a free > > > >>>>> organization > > > >>>>> at > > > >>>>> > > > >>>>> https://buildkite.com/apache-arrow > > > >>>>> > > > >>>>> Configuring a new machine to accept builds is very easy [1] and > > > >>>>> takes > > > >>>>> less than 60 seconds on Linux or macOS (though maybe a bit more work > > > >>>>> on Windows). Currently I've attached 6 machines: > > > >>>>> > > > >>>>> * 2 CUDA-capable Linux x86 > > > >>>>> * 3 armhf machines (not super high-powered), 1 CUDA-capable > > > >>>>> * 1 macOS > > > >>>>> > > > >>>>> We're still waiting on ASF Infra to twiddle some bits so that builds > > > >>>>> triggered in BK can report commit statuses on GitHub [2] > > > >>>>> > > > >>>>> It's possible we can use self-hosted GitHub Actions (GHA) for this > > > >>>>> also but the workflow for new machines to be contributed needs to be > > > >>>>> proven out. > > > >>>> I've already tried it out, and setting up self-hosted github runners > > > >>>> is just as > > > >>>> easy as with buildkite, drawbacks: > > > >>> > > > >>> I don't mean to be argumentative, but I don't see how this can be true > > > >>> if we don't have access to the "Settings" tab on GitHub > > > >> On a fork where I have access for that tab. > > > >>> > > > >>> https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners > > > >>> > > > >>>> - I'm unsure how would the tagging selection work in practice [1] > > > >>>> - We won't have access to the runners dashboard in lack of admin > > > >>>> rights > > > >>>> for the apache/arrow repository - so we need to test out the > > > >>>> workflow. > > > >>>> > > > >>>> I've created an INFRA ticket to get some information and to track it: > > > >>>> https://issues.apache.org/jira/browse/INFRA-19875 > > > >>>> > > > >>>> [1] > > > >>>> https://help.github.com/en/actions/configuring-and-managing-workflows/configuring-a-workflow#using-a-self-hosted-runner > > > >>>>> > > > >>>>> Thanks, > > > >>>>> Wes > > > >>>>> > > > >>>>> [1]: > > > >>>>> https://github.com/ursa-labs/dev-tools/blob/master/buildkite/debian_agent_bootstrap.sh > > > >>>>> [2]: https://issues.apache.org/jira/browse/INFRA-19217 > > > >>>>> > > > >>>>> On Wed, Feb 19, 2020 at 3:38 PM Ganesh Raju > > > >>>>> <ganesh.r...@linaro.org> wrote: > > > >>>>>> > > > >>>>>> Hi, > > > >>>>>> I am following up on the discussion from here > > > >>>>>> <https://github.com/apache/arrow/pull/6253>, with interest to have > > > >>>>>> dedicated arm hardware for CI setup. We can surely help with that > > > >>>>>> if we get > > > >>>>>> a go-ahead from the project. > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> Ganesh > > > >>>>>> > > > >>>>>> -- > > > >>>>>> IRC: ganeshraju@#linaro on irc.freenode.ne > > > >>>>>> <http://irc.freenode.net/>t
