On Fri, Apr 16, 2021 at 3:44 PM Hyukjin Kwon <gurwls...@gmail.com> wrote:
> I thought Jarek was pretty clear on that. I meant this: > > > So it all has to start with 'per-project' resource limitation and self- > > budgeting. It would be GREAT if infra.could provide self-hosted GitHub > > Runners SERVICE per project, where project could donate credits or money > > for their own account, then the projects would have incentive to optimize > > their own usage. I imagine this would be the best thing since the sliced > > bread that INFRA could provide to all the projects. > > Maintaining and providing a self-hosted runners in GitHub Actions where > the resources are managed in project level where each project can donate > credits. > > In addition, Jarek mentioned that Airflow already has a working version - > is it correct Jarek? > > If the infra team takes and improves it for other ASF projects, that would > permanently resolve this issue. > > This suggestion looks reasonable and realistic to me. > > How do you think about this? > I'll let Infra team respond for themselves but to me all these improvements should be done by Github Actions team, not by each and every project out there. But if your project wants to use Apache Airflow's modifications then you can do it - just follow what they did at https://github.com/apache/airflow/blob/master/.github/workflows/ci.yml > > > On Fri, 16 Apr 2021, 21:36 Martin Grigorov, <mgrigo...@apache.org> wrote: > >> Hi Hyukjin, >> >> On Fri, Apr 16, 2021 at 3:04 AM Hyukjin Kwon <gurwls...@gmail.com> wrote: >> >> > Hi all, >> > >> > Is here the right place to expect feedback from the infra team or >> related >> > people? >> > It would be great to hear what the infra team thinks about Jarek's >> > suggestion. >> > >> >> What suggestion exactly do you mean ? >> I've just re-read Jarek's email and I see 3 tasks for Github Actions team, >> but nothing specific for Apache Infra team. >> >> >> > >> > >> > 2021년 4월 13일 (화) 오전 11:15, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: >> > >> >> Hi all, >> >> >> >> Could we have any update and feedback from the INFRA team about Jarek's >> >> suggestion please? >> >> >> >> 2021년 4월 9일 (금) 오전 7:06, Jarek Potiuk <ja...@potiuk.com>님이 작성: >> >> >> >>> >> >>>> That's a good idea. We do need to thank Github to give free >> resources to >> >>>> ASF projects, but it's better if we can make it a business: we allow >> >>>> individual projects to sign deals with Github to get dedicated >> >>>> resources. >> >>>> It's a bit wasteful to ask every project to set up its own dev ops, >> >>>> using Github Action is more convenient. Maybe we should raise it to >> >>>> Github? >> >>>> >> >>> >> >>> I do not think you can get per-project resources in GH - the most you >> >>> can do are self-hosted runners for your project. >> >>> >> >>> (BTW I am not from the INFRA team - just a humble "CI person" of >> Apache >> >>> Airflow but very much vested into Github Actions) >> >>> maybe the infra team can chime in here. We did raise it to GitHub, we >> >>> even had meeting with them >> >>> organized by Gavin and several topics were raised that could be >> >>> eventually addressed by Github: >> >>> >> >>> - observability (they could not give us per-project usage dashboard - >> we >> >>> built our own imperfect (with API limitations) one by Tobiasz from >> Airllow >> >>> - security (limiting access to only project committers) - this we >> >>> handled by the Ash's fork of Runner (but it's also imperfect - even >> today I >> >>> had to fix a problem where we had list of committers desynchronised >> between >> >>> our infra/CI.yml) >> >>> - manageability (assigning resources per-project) - this works by >> having >> >>> self-hosted runners assigned per project (we needed infra JIRA ticket >> and >> >>> generation of a bunch of tokens for our runners and our own AWS >> account >> >>> with auto-scaling). >> >>> >> >>> It would be indeed great if it could be available from GitHub, but so >> >>> far we do not have any of those. >> >>> >> >>> J. >> >>> >> >>> >> >>> >> >>>> On Wed, Apr 7, 2021 at 9:31 PM Hyukjin Kwon <gurwls...@gmail.com> >> >>>> wrote: >> >>>> >> >>>> > Thanks Martin for your feedback. >> >>>> > >> >>>> > > What was your reason to migrate from Apache Jenkins to Github >> >>>> Actions ? >> >>>> > >> >>>> > I am sure there were more reasons for migrating from Amplap Jenkins >> >>>> > <https://amplab.cs.berkeley.edu/jenkins/> to GitHub Actions but as >> >>>> far as >> >>>> > I can remember: >> >>>> > - To reduce the maintenance cost of machines >> >>>> > - The Jenkins machines became unstable and slow causing CI jobs to >> >>>> fail or >> >>>> > be very flaky. >> >>>> > - Difficulty to manage the installed libraries. >> >>>> > - Intermittent unknown issues in the machines >> >>>> > >> >>>> > Yes, one option might be to consider other options to migrate >> again. >> >>>> > However, other projects will very likely suffer the >> >>>> > same problem. In addition, the migration in a large project is not >> an >> >>>> > easy work to do >> >>>> > >> >>>> > I would like to know the feasibility of having more resources in >> >>>> GitHub >> >>>> > Actions, or, for example, having sub-groups where >> >>>> > each group shares the resources - currently one GitHub organisation >> >>>> shares >> >>>> > all resources across the projects. >> >>>> > >> >>>> > >> >>>> > 2021년 4월 7일 (수) 오후 10:04, Martin Grigorov <mgrigo...@apache.org>님이 >> >>>> 작성: >> >>>> > >> >>>> >> >> >>>> >> >> >>>> >> On Wed, Apr 7, 2021 at 3:41 PM Hyukjin Kwon <gurwls...@gmail.com> >> >>>> wrote: >> >>>> >> >> >>>> >>> Hi Greg, >> >>>> >>> >> >>>> >>> I raised this thread to figure out a way that we can work >> together >> >>>> to >> >>>> >>> resolve this issue, gather feedback, and to understand how other >> >>>> projects >> >>>> >>> work around. >> >>>> >>> Several projects I observed, as far as I can tell, have made >> enough >> >>>> >>> efforts >> >>>> >>> to save the resources in GitHub Actions but still suffer from the >> >>>> lack of >> >>>> >>> resources. >> >>>> >>> >> >>>> >> >> >>>> >> And it will get even worse because: >> >>>> >> 1) more and more Apache projects migrate from TravisCI to Github >> >>>> Actions >> >>>> >> (GA) >> >>>> >> 2) new projects join ASF and many of them already use GA >> >>>> >> >> >>>> >> >> >>>> >> What was your reason to migrate from Apache Jenkins to Github >> >>>> Actions ? >> >>>> >> If you want dedicated resources then you will need to manage the >> CI >> >>>> >> yourself. >> >>>> >> You could use Apache Jenkins/Buildbot with dedicated agents for >> your >> >>>> >> project. >> >>>> >> Or you could set up your own CI infrastructure with Jenkins, >> DroneIO, >> >>>> >> ConcourceCI, ... >> >>>> >> >> >>>> >> Yet another option is to move to CircleCI or Cirrus. They are >> >>>> similar to >> >>>> >> TravisCI / GA and less crowded (for now). >> >>>> >> >> >>>> >> Martin >> >>>> >> >> >>>> >> I appreciate the resources provided to us but that does not >> resolve >> >>>> the >> >>>> >>> issue of the development being slowed down. >> >>>> >>> >> >>>> >>> >> >>>> >>> 2021년 4월 7일 (수) 오후 5:52, Greg Stein <gst...@gmail.com>님이 작성: >> >>>> >>> >> >>>> >>> > On Wed, Apr 7, 2021 at 12:25 AM Hyukjin Kwon < >> gurwls...@gmail.com >> >>>> > >> >>>> >>> wrote: >> >>>> >>> > >> >>>> >>> >> Hi all, >> >>>> >>> >> >> >>>> >>> >> I am an Apache Spark PMC, >> >>>> >>> > >> >>>> >>> > >> >>>> >>> > You are a member of the Apache Spark PMC. You are *not* a PMC. >> >>>> Please >> >>>> >>> stop >> >>>> >>> > with that terminology. The Foundation has about 200 PMCs, and >> you >> >>>> are a >> >>>> >>> > member of one of them. You are NOT a "PMC" .. you're a person. >> A >> >>>> PMC >> >>>> >>> is a >> >>>> >>> > construct of the Foundation. >> >>>> >>> > >> >>>> >>> > >... >> >>>> >>> > >> >>>> >>> >> I am aware of the limited GitHub Actions resources that are >> >>>> shared >> >>>> >>> >> across all projects in ASF, >> >>>> >>> >> and many projects suffer from it. This issue significantly >> slows >> >>>> down >> >>>> >>> the >> >>>> >>> >> development cycle of >> >>>> >>> >> other projects, at least Apache Spark. >> >>>> >>> >> >> >>>> >>> > >> >>>> >>> > And the Foundation gets those build minutes for GitHub Actions >> >>>> >>> provided to >> >>>> >>> > us from GitHub and Microsoft, and we are thankful that they >> >>>> provide >> >>>> >>> them to >> >>>> >>> > the Foundation. Maybe it isn't all the build minutes that every >> >>>> group >> >>>> >>> > wants, but that is what we have. So it is incumbent upon all of >> >>>> us to >> >>>> >>> > figure out how to build more, with fewer minutes. >> >>>> >>> > >> >>>> >>> > Say "thank you" to GitHub, please. >> >>>> >>> > >> >>>> >>> > Regards, >> >>>> >>> > -g >> >>>> >>> > >> >>>> >>> > >> >>>> >>> >> >>>> >> >> >>>> >> >>> >> >>> >> >>> -- >> >>> +48 660 796 129 >> >>> >> >> >> >
