śr., 7 kwi 2021, 18:45 użytkownik ocket 8888 <ocket8...@gmail.com> napisał:
> If your project can afford it, you can add self-hosted GHA runners: > > https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners > The issue with that being that the machine running your actions will > necessarily have write access to the repository through the API, so you > can't just use a server donated by a company . I'm not sure if there's a > way to limit its access based on what your actions actually need, you'll > need to consult the documentation on that topic. > > What might be neat is to see some of the infra resources currently > allocated to Jenkins - if not needed to keep up with load, which is > something I have no idea about - be repurposed as trusted self-hosted GHA > runners. > Just to repeat that again. This is official recommendation from GitHub to NOT USE self hosted runners for public repos. We had to fork the runner and modify it to use it in Airflow. We ever had discussion with GitHub Developer Advocate organized by Gavin and they said 'we can't expect any security improvement here in a foreseeable time. But maybe infra can use our fork of runner and host it :) . The problem with that is that if this is 'free, the projects will have no incentive to optimize their builds - they will quickly use whatever is available. I repeated that many times and repeat it again. Building wider free highways causes more traffic and does not stop traffic jams. They drop a little initially but quickly come back to what they were before. So it all has to start with 'per-project' resource limitation and self- budgeting. It would be GREAT if infra.could provide self-hosted GitHub Runners SERVICE per project, where project could donate credits or money for their own account, then the projects would have incentive to optimize their own usage. I imagine this would be the best thing since the sliced bread that INFRA could provide to all the projects. Here is more info about Airflow version of runners and why it is REALLY dangerous to use unmodified self-hosted runners. As of few days hacker ARE MINING CRYPTOCURRENCY using pull requests for public repos. I warned against this scenario for months and now IT IS HAPPENING. 2) We've forked ([~ash]) github runner to add security layer to only allow our maintainers to run the PRs build on it. https://github.com/ashb/runner This is important because we already know that hackers already use GitHub Actions to mine the cryptocurrencies: https://www.bleepingcomputer.com/news/security/github-actions-being-actively-abused-to-mine-cryptocurrency-on-github-servers/ > On Wed, Apr 7, 2021 at 7:31 AM Hyukjin Kwon <gurwls...@gmail.com> wrote: > > > Thanks Martin for your feedback. > > > > > What was your reason to migrate from Apache Jenkins to Github Actions ? > > > > I am sure there were more reasons for migrating from Amplap Jenkins > > <https://amplab.cs.berkeley.edu/jenkins/> to GitHub Actions but as far > as > > I > > can remember: > > - To reduce the maintenance cost of machines > > - The Jenkins machines became unstable and slow causing CI jobs to fail > or > > be very flaky. > > - Difficulty to manage the installed libraries. > > - Intermittent unknown issues in the machines > > > > Yes, one option might be to consider other options to migrate again. > > However, other projects will very likely suffer the > > same problem. In addition, the migration in a large project is not an > > easy work to do > > > > I would like to know the feasibility of having more resources in GitHub > > Actions, or, for example, having sub-groups where > > each group shares the resources - currently one GitHub organisation > shares > > all resources across the projects. > > > > > > 2021년 4월 7일 (수) 오후 10:04, Martin Grigorov <mgrigo...@apache.org>님이 작성: > > > > > > > > > > > On Wed, Apr 7, 2021 at 3:41 PM Hyukjin Kwon <gurwls...@gmail.com> > wrote: > > > > > >> Hi Greg, > > >> > > >> I raised this thread to figure out a way that we can work together to > > >> resolve this issue, gather feedback, and to understand how other > > projects > > >> work around. > > >> Several projects I observed, as far as I can tell, have made enough > > >> efforts > > >> to save the resources in GitHub Actions but still suffer from the lack > > of > > >> resources. > > >> > > > > > > And it will get even worse because: > > > 1) more and more Apache projects migrate from TravisCI to Github > Actions > > > (GA) > > > 2) new projects join ASF and many of them already use GA > > > > > > > > > What was your reason to migrate from Apache Jenkins to Github Actions ? > > > If you want dedicated resources then you will need to manage the CI > > > yourself. > > > You could use Apache Jenkins/Buildbot with dedicated agents for your > > > project. > > > Or you could set up your own CI infrastructure with Jenkins, DroneIO, > > > ConcourceCI, ... > > > > > > Yet another option is to move to CircleCI or Cirrus. They are similar > to > > > TravisCI / GA and less crowded (for now). > > > > > > Martin > > > > > > I appreciate the resources provided to us but that does not resolve the > > >> issue of the development being slowed down. > > >> > > >> > > >> 2021년 4월 7일 (수) 오후 5:52, Greg Stein <gst...@gmail.com>님이 작성: > > >> > > >> > On Wed, Apr 7, 2021 at 12:25 AM Hyukjin Kwon <gurwls...@gmail.com> > > >> wrote: > > >> > > > >> >> Hi all, > > >> >> > > >> >> I am an Apache Spark PMC, > > >> > > > >> > > > >> > You are a member of the Apache Spark PMC. You are *not* a PMC. > Please > > >> stop > > >> > with that terminology. The Foundation has about 200 PMCs, and you > are > > a > > >> > member of one of them. You are NOT a "PMC" .. you're a person. A PMC > > is > > >> a > > >> > construct of the Foundation. > > >> > > > >> > >... > > >> > > > >> >> I am aware of the limited GitHub Actions resources that are shared > > >> >> across all projects in ASF, > > >> >> and many projects suffer from it. This issue significantly slows > down > > >> the > > >> >> development cycle of > > >> >> other projects, at least Apache Spark. > > >> >> > > >> > > > >> > And the Foundation gets those build minutes for GitHub Actions > > provided > > >> to > > >> > us from GitHub and Microsoft, and we are thankful that they provide > > >> them to > > >> > the Foundation. Maybe it isn't all the build minutes that every > group > > >> > wants, but that is what we have. So it is incumbent upon all of us > to > > >> > figure out how to build more, with fewer minutes. > > >> > > > >> > Say "thank you" to GitHub, please. > > >> > > > >> > Regards, > > >> > -g > > >> > > > >> > > > >> > > > > > >
