Ok. The issue is solved for Airflow now. We moved 7 repos in and merged PR changing the location, enabled workflows back and it seems to be back to normal - but I think we need to conclude the discussion and make good learning from the problem.
If anyone has similar problems, GitHub support confirmed that disabling the workflows solves the "infinite loop" issue. Not the best Sunday after Xmas I could imagine. J. On Sun, Dec 27, 2020 at 5:03 PM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > FYI. Following Github Advise - I disabled the workflows manually and seems > that it stopped the infinite loop. > > Anyone else having a similar issue should do the same I think. > > J > > > On Sun, Dec 27, 2020 at 4:32 PM Jarek Potiuk <jarek.pot...@polidea.com> > wrote: > >> I updated the >> https://issues.apache.org/jira/projects/INFRA/issues/INFRA-21234 but I >> think we might have even bigger problem. >> >> I think disabling my account did not help. >> >> It looks like the infinite loop is still running ... In Airflow we >> continue having 30K failed jobs/hour... >> >> I think you will have to enable it back for a short while or work with >> the GitHub Support to fix it - I raised the issue to GitHub support, but >> since it is Sunday after Xmas - I think the easiest way to fix the problem >> is to disable the policy for a moment so that the infinite loop is stopped >> (hopefully). >> >> >> >> On Sun, Dec 27, 2020 at 4:09 PM Jarek Potiuk <jarek.pot...@polidea.com> >> wrote: >> >>> I think we have far bigger problem now. >>> >>> Seems that the change triggered an infinite loop of some kind and my >>> user now is disabled from Github Actions. I pushed a change just before the >>> policy change went live. I guess other people might have similar pro >>> >>> J. >>> >>> >>> Hello Jarek, >>> >>> I work on the Support team at GitHub, and we have some concerns about >>> your Actions runs on the apache/airflow repository. >>> >>> We've noticed that there are around 30k failed Actions runs in the last >>> hour for your potiuk user, and this has been ongoing for a few hours >>> now. All of those runs are in the apache/airflow repository and can be >>> seen on this page: >>> >>> https://github.com/apache/airflow/actions. >>> >>> It looks like you may have triggered an infinite loop of some kind. As a >>> result, the repository is consuming a non-trivial amount of resources and >>> has the potential to negatively affect the GitHub service for other users. >>> >>> To reduce the number of workflows being run, we've had to temporarily >>> disable Actions for the potiuk user. We'd ask that you please take a >>> look at what may have triggered these runs, and once it's resolved we'll be >>> happy to work with you to turn Actions back on. >>> >>> Please let us know if you have any questions or concerns. >>> >>> Thanks, >>> GitHub Support >>> >>> Thanks, >>> >>> Tammy >>> GitHub Support >>> >>> -- >>> >>> Jarek Potiuk >>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>> >>> M: +48 660 796 129 <+48660796129> >>> [image: Polidea] <https://www.polidea.com/> >>> >>> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
