I created a INFRA ticket requesting the usage report for GitHub Actions: https://issues.apache.org/jira/browse/INFRA-23633 This information should be available at https://github.com/organizations/apache/settings/billing for those who have access.
The reason for this request is to see if we have reached a quota for the apache org in GitHub. We are experiencing an issue in apache/pulsar that builds aren't starting or a very low number of builds can run at a time. This has stalled our ability to process PRs. I wonder if other projects are experiencing such issues? BR, Lari Hotari Apache Pulsar PMC member On 2022/08/16 16:15:13 Jarek Potiuk wrote: > As an original author - not much has changed since. > > I am regularly following what Github Action released to see if the > problems we raised to them were addressed or not. All the updates > regarding the security tightening are updated in the wiki page (not > much changed - there are a number of watchouts and if you follow them > carefully you can have fairly secure environment - but still > self-hosted runners's security problem for Public repos is not > entirely solved (it's mitigated in parts but still there are workflows > where bots might mine bitcoin using your infrastructure by creating > PRs or take over your Self-Hosted environment if you are not careful > enough. That is the root cause of the problem for using Public Runners > as even if you have your credits/sponsors for infrastructure, it's > hard to deploy a secure self-hosted runner. We've done that in Airflow > but it's difficult to replicate due to our design choices. > > One thing not mentioned there which I am aware of is that the Apache > Beam team with some Apache Airflow team involvement are working on a > reusable infrastructure for running self-hosted runners on GKE/K8S in > a secure way that should be easier to replicate. But it will still > take some time to conclude (and vacation period is I think not > helping). > > I do not know if there was any progress in changes of our GitHub > involvement in terms of numbers of Public runners available (maybe the > INFRA team/Gavin can provide more info). But we build a small > "tracking" of the usage in Airflow - not perfect, in some cases it is > just an indication rather than "very accurate" data but it shows some stats. > We keep track of those continuously and I refresh the data from time > to time (and I know Infra was looking into building own "proper" > dashboard based on it): > > Here are stats from today: > > * Growth in the number of ASF projects using GA : > https://pasteboard.co/Ms3YqkTj3SKf.png > * Biggest "users" of GA (in terms of number of workflows): May - > August 2022: https://pasteboard.co/vC9alxKqx1xd.png > * Usage per projects ("Average workflows in progress): > https://pasteboard.co/U8WrhVb19nnI.png > > Quick look indicates that July/August we again started to get more > fluctuations as some projects originated some spikes in a number of > workflows used. > > That's all the information I can add to it - from the Apache Airflow > side at least. We are mostly in monitoring mode (infrequently) as for > us most of the problems with GA were solved by introducing our own > (paid by our sponsors) self-hosted runners, so we are not really > affected too much. Seems that also GA became much better in "fair" > distribution of workflows among multiple projects which means that > when things are slow - everyone is affected and it's difficult to > abuse it by a single project that starts many workflows (though this > is mostly anecdotal evidence - the problems the "unfairness" caused ~ > 2 years ago was really very well visible and at least there are no > indicators of that returning (or maybe I have not seen that). > > J. > > On Tue, Aug 16, 2022 at 4:57 PM Slawomir Jaranowski > <s.jaranow...@gmail.com> wrote: > > > > Thanks Richard, > > > > Mentioned page contains data which is one year old ... > > Can anyone update it? How does it look today? > > > > wt., 16 sie 2022 o 16:36 Zowalla, Richard <richard.zowa...@hs-heilbronn.de> > > napisał(a): > > > > > Hi, > > > > > > https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status > > > > > > contains some performance details + background infos. > > > > > > Gruß > > > Richard > > > > > > ________________________________ > > > Von: Slawomir Jaranowski <s.jaranow...@gmail.com> > > > Gesendet: Dienstag, 16. August 2022 15:56:44 > > > An: builds@apache.org > > > Betreff: GitHub Actions runners > > > > > > Hi, > > > > > > Do we monitoring usage of GitHub action runners? > > > How many projects use GitHub actions? > > > How runners are utilized by each ASF project? > > > > > > Last time I see that Maven builds spent more time waiting for free runners > > > and builds took about 1 hour instead of a couple of minutes. > > > > > > Eg on https://github.com/apache/maven-remote-resources-plugin/actions > > > > > > Each job takes about 2 - 4 minutes but to complete the whole workflow we > > > need about 1 hour. > > > > > > If more projects will use GitHub actions it can be a problem. > > > > > > > > > -- > > > Sławomir Jaranowski > > > > > > > > > -- > > Sławomir Jaranowski >
