On Fri, Jan 8, 2021, 12:08 PM Jarek Potiuk <jarek.pot...@polidea.com> wrote:
> > There is one problem with the charts, They are flawed. They show > 'workflows' not 'jobs' and one workflow might mean many jobs :(. For > example the big number of workflows you can see in Airflow yesterday come > from "Label when reviewed" workflows - each of which has 1 job that > takes 10 seconds or so. One workflow can be 20/30 times more important > than another. > > We cannot easily drill down to jobs, because we are using Github API to get > the information, but there are limits (max num requests/hr) and we are > already close to hitting it with the current setup. > > Going to the jobs level would mean 20x more API requests. This is the 2nd > thing where INFRA <> GitHub relation I believe there was the option that > GitHub provides some better and more reliable stats to analyse. > > J. > We should be able to make an efficient query via GraphQL API right? I found the REST API for actions to be a little underwhelming. I was actually a little surprised to not see us (NuttX) not on the offender list. I did open a ticket for us to look into how we can play better using the auto cancel action that Airflow is using, that would make our own lives easier. We have tried to make our builds faster with more caching but it's not easy since it's an embedded systems project we need to target a lot of configurations and most changes impact all builds. We too would like to would like to take advantage of our own runners but more for the ability to do Hardware In the Loop testing but have avoided it for the reasons already mentioned. --Brennan
