Hi All. It seems next week is better for some so I'll push this out to the 11th.
HTH On Mon, Nov 1, 2021 at 4:08 PM Josh Fischer <j...@joshfischer.io> wrote: > I'm interested in participating. The Heron project is hitting limits with > TravisCI, I'd like to find a way to make our process better whether > it's with Jenkins/GA or something else. Next week would be better for me, > if we can push it back. > > On Mon, Nov 1, 2021 at 8:49 AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> I updated the graphs. I think things have changed a lot since December >> last >> year. >> >> There are no absurd-high (and way above the limits) peaks of workflows in >> progress for some projects (e.g. Pulsar). However a number of apache repos >> using GA grows steadily. Number of workflows "queued" fluctuates for many >> projects and for some projects (spark) it seems it steadily grows. >> >> Observation from the Airflow side (and I think lack of "escalation" >> indicates that other projects have the same) is that we stopped observing >> prolonged periods where a lot of jobs are queued for a long time (many >> hours). One thing is that we moved a lot of the workload to our >> self-hosted >> runners - when it comes to public runners, there are short peaks of queued >> jobs but they seem to catch-up relatively quickly (max few hours). >> >> When I see those graph, my interpretation is that Github got a lot better >> queuing algorithms.Previously it seemed to be possible by a single project >> to take over the queue and get the number of "in progress" way higher than >> the 150 jobs limit, and I think that effectively blocked other projects >> for >> a long time. Currently it seems that the algorithm they have balances out >> the workload between projects much better - and if there are projects that >> have more and more workloads, it impacts their own queue size much more >> than queuing time of other projects - see Spark's average number of queued >> workflows. >> >> The graphs IMHO are much healthier than they were a year ago. >> >> I wonder what is the perception of others :). >> >> BTW. It would be great if we could move it by a week - this week is >> shorter >> for many and some have longer holidays (1st of Nov is a holiday in a >> number >> of countries). >> >> On Mon, Nov 1, 2021 at 2:13 PM Gavin McDonald <gmcdon...@apache.org> >> wrote: >> >> > Hi All, >> > >> > I think it is past time to have another meeting. >> > The stresses and strains of our current CI makes me think we should have >> > another discussion. >> > >> > Link to the cwiki page: >> > >> > >> https://cwiki.apache.org/confluence/display/INFRA/Builds+Agenda+2021-11-04 >> > >> > If this week doesn't work then we can postpone until next week. >> > >> > HTH >> > >> > -- >> > >> > *Gavin McDonald* >> > Systems Administrator >> > ASF Infrastructure Team >> > >> > -- *Gavin McDonald* Systems Administrator ASF Infrastructure Team
