Reminder to all that the builds@ meeting will take place this week, in 2 days.
Hope you can all make it! Gav... On Thu, Nov 4, 2021 at 8:58 AM Gavin McDonald <gmcdon...@apache.org> wrote: > Hi All. > > It seems next week is better for some so I'll push this out to the 11th. > > HTH > > On Mon, Nov 1, 2021 at 4:08 PM Josh Fischer <j...@joshfischer.io> wrote: > >> I'm interested in participating. The Heron project is hitting limits >> with TravisCI, I'd like to find a way to make our process better whether >> it's with Jenkins/GA or something else. Next week would be better for me, >> if we can push it back. >> >> On Mon, Nov 1, 2021 at 8:49 AM Jarek Potiuk <ja...@potiuk.com> wrote: >> >>> I updated the graphs. I think things have changed a lot since December >>> last >>> year. >>> >>> There are no absurd-high (and way above the limits) peaks of workflows in >>> progress for some projects (e.g. Pulsar). However a number of apache >>> repos >>> using GA grows steadily. Number of workflows "queued" fluctuates for many >>> projects and for some projects (spark) it seems it steadily grows. >>> >>> Observation from the Airflow side (and I think lack of "escalation" >>> indicates that other projects have the same) is that we stopped observing >>> prolonged periods where a lot of jobs are queued for a long time (many >>> hours). One thing is that we moved a lot of the workload to our >>> self-hosted >>> runners - when it comes to public runners, there are short peaks of >>> queued >>> jobs but they seem to catch-up relatively quickly (max few hours). >>> >>> When I see those graph, my interpretation is that Github got a lot better >>> queuing algorithms.Previously it seemed to be possible by a single >>> project >>> to take over the queue and get the number of "in progress" way higher >>> than >>> the 150 jobs limit, and I think that effectively blocked other projects >>> for >>> a long time. Currently it seems that the algorithm they have balances out >>> the workload between projects much better - and if there are projects >>> that >>> have more and more workloads, it impacts their own queue size much more >>> than queuing time of other projects - see Spark's average number of >>> queued >>> workflows. >>> >>> The graphs IMHO are much healthier than they were a year ago. >>> >>> I wonder what is the perception of others :). >>> >>> BTW. It would be great if we could move it by a week - this week is >>> shorter >>> for many and some have longer holidays (1st of Nov is a holiday in a >>> number >>> of countries). >>> >>> On Mon, Nov 1, 2021 at 2:13 PM Gavin McDonald <gmcdon...@apache.org> >>> wrote: >>> >>> > Hi All, >>> > >>> > I think it is past time to have another meeting. >>> > The stresses and strains of our current CI makes me think we should >>> have >>> > another discussion. >>> > >>> > Link to the cwiki page: >>> > >>> > >>> https://cwiki.apache.org/confluence/display/INFRA/Builds+Agenda+2021-11-04 >>> > >>> > If this week doesn't work then we can postpone until next week. >>> > >>> > HTH >>> > >>> > -- >>> > >>> > *Gavin McDonald* >>> > Systems Administrator >>> > ASF Infrastructure Team >>> > >>> >> > > -- > > *Gavin McDonald* > Systems Administrator > ASF Infrastructure Team > -- *Gavin McDonald* Systems Administrator ASF Infrastructure Team
