Re: CI Utilisation and system efficiency

2025-05-07 Thread Milian Wolff
On Dienstag, 6. Mai 2025 13:05:55 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > On Tue, May 6, 2025 at 9:35 PM Milian Wolff wrote: > > On Mittwoch, 30. April 2025 21:07:35 Mitteleuropäische Sommerzeit Ben > > Cooksley > > > > wrote: > > > On Thu, May 1, 2025 at 3:42 AM Milian Wolff wrote:

Re: CI Utilisation and system efficiency

2025-05-07 Thread Johnny Jazeix
Hi, in the .kde-ci.yml: https://lxr.kde.org/search?%21v=kf6-qt6&_filestring=&_string=use-ccache. Cheers, Johnny Le mer. 7 mai 2025 à 14:20, Milian Wolff a écrit : > > On Dienstag, 6. Mai 2025 13:05:55 Mitteleuropäische Sommerzeit Ben Cooksley > wrote: > > On Tue, May 6, 2025 at 9:35 PM Milian W

Re: CI Utilisation and system efficiency

2025-05-06 Thread Volker Krause
On Dienstag, 6. Mai 2025 11:26:30 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > On Tue, May 6, 2025 at 5:36 AM Volker Krause wrote: > > On Freitag, 2. Mai 2025 22:55:34 Mitteleuropäische Sommerzeit Ben Cooksley > > > > wrote: > > > On Sat, May 3, 2025 at 8:26 AM Christoph Cullmann > > > >

Re: CI Utilisation and system efficiency

2025-05-06 Thread Ben Cooksley
On Tue, May 6, 2025 at 9:35 PM Milian Wolff wrote: > On Mittwoch, 30. April 2025 21:07:35 Mitteleuropäische Sommerzeit Ben > Cooksley > wrote: > > On Thu, May 1, 2025 at 3:42 AM Milian Wolff wrote: > > > On Freitag, 18. April 2025 21:25:36 Mitteleuropäische Sommerzeit Ben > > > Cooksley > > > >

Re: CI Utilisation and system efficiency

2025-05-06 Thread Milian Wolff
On Mittwoch, 30. April 2025 21:07:35 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > On Thu, May 1, 2025 at 3:42 AM Milian Wolff wrote: > > On Freitag, 18. April 2025 21:25:36 Mitteleuropäische Sommerzeit Ben > > Cooksley > > > > wrote: > > > Hi all, > > > > > > Over the past week or two the

Re: CI Utilisation and system efficiency

2025-05-06 Thread Ben Cooksley
On Tue, May 6, 2025 at 5:36 AM Volker Krause wrote: > On Freitag, 2. Mai 2025 22:55:34 Mitteleuropäische Sommerzeit Ben Cooksley > wrote: > > On Sat, May 3, 2025 at 8:26 AM Christoph Cullmann > > > > > wrote: > > > On Friday, May 2nd, 2025 at 22:18, Ben Cooksley > wrote: > > > > > > On Sat, May

Re: CI Utilisation and system efficiency

2025-05-05 Thread Volker Krause
On Freitag, 2. Mai 2025 22:55:34 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > On Sat, May 3, 2025 at 8:26 AM Christoph Cullmann > > wrote: > > On Friday, May 2nd, 2025 at 22:18, Ben Cooksley wrote: > > > > On Sat, May 3, 2025 at 3:27 AM Christoph Cullmann > > > > wrote: > >> Hi, > >>

Re: CI Utilisation and system efficiency

2025-05-02 Thread Ben Cooksley
On Sat, May 3, 2025 at 8:26 AM Christoph Cullmann wrote: > > > On Friday, May 2nd, 2025 at 22:18, Ben Cooksley wrote: > > On Sat, May 3, 2025 at 3:27 AM Christoph Cullmann > wrote: > >> >> Hi, >> >> at work we use cmake unity build to save time & costs. >> >> Would that be some idea here, too?

Re: CI Utilisation and system efficiency

2025-05-02 Thread Christoph Cullmann
On Friday, May 2nd, 2025 at 22:18, Ben Cooksley wrote: > On Sat, May 3, 2025 at 3:27 AM Christoph Cullmann > wrote: > > > > > > > Hi, > > > > at work we use cmake unity build to save time & costs. > > > > Would that be some idea here, too? > > > > Naturally as side effect that ca

Re: CI Utilisation and system efficiency

2025-05-02 Thread Ben Cooksley
On Sat, May 3, 2025 at 3:27 AM Christoph Cullmann wrote: > > Hi, > > at work we use cmake unity build to save time & costs. > > Would that be some idea here, too? > > Naturally as side effect that can hide compile issues > or introduce ones. > We could look into that, just not sure how it would

Re: CI Utilisation and system efficiency

2025-05-02 Thread Christoph Cullmann
Hi, at work we use cmake unity build to save time & costs. Would that be some idea here, too? Naturally as side effect that can hide compile issues or introduce ones. Greetings Christoph On Friday, April 18th, 2025 at 21:25, Ben Cooksley wrote: > Hi all, > Over the past week or two there h

Re: CI Utilisation and system efficiency

2025-04-30 Thread Ben Cooksley
On Thu, May 1, 2025 at 3:42 AM Milian Wolff wrote: > On Freitag, 18. April 2025 21:25:36 Mitteleuropäische Sommerzeit Ben > Cooksley > wrote: > > Hi all, > > > > Over the past week or two there have been a number of complaints > regarding > > CI builder availability which i've done some investiga

Re: CI Utilisation and system efficiency

2025-04-30 Thread Milian Wolff
On Freitag, 18. April 2025 21:25:36 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > Hi all, > > Over the past week or two there have been a number of complaints regarding > CI builder availability which i've done some investigating into this > morning. > Plasma Workspace/Desktop: At least i

Re: CI Utilisation and system efficiency

2025-04-23 Thread Dmitry Kazakov
>> Another point that requires extra build time for Krita is an inappropriate timeout on 100 minutes. A lot of our windows builds are terminated at around 95% completion because of this timeout, so we have to rerun them and, effectively, consume more and more CI time. >Have you got a list of these

Re: CI Utilisation and system efficiency

2025-04-23 Thread Dmitry Kazakov
> This is probably at least in part due to Windows on Docker having extremely poor file system performance even vs. straight NTFS (which isn't great to begin with). Well, Krita builds on a normal Windows system are also quite slow (at least much slower than on Linux). Though they are still much fa

Re: CI Utilisation and system efficiency

2025-04-22 Thread Maciej Jesionowski
Yes, that is a full clean build after the build directory is configured from scratch. The dependencies are prepared before that, and this is just the time to run ninja which lists 4900 build steps or so (I can't remember). I mean, the build isn't exactly very fast even on a reasonably high-end syst

Re: CI Utilisation and system efficiency

2025-04-22 Thread Ben Cooksley
On Tue, Apr 22, 2025 at 10:53 PM Maciej Jesionowski wrote: > Yes, that is a full clean build after the build directory is configured > from scratch. The dependencies are prepared before that, and this is just > the time to run ninja which lists 4900 build steps or so (I can't > remember). I mean,

Re: CI Utilisation and system efficiency

2025-04-22 Thread Ben Cooksley
On Tue, Apr 22, 2025 at 10:44 PM Konstantin Kharlamov wrote: > On Tue, 2025-04-22 at 07:15 +1200, Ben Cooksley wrote: > > On Tue, Apr 22, 2025 at 5:57 AM Dmitry Kazakov > > wrote: > > > Hi, Ben! > > > > > > > > > Hey Dmitry, > > > > > > > > As for Krita, most of CI time is spent on the Windows p

Re: CI Utilisation and system efficiency

2025-04-22 Thread Konstantin Kharlamov
On Tue, 2025-04-22 at 07:15 +1200, Ben Cooksley wrote: > On Tue, Apr 22, 2025 at 5:57 AM Dmitry Kazakov > wrote: > > Hi, Ben! > > > > > Hey Dmitry, >   > > > > As for Krita, most of CI time is spent on the Windows pipeline, > > which build extremely slowly due to done obscure filesystem issues

Re: CI Utilisation and system efficiency

2025-04-22 Thread Ben Cooksley
On Tue, Apr 22, 2025 at 9:53 AM Maciej Jesionowski wrote: > Hi, > Hi Maciej, > Are these servers running multiple builds at a time, or is a Windows build > given the full host resources, i.e. 8c/16t and 64GB of RAM? If you can > monitor the resources in real time, it would be interesting to co

Re: CI Utilisation and system efficiency

2025-04-22 Thread Maciej Jesionowski
Hi, Are these servers running multiple builds at a time, or is a Windows build given the full host resources, i.e. 8c/16t and 64GB of RAM? If you can monitor the resources in real time, it would be interesting to confirm if indeed the CPU utilization is significantly lower than 100%, meaning someth

Re: CI Utilisation and system efficiency

2025-04-22 Thread Dmitry Kazakov
Hi, Ben! As for Krita, most of CI time is spent on the Windows pipeline, which build extremely slowly due to done obscure filesystem issues (searching includes is extremely slow). I personally don't know how to fix it. I tried: 1) PCH builds, 2) relative includes, 3) split debug info (dwo). The on

Re: CI Utilisation and system efficiency

2025-04-21 Thread Ben Cooksley
On Tue, Apr 22, 2025 at 5:57 AM Dmitry Kazakov wrote: > Hi, Ben! > Hey Dmitry, > > As for Krita, most of CI time is spent on the Windows pipeline, which > build extremely slowly due to done obscure filesystem issues (searching > includes is extremely slow). I personally don't know how to fix i

Re: CI Utilisation and system efficiency

2025-04-20 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 8:26 AM Ben Cooksley wrote: > On Sun, Apr 20, 2025 at 1:50 AM Linus Jahn wrote: > >> On Sat, 19 Apr 2025 13:23:59 +0100 >> David Edmundson wrote: >> >> > >Realistically merge requests shouldn't be proposed until people are >> > >ready to get something reviewed and merged

Re: CI Utilisation and system efficiency

2025-04-20 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 9:10 PM Albert Astals Cid wrote: > El divendres, 18 d’abril del 2025, a les 21:25:36 (Hora d’estiu d’Europa > central), Ben Cooksley va escriure: > > Hi all, > > > > Over the past week or two there have been a number of complaints > regarding > > CI builder availability wh

Re: CI Utilisation and system efficiency

2025-04-20 Thread Albert Astals Cid
El divendres, 18 d’abril del 2025, a les 21:25:36 (Hora d’estiu d’Europa central), Ben Cooksley va escriure: > Hi all, > > Over the past week or two there have been a number of complaints regarding > CI builder availability which i've done some investigating into this > morning. > > Part of this

Re: CI Utilisation and system efficiency

2025-04-20 Thread Albert Astals Cid
El dissabte, 19 d’abril del 2025, a les 10:47:13 (Hora d’estiu d’Europa central), Ben Cooksley va escriure: > On Sat, Apr 19, 2025 at 7:25 AM Ben Cooksley wrote: > > Hi all, > > > > Over the past week or two there have been a number of complaints regarding > > CI builder availability which i've

Re: CI Utilisation and system efficiency

2025-04-19 Thread Neal Gompa
On Fri, Apr 18, 2025 at 3:26 PM Ben Cooksley wrote: > > > When reviewing the list of CI builds projects have enabled, it is important > to consider to what degree your project benefits from having various builds > enabled. One common pattern i've seen is having Alpine, SUSE Qt 6.9 and SUSE > Qt

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ingo Klöcker
On Samstag, 19. April 2025 21:29:07 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > On Sun, Apr 20, 2025 at 7:21 AM Ingo Klöcker wrote: > > Sounds like we shouldn't be so trigger-happy clicking the Rebase button on > > our > > MRs when it's not really necessary. We could rebase locally and the

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 3:05 AM Thomas Friedrichsmeier < thomas.friedrichsme...@kdemail.net> wrote: > On Sat, 19 Apr 2025 07:25:36 +1200 > Ben Cooksley wrote: > [...] > > This morning I ran a basic query to ascertain the top 20 projects for > > CI time utilisation on invent.kde.org which revealed

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 1:50 AM Linus Jahn wrote: > On Sat, 19 Apr 2025 13:23:59 +0100 > David Edmundson wrote: > > > >Realistically merge requests shouldn't be proposed until people are > > >ready to get something reviewed and merged in... > > > > That's not realistic with how it is on the fron

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 12:24 AM David Edmundson wrote: > >Realistically merge requests shouldn't be proposed until people are ready > to get something reviewed and merged in... > > That's not realistic with how it is on the frontlines. The earlier we > share feedback the better, and reviews can

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 3:05 AM Thomas Friedrichsmeier < thomas.friedrichsme...@kdemail.net> wrote: > On Sat, 19 Apr 2025 07:25:36 +1200 > Ben Cooksley wrote: > [...] > > This morning I ran a basic query to ascertain the top 20 projects for > > CI time utilisation on invent.kde.org which revealed

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ingo Klöcker
On Samstag, 19. April 2025 10:33:45 Mitteleuropäische Sommerzeit Ben Cooksley wrote: > A big part of the issue here is our rebase centric workflow - normally with > Gitlab you use merge commits and in this workflow CI only runs when the MR > changes. > With a rebase workflow like ours though, othe

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sun, Apr 20, 2025 at 7:21 AM Ingo Klöcker wrote: > On Samstag, 19. April 2025 10:33:45 Mitteleuropäische Sommerzeit Ben > Cooksley > wrote: > > A big part of the issue here is our rebase centric workflow - normally > with > > Gitlab you use merge commits and in this workflow CI only runs when

Re: CI Utilisation and system efficiency

2025-04-19 Thread Thomas Friedrichsmeier
On Sat, 19 Apr 2025 07:25:36 +1200 Ben Cooksley wrote: [...] > This morning I ran a basic query to ascertain the top 20 projects for > CI time utilisation on invent.kde.org which revealed the following: What (rough) percentage is that of the total? I.e. is the problem more one of a few projects h

Re: CI Utilisation and system efficiency

2025-04-19 Thread Linus Jahn
On Sat, 19 Apr 2025 13:23:59 +0100 David Edmundson wrote: > >Realistically merge requests shouldn't be proposed until people are > >ready to get something reviewed and merged in... > > That's not realistic with how it is on the frontlines. The earlier we > share feedback the better, and review

Re: CI Utilisation and system efficiency

2025-04-19 Thread David Edmundson
>Realistically merge requests shouldn't be proposed until people are ready to >get something reviewed and merged in... That's not realistic with how it is on the frontlines. The earlier we share feedback the better, and reviews can take many many iterations. There's a "CI_MERGE_REQUEST_APPROVED"

Re: CI Utilisation and system efficiency

2025-04-19 Thread Gleb Popov
On Sat, Apr 19, 2025 at 11:47 AM Ben Cooksley wrote: > > - FreeBSD Qt 6.9 monitors buildability on BSD libc + Clang + Qt 6.9 FreeBSD is at Qt 6.8 at the moment. We're close to updating to 6.9 though.

Re: CI Utilisation and system efficiency

2025-04-19 Thread Konstantin Kharlamov
On Sat, 2025-04-19 at 20:33 +1200, Ben Cooksley wrote: > On Sat, Apr 19, 2025 at 8:07 PM Vlad Zahorodnii > wrote: > > On 4/19/25 10:53 AM, Kai Uwe Broulik wrote: > > > Hi, > > > > > > thanks for looking into this. > > > > > > It seems a major contributor to CI time is the fact that a merge > > >

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sat, Apr 19, 2025 at 7:25 AM Ben Cooksley wrote: > Hi all, > > Over the past week or two there have been a number of complaints regarding > CI builder availability which i've done some investigating into this > morning. > > Part of this is related to the Windows CI builders falling offline due

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sat, Apr 19, 2025 at 8:05 PM laurent Montel wrote: > Le vendredi 18 avril 2025, 21:25:36 heure d’été d’Europe centrale Ben > Cooksley a écrit : > > Hi all, > > > Ruqola: Appears to be conducting a development process whereby changes > are > > made in stable then immediately merged to master in

Re: CI Utilisation and system efficiency

2025-04-19 Thread Ben Cooksley
On Sat, Apr 19, 2025 at 8:07 PM Vlad Zahorodnii wrote: > On 4/19/25 10:53 AM, Kai Uwe Broulik wrote: > > Hi, > > > > thanks for looking into this. > > > > It seems a major contributor to CI time is the fact that a merge > > request runs and once merged, master runs another pretty much > > identic

Re: CI Utilisation and system efficiency

2025-04-19 Thread Vlad Zahorodnii
On 4/19/25 10:53 AM, Kai Uwe Broulik wrote: Hi, thanks for looking into this. It seems a major contributor to CI time is the fact that a merge request runs and once merged, master runs another pretty much identical pipeline right after that. As for KWin: Not sure which one we could axe. KWi

Re: CI Utilisation and system efficiency

2025-04-19 Thread laurent Montel
Le vendredi 18 avril 2025, 21:25:36 heure d’été d’Europe centrale Ben Cooksley a écrit : > Hi all, > Ruqola: Appears to be conducting a development process whereby changes are > made in stable then immediately merged to master in a ever continuing loop. > Please discontinue this behaviour and onl