On Thu, Nov 7, 2019 at 1:18 AM Harald Sitter <sit...@kde.org> wrote: > > On Wed, Nov 6, 2019 at 8:07 AM Ben Cooksley <bcooks...@kde.org> wrote: > > > > On Tue, Nov 5, 2019 at 11:50 PM Harald Sitter <sit...@kde.org> wrote: > > > > > > I get where you are coming from, but honestly, none of this makes > > > pushing unreviewed commits that disable the entire test collection of > > > an entire framework any more correct. If it had only affected the one > > > test I wouldn't mind so much, but since it hasn't it only goes to > > > proof that we have reviews for a reason. In particular for repos that > > > are part of frameworks where we rely on tests to tell us if the > > > frameworks are good for the monthly release. > > > > Unfortunately reviews require responsive developers. > > > > In this instance, both Frameworks have been found to have developers > > which are not responsive (because my previous requests have been > > ignored), so sending a patch for review would have been pointless - > > because there would have been nobody to review it. > > > > Therefore bypassing review and taking any necessary action against the > > offending code (even if that does happen to be all of the tests) to > > correct the situation is the only viable option forward. > > This is really not true. > We got the workaround for KCrash landed in not even half a business > day - including review. >
Unfortunately it is certainly my perception - which is based off of my emails on the subject not receiving any response. This experience is one that hasn't only happened with Frameworks - it has also happened (several times) with PIM. Taking direct action against a project is pretty much guaranteed to trigger a fairly rapid response (which is exactly what happened in this case). It should be noted that saying i'm going to take action usually results in either no response, or a response along the lines of "no, you can't do that" rather than the issue actually being addressed (which once again, is what happened in this case too) > > > There are many shades of grey between sending a "someone please fix" > > > mail to a mailing list and the nuclear option you implemented. The > > > most notable one being that you can ask someone who worked on the > > > repo, or tests recently "Hey, X, can you please disable the test on > > > windows because its impairing CI?" to which the answer is probably yes > > > because you'd not only address a very specific person but also the > > > task is doable in less than 5 minutes. > > > I understand that there's an element of cat herding in this, but > > > quality must matter for our products, and the quality of frameworks is > > > very tightly linked to autotesting and reviews because of how we > > > release them. > > > > This would require spending more time on the issue, which is something > > i'd very much rather avoid. > > > > It is expensive enough having to login to the CI builders to clear out > > these jobs when they get stuck (looking at anything that uses Akonadi > > especially here, along with these two Frameworks) and then asking the > > project lists to please fix their faulty code (which is then ignored, > > indicating that the concept of community maintained does not really > > work). > > > > Having to then lookup someone and then forward it on to them requires > > yet more time, with no guarantee that it will work - especially > > because some contributors are still hostile to any platform that is > > not FOSS and outright refuse to do anything to help those platforms. > > > > In the event that it does not work - then what would I do? Pick on the > > next person in the list (requiring yet more time)? > > > > Sorry, but that is simply too expensive, and if that is the policy > > you'd like to run with, then i'll stick to stripping projects of their > > test execution privileges outright across all platforms in the event > > they cause issues like this (which if ECM is anything to go by, is > > something people won't even notice even if the commit that puts that > > in effect is CC'ed to the project mailing list, which means that > > nobody will notice when the tests break because the CI system won't be > > running them) > > I did not express myself well enough. > What I'd like you to do is talk to someone, not anyone, a very > specific someone of your choosing, to take care of the issue for you. > IOW: The sysadmins should delegate to a developer if the sysadmins > find themselves unable to accurately deal with a code problem (such as > a failing test that needs deactivating). If you tell an actually > relevant developer that's obviously better, if you don't have time to > look at the git log then that's fine too. > > There are two different work items to differentiate here: > > 1. fixing something > 2. disabling something because a fix was not made in an acceptable time frame > > 1 likely requires domain-knowledge of the specific framework and > probably quit a bit of time. > 2 would generally only require understanding of cmake and a fairly > small time investment. > > Pretty much any dev can do task 2. You just need to make someone care, > and the easiest way is to talk to a buddy directly and have them have > a look. Like if you ask me to deal with this for sysadmins I'll > definitely do. So, you have my permission to ask me to disable tests > on your behalf ;) > > In short the process I would like is: > > 1. mail to list "yo, this framework has broken test xyz that is > severely impairing CI. plz fix" > 2. if nothing happens ask someone (e.g. me) to force an action on the > repo (i.e. disable a specific test) > 3. if the someone starts ghosting you you can still go in and do what > you did the other day > > If you do 2 that has little overhead and a good chance of netting you > less work, not more. That could potentially work (I will note though that it is basically just shifting the problem - the problem being #1 not resulting in a response). I do have some concerns about the long term scalability of the approach though. > > HS Cheers, Ben
