On Fri, 2025-02-14 at 10:34 +0100, Clement Verna wrote: > On Thu, 13 Feb 2025 at 17:44, Kevin Fenzi <ke...@scrye.com> wrote: > > > I agree with downthread folks that that seems like way too high a > > failure rate to enable gating on. However, a few questions if I can: > > > > Yes the failure rate is quite high and most of these are real failures, > that we deal with in Fedora CoreOS. So I am reading this like, because the > tests are catching too many failures we should continue ignoring them 🫤 > > > > > > Is this reporting to bodhi for all these components? > > Off hand checking a few I don't see any results from this? > > Wouldn't a good first step be to enable (non gating) these to show up > > there so maintainers/you can be more aware and help reduce problems? > > > > In practice, unfortunately if tests are not gating they are most of the > time ignored, even more in rawhide where the update is automatically > created and pushed.
Yeah, this is absolutely a real thing. When we had openQA tests running on updates but not gating, I was spending *considerable* chunks of time running around after the fact trying to get the bugs fixed. Gating on them is a way nicer experience all round: tests don't fail on other updates, users don't get broken packages, maintainers can take more time to fix the bugs. If the failure rate is not "false failures" but "real and significant bugs that we could catch and fix", then it should be a *positive* factor in deciding to gate on these tests, IMO. -- Adam Williamson (he/him/his) Fedora QA Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org https://www.happyassassin.net -- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
