On Tue, Mar 3, 2020 at 2:44 PM Kamil Paral <kpa...@redhat.com> wrote:
> On Mon, Feb 17, 2020 at 3:20 PM Kamil Paral <kpa...@redhat.com> wrote: > >> This proposal intends to reduce the scope of the “*Default application >> functionality*” release criterion [0]: >> >> All applications that can be launched using the standard graphical >>> mechanism of a release-blocking desktop after a default installation of >>> that desktop must start successfully and withstand a basic functionality >>> test. >>> >> >> *= Background =* >> >> The area which QA is responsible for has been growing and growing in >> recent years. We used to have just a single Fedora release with two >> blocking desktops (GNOME and KDE). Then Editions got introduced and we >> started testing Workstation, Server, Cloud and the KDE Spin. Additional >> architectures were introduced (fortunately i386 got obsolete) and we >> started testing and blocking on specific images on armhfp and aarch64. >> Currently there are 13 release blocking deliverables mentioned on the >> ReleaseBlocking page [1]. That list is not complete, though. Fedora CoreOS >> became an official edition lately, and even though its release cycle is not >> tied to traditional Fedora release cycle (and that’s why it’s not mentioned >> on that wiki page), as an official edition QA should care about it as well. >> It is the same story with Fedora IoT, another recent release-blocking >> addition [2]. Desktop testing is one of the most time-consuming jobs with >> the most frequent bug occurrence. Right now, we’re supposed to be fully >> testing and blocking on GNOME, KDE and XFCE (although XFCE might get >> dropped in favor or aarch64 Workstation [3]). We cannot test all of this >> and honestly claim that we verified basic functionality of all the included >> apps on all these desktops. I believe we need to adjust the criterion and >> align it closer with reality. In my eyes, it’s better to have a narrower >> scope and perform it well than having a large scope and perform it poorly. >> >> *= Proposal =* >> >> Change the criterion to something along these lines: >> >> All applications that can be launched using the standard graphical >>> mechanism after a default installation of Fedora Workstation on x86_64 >>> architecture must start successfully and withstand a basic functionality >>> test. >>> >>> For other release-blocking desktops (on any architecture), the >>> requirements only apply to the following types of applications: >>> >>> * web browser (e.g. firefox) [4] >>> >>> * file browser (e.g. nautilus) >>> >>> * package manager (e.g. gnome-software) >>> >>> * image viewer (e.g. eog) >>> >>> * document viewer (e.g. evince) >>> >>> * text editor (e.g. gedit) >>> >>> * archive manager (e.g. file-roller) >>> >>> * terminal emulator (e.g. gnome-terminal) [4] >>> >>> * problem reporter (e.g. abrt) >>> >>> If there are multiple applications of the same type (e.g. several web >>> browsers), only one of them needs to satisfy the requirements. >>> >> >> As you can see, the original criterion was kept for Fedora’s flagship >> desktop edition, the one that is most prominent on https://getfedora.org >> and probably the one that most newcomers download. We would still verify >> everything on Fedora Workstation on x86_64. But any other desktop >> (including Workstation on aarch64) would get just reduced criteria, because >> we simply can’t ensure the same quality bar for the smaller desktop >> editions/spins. There are some high-profile types of applications that I >> considered including in the list above, but didn’t in the end: >> >> * word editor (e.g. libreoffice-writer) >> >> * spreadsheet editor (e.g. libreoffice-calc) >> >> * video player (e.g. totem) >> >> * help viewer (e.g. yelp) >> >> I’d like to hear your thoughts on whether they should be included or not. >> Of course from an end-user point of view, it would be beneficial. But the >> question is whether we as QA can promise their testing. And also whether we >> want to block the release e.g. if Gnumeric is broken on armhfp XFCE or if >> totem doesn’t work on aarch64. Yes, it’s unpleasant, but people using >> alternative desktops and architectures are usually far from beginners. It’s >> usually not difficult to install a different application. Also note that >> the apps included in x86_64 Workstation will be thoroughly tested so they >> should be very likely to work well even on other architectures (minus some >> arch-specific issues). >> >> I’d like to target Fedora 32 with this proposal, which means we should >> decide on this proposal fast. (I wanted to propose it much sooner, but I >> was waiting on clarifications about new release-blocking images and also on >> some fesco tickets [3], some of which is still not clarified; so I’m sorry >> about a late proposal). >> >> Please comment, thanks. >> >> >> [0] >> https://fedoraproject.org/wiki/Fedora_32_Final_Release_Criteria#Default_application_functionality >> >> [1] https://fedoraproject.org/wiki/Releases/32/ReleaseBlocking >> >> [2] https://pagure.io/fedora-iot/issue/23 >> >> [3] https://pagure.io/fesco/issue/2339 >> >> [4] These two are required to work even in Basic release criteria [5], >> but I decided to include them anyway to avoid confusion (“why is the web >> browser missing?”), and to make it clear that they need to satisfy the >> basic functionality (whereas for Basic release criteria just the barebone >> functionality might be deemed enough) >> >> [5] >> https://fedoraproject.org/wiki/Basic_Release_Criteria#Required_applications >> >> >> > > I haven't received much feedback on this proposal. On test list and kde > list, there was a short follow-up regarding the list of release-blocking > applications. On kde list, Kevin disagreed because it lowers the guaranteed > quality bar for KDE, but he didn't continue discussing it with me. I > haven't received any alternative proposals either. On desktop list, there > was silence, but Chris Murphy gave me a thumbs up on IRC. > > I'll give it a few more days, and unless somebody yells hard (and ideally > also provides some constructive ways how to optimize desktop testing, > because we need to cut it down /somehow/), I'll put the change into effect. > I'll also probably move the help viewer into the list of release-blocking > applications, based on the discussion I saw. > Hi Kamil, I think this change makes sense. +1 from me as well. Kalev
_______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
