On Mon, Feb 17, 2020 at 3:20 PM Kamil Paral <kpa...@redhat.com> wrote:
> Change the criterion to something along these lines: > >> All applications that can be launched using the standard graphical >> mechanism after a default installation of Fedora Workstation on x86_64 >> architecture must start successfully and withstand a basic functionality >> test. >> >> For other release-blocking desktops (on any architecture), the >> requirements only apply to the following types of applications: >> >> * web browser (e.g. firefox) [4] >> >> * file browser (e.g. nautilus) >> >> * package manager (e.g. gnome-software) >> >> * image viewer (e.g. eog) >> >> * document viewer (e.g. evince) >> >> * text editor (e.g. gedit) >> >> * archive manager (e.g. file-roller) >> >> * terminal emulator (e.g. gnome-terminal) [4] >> >> * problem reporter (e.g. abrt) >> >> If there are multiple applications of the same type (e.g. several web >> browsers), only one of them needs to satisfy the requirements. >> > There are some alternative approaches to this, but I'm not convinced they are better: *Alt#1: Lower the bar for everyone* In the past, we didn't have different rules for products based on their popularity. The actual testing coverage was probably different, because more popular products tend to receive more testing, but the rules were the same. If somebody feels that we should keep having exactly the same rules regardless of popularity (desktop environments, architectures, etc), the solution is to lower the bar for everyone. All desktop environments (including Workstation x86_64) would be subject to the same list of application types (as described in the proposal) that could block the release. Other apps wouldn't block. *Alt#2: Keep the original release criterion, but make testing best effort* If we have a QA test case, we currently make sure it's fully tested before we vote GO for the release candidate. We don't need to do that. We can say that this test case is just best effort, and if it's not tested at all, it doesn't stop us from voting GO. But if anyone finds a severe issue, it can still be a blocker. We used this approach for optical media testing [1]. But in this particular case, I think it's not a good idea, and will lead to frustration. The desktop apps are too numerous, they are complex, and it's way too easy to find bugs in them. If we keep it just best effort, everything else gets the priority treatment, and the likely outcome will be that many bugs will be found (either by QA or the community) shortly before the final release date, and everyone will be angry that it wasn't found earlier. This area is very unlike the optical media example mentioned, which is very specific and quite stable in features and behavior. [1] https://pagure.io/fesco/issue/2303 Do you consider one of these alternative approaches better than my original proposal? Or do you think there's a better way?
_______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
