I have a proposal regarding installation matrix [1] related to environments. Adam did a truly stellar job in his latest results table restructuring, he eliminated a lot of duplicated testing and simplified the tables a lot (for example, there's no point in testing additional HTTP repos in anaconda on all three major architectures, because that functionality it architecture independent). We don't have limitless resources and must keep focused on areas where it's likely for a bug to occur, instead of testing everything. Here's a proposal of some last touches to installation tables:
1. Testcase_Anaconda_User_Interface_Basic_Video_Driver was changed to a generic x86 test. In this case, it is problematic, because BIOS and UEFI fallback video driver are completely different code paths. I have a personal experience from that past when one of those worked and the other didn't. We should distinguish BIOS and UEFI in this case. To fix this, I see two viable options: a) move the test case to Miscellaneous section, so that the table in User interface section doesn't get more complicated b) keep the test case in User interface section, but create a separate table for it (right under the first table). This new table would have x86 BIOS and x86 UEFI columns. Again, the purpose is to keep the original table simple (avoid adding columns which are not required for anything else). 2. Testcase_Install_to_Previous_KVM and Testcase_Install_to_Current_KVM differentiate BIOS and UEFI. I always forget whether this is about the host or the guest (or both, or which combination). But I think host differentiation doesn't make any difference - libvirt should work the same, I assume. It makes sense to differentiate the guests. Unfortunately I believe the UEFI mode for libvirt/kvm [2] is not available under an open source license, therefore it's not packaged and not trivial to use. Which means not that many people probably use it. I wonder whether we really need to keep this separated. I would gladly merge this together to a single x86 column and split up once OVMF is available under a better license. If we keep it separate, we should provide better instructions for the UEFI part. PS: I know nothing about Xen, which is also in the table, so I have no idea whether it's a similar story or not. 3. Testcase_Anaconda_save_traceback_to_bugzilla, Testcase_Anaconda_updates.img_via_URL, Testcase_Anaconda_user_creation, Testcase_Anaconda_updates.img_via_installation_source and Testcase_Anaconda_traceback_debug_mode all differentiate BIOS and UEFI. I think that's a waste of our energy. None of these testcases should IMO be affected by firmware. We should make them generic x86 tests. We could go even further and make them completely arch-independent (at least some of them), but I'm a bit careful around ARM, so not completely sure about that. What da ya think? Thanks, Kamil [1] https://fedoraproject.org/wiki/Test_Results:Fedora_21_Final_TC1_Installation [2] http://www.linux-kvm.org/page/OVMF -- test mailing list [email protected] To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
