I have simplified and rewritten the test case to suit manual testers. I am keeping the long version for future automation.
On Tue, Dec 9, 2025 at 3:17 AM Linux Pundit <[email protected]> wrote: > > On 09-12-2025 00:19, Brandon Nielsen via test wrote: > > On 12/8/25 9:31 AM, Kamil Paral wrote: > >> On Fri, Dec 5, 2025 at 6:25 PM Adam Williamson > >> <[email protected] <mailto:[email protected]>> wrote: > >> > >> On Fri, 2025-12-05 at 17:14 +0100, Lukas Ruzicka wrote: > >> > Hello, > >> > > >> > The QA team has been missing test cases to test Basic networking. > >> I have > >> > written a draft of the test case and now I am collecting > >> suggestions on how > >> > to improve it, check if anything is missing, or if something > >> needs to be > >> > changed. > >> > > >> > The Basic networking test case is here: > >> > https://fedoraproject.org/wiki/QA:Testcase_Basic_networking > >> <https://fedoraproject.org/wiki/QA:Testcase_Basic_networking> > >> > > >> > We will also need to create a WIFI test case and a VPN test case, > >> both the > >> > CLI and GUI variants, so if anybody has ideas how to do it, let > >> me know. > >> > > >> > Thanks for your help. > >> > >> Thanks for writing this! > >> > >> It seems...long. :D I get that it's trying to encapsulate > >> everything in > >> the criterion, but it's a bit overwhelming. Still, it is very > >> comprehensive and robust. > >> > >> I think when I wrote the criterion I was envisaging something a bit > >> shorter, sloppier and vaguer, that mostly relied on the tester's > >> 'typical' network configuration and was just 'try and open a web > >> site, > >> does that work?' kinda stuff. But...maybe this is better? > >> > >> Curious to know what others think. Do we prefer something very > >> detailed > >> and 'clean room' like this? > >> > >> > >> Hmmm. First of all, thanks, Lukas, for clearly giving it a lot of > >> effort! Unfortunately, I'm afraid that this amount of effort will > >> make it one of those "nobody wants to do it" test cases, because it's > >> long and complex. But it depends what we want to do with this. If > >> this is to be automated, then big +1 from me, let's do it. If this is > >> intended to be a manual test, then I think we need to simplify it, > >> and not just because of length, that's just one of my concerns. One > >> of my other concerns is that this is... too short. Networking > >> requires a lot of know-how and in many areas it seems it doesn't go > >> deep enough, which means further expansion to explain some steps in > >> more detail would be needed. For example, it uses keywords like > >> "enp1s0" but doesn't explain how to figure out what *your* actual > >> network device is called. The same goes for IP ranges, connection > >> names ("Wired connection 1"), and maybe some more. > >> > >> Using VM as a remote server is of course the most straightforward > >> idea, but it has its own pitfalls. For example, ping doesn't work in > >> libvirt user session, only system session (at least according to my > >> past experience). The IP ranges will vary (unlike in the testcase, > >> mine is 192.168.124.0/24 <http://192.168.124.0/24>). And of course, > >> the tested system is often a VM itself. Does this testcase assume > >> nested virt in that case? Or VM<->VM connection? This will definitely > >> get even more complex, if we want this to be followed by the general > >> public with heavily varied networking environments. > >> > >> I also see two possible goals of this test case, and I'm not sure > >> which one was the intended one. The first goal is to verify that the > >> very networking basics work, setting an IP address, ping, curl, etc. > >> This test case verifies that. The other goal is to test that the > >> real-world network works on a particular device of the tester. So > >> actually sending packets to your router and to the internet, > >> receiving a web page, etc. Since this test case uses a local VM, are > >> those packets even going through the network card, or are they just > >> "virtualized" in the kernel? I have no idea. But it surely doesn't > >> test that your wifi connection works fine, or cable speed negotiation > >> with your router, and that you can open fedoraproject.org > >> <http://fedoraproject.org>. The first goal is great for automation, > >> the second goal is good for human testers with varied hardware. > >> > >> It might be best if we can do both. Convert this into an OpenQA > >> script that will run tirelessly each compose. The steps are exact, no > >> need for a human tester to repeat it. And let's have a simplified > >> version for humans, that will test connecting to a wifi/cable, > >> pinging a well-known server, opening a website (note that we already > >> have https:// fedoraproject.org/wiki/QA:Testcase_desktop_browser > >> <https:// fedoraproject.org/wiki/QA:Testcase_desktop_browser> for > >> that, though), and possibly some optional stuff like switching a dns > >> server. The wifi seems like the most important stuff to me (for human > >> testing), because we can't easily replicate that in OpenQA (most of > >> the other stuff we can). > >> > >> > > > > I generally agree with all the above, it looks like "too much" and > > likely to drive people off. If it could be largely automated, that > > would be excellent. > > > > That being said, I was able just now to complete the tests between two > > usermode VMs created and run via Boxes. It was pretty straightforward > > to complete. > > > > As for the wording of the test case itself, I would like to see the: > > > > `sudo nmcli con add type ethernet ifname enp1s0 con-name static` > > > > invocation added to the "Set up static networking on the tested > > machine (nmcli)" section as well so you can just copy paste the > > following commands. > > > > It might also be nice if the "Deactivate the existing connection" > > verbiage made it clear "existing" isn't some network-manager magic, > > and that you need to figure out your "existing" connection name via > > `nmcli connection show` as is made explicit in the server > > configuration section. > > > detailed testing layout is a must. > > maybe broken/arranged into categories based on "layers" of networking > > -- > _______________________________________________ > test mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/[email protected] > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > -- Lukáš Růžička FEDORA QE, RHCE Red Hat <https://www.redhat.com> Purkyňova 115 612 45 Brno - Královo Pole [email protected] TRIED AND PERSONALLY TESTED, ERGO TRUSTED. <https://redhat.com/trusted>
-- _______________________________________________ test mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
