Martin McCarthy <martin.c.mccar...@outlook.com> writes: > 4) Automation. I read a thread on this mailing list recently talking > about openQA and whilst I am not opposed to automatic testing by > machines, I would insist that the manual testers are still retained as > relying solely on machines for QA can be problematic (machines are not > perfect just like humans). Not only that, the testers over on #debian-cd > is a community and having full automation would mean that community no > longer meets regularly to test stuff...and thus the community disperses > and fades away over time. FOSS to me is all about community so I would > hope that openQA works alongside the ISO testers rather than replace us. > This is slightly off-topic but AI is doing a lot of damage to corporate > tech and I hope that the FOSS projects don't fall victim to it too.
I don't think there's any danger of OpenQA doing this (speaking as the person who set openqa.debian.net up) because the by-hand testing and openQA are complementary rather than competing. They're aimed at differing targets. The automated tests are mostly about catching regressions, mostly during development, and I'd hope that as the salsa integration comes together they'll become something that can provide confidence to people as they contribute to D-I. My perception is that the by-hand testing is more about making sure things are working across as wide a set of use cases on as wide a range of hardware as possible, and is focussed around release time. Writing automated tests that can notice things looking "weird" is rather hard -- OpenQA is very good at noticing that someone changed the font kerning very slightly (which is where many of our false positives come from), but if something is broken on a bit of the screen that's not specified as something to look out for in the test, it'll never notice. Also, testing on real hardware is not exactly trivial with OpenQA, so that has not been a priority, but even if we do get to the point where we can do openQA testing of real hardware, I wouldn't expect that the spectrum of hardware under test would be anywhere near as wide as currently gets tested by hand, nor would I expect for it to be useful (or even possible) to automate all of the tests done by hand. Of course, if fleshing out the set of tests that OpenQA runs (on VM or real hardware) renders some of the by-hand tests mostly redundant, that doesn't seem terrible, especially if we concentrate on automating the tests that are particularly tedious to do by hand. (suggestions welcome :-) ) Cheers, Phil. -- Philip Hands -- https://hands.com/~phil
signature.asc
Description: PGP signature
