Reply on:
I think QA should do some exploratory testing of major new features as time allows, but just verifying existing test cases that often are running automatically anyhow isn't a good use of time, I guess. Response: QA is mostly doing that already. We try to cover as much as possible in our test cases (many times more than the automated tests cover) and always add exploratory testing to that. We don't usually sign off on features without doing this. The reason I am using "mostly" in my first sentence is that there are bug fixes that are substantial changes to existing features and they don't get the right amount of QA coverage. Getting at least some of the time spent doing bug verifications and using it in following up with the features you signed off (even a couple of releases ago) might be a good move for QA. This would after all also involve verifying bugs that although not necessarily critical, are serious enough to produce regressions. Reply on: To that end, bug verifications are something we should be throwing at community members who want to help but don't have other special skills, and at people who are onboarding. Bug verifications are a great way to ramp up on a section of the product and to give back value to the project at the same time. Response: I couldn't agree more with the part where we could involve the community members in bug verifications. It happened several times that a QA was verifying a bug and, while running that test case from the reporter, he/she found one or two new bugs (The initial bug was fixed, but that fix revealed -or created- other ones). Those bugs wouldn't add enough value for QA to keep wasting so much time with verifications but they would be welcomed from the community. Response on all the replies about getting involved earlier with specifications &co: I'm starting to feel like we're trying to rediscover agile here (or at least an important part of it). Thousands of projects implementing agile methodologies have proven that involving QA from the beginning - or, most usually, the moment when settling on a list of requirements - pays off. E.g.: QA should be able to (more or less) easily notice that certain requirements won't end up as implementations the users will appreciate and raise that issue from the start. With our current procedures, QA starts working on a feature after it's already implemented and can only file bugs hoping that they will get fixed some time soon. The difference in the cost of fixing such an issue should be obvious. I should mention that we did try multiple times to get involved in feature work early on, but without up-to-date feature pages, specifications and/or requirements list, there really isn't much QA can do. If this is something everyone wants to see done, we'll have to make sure all the involved parts participate on their end. Ioana Budnar _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
