2016-07-12 10:49 GMT-06:00 Adam Williamson <adamw...@fedoraproject.org>:
> On Sun, 2016-07-10 at 21:30 +0530, Sayan Chowdhury wrote: > > Hi, > > > > I recently packaged and pushed an update for > fedmsg-meta-fedora-infrastructure > > to bodhi and exactly 40 secs[1] later I got a +1 to the update. I am > sure that > > testing a package surely takes more than 40 secs. This makes me really > curious > > that are the packages really being tested before giving out the karma. > > > > After going through messages in datagrepper[2][3], I found that few > people are > > giving out karma in one go (4-5 packages under a minute). If these > packages > > really are not-tested and the karma are given out randomly then I am > sure that > > this sure going to affect the release, infrastructure and our users. > > > > Does anybody know what is going on? > > So I've been discussing this with various people in the last few days, > and one specific idea has come out of that which I'd like to float. > > We've been hesitant to suggest this before as we thought packagers > might not like the idea, but we figured it can't hurt to at least > suggest it. > > The idea is this: there could be a requirement for all packages to > provide at least *some* kind of 'how to test' information. > > Looked at from the perspective of a new tester, the current system is > quite difficult to handle when it comes to upgrades of packages which > aren't obviously part of the critical path (e.g. kernel) or a well- > known GUI application package (e.g. firefox). > > How do we *expect* a new tester to respond when they come to an update > for fedmsg-meta-fedora-infrastructure , in Bodhi or fedora-easy-karma? > It's very difficult - probably impossible - for them to know or work > out what they should actually do to test this package. > > Of course, writing instructions for every single package is a lot of > work, but right now we have test cases for very few packages, and it > would definitely help if we could significantly increase that number. > > Sound like a good idea for a virtual FAD, this way more people can help > What do people think about this idea? > > To be clear, the idea would be to have general-purpose instructions for > basic functionality testing of each package, not requiring new 'how to > test' text to be written for every individual package update, > specifically tailored to the changes in that update. > > The way the system works at present is that Bodhi will show all wiki > pages in a specific category based on the package name. For package > 'foo', Bodhi's web interface and fedora-easy-karma will list all wiki > pages in the category "Category:Package_foo_test_cases" . These are > expected to be typical 'test case' pages, though there isn't actually > any technical *requirement* for them to be, or enforcement of that. > -- > > Adam Williamson > Fedora QA Community Monkey > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net > http://www.happyassassin.net > -- > devel mailing list > devel@lists.fedoraproject.org > https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org >
-- devel mailing list devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
