On Tue, 2010-12-21 at 15:53 -0700, Kevin Fenzi wrote: > On Tue, 21 Dec 2010 17:11:15 +0000 > Adam Williamson <awill...@redhat.com> wrote: > > > Hi, everyone. So, in the recent debate about the update process it > > again became clear that we were lacking a good process for providing > > package-specific test instructions, and particularly specific > > instructions for testing critical path functions. > > ...snip... > > > Comments, suggestions and rotten fruit welcome :) I'm particularly > > interested in feedback from package maintainers and QA contributors in > > whether you feel, just after reading these pages, that you'd be > > confident in going ahead and creating some test cases, or if there's > > stuff that's scary or badly explained or that you feel like something > > is missing and you wouldn't know where to start, etc. > > ...snip... > > Great work Adam and QA folks. ;) > > From a quick glance over this looks good to me, and I would be happy to > start trying to make test cases. A few random things: > > * Would it be worth noting that anyone can make a test case, it doesn't > need to be the maintainer, right?
Oh that raises another good point. Unlike the Test_Results: wiki name space used for Test Days and similar test events, the QA: namespace does not allow anonymous edits. > * Also, might be worth noting that if you run into a specific bug as a > maintainer and fix it, thats a great time to go add a test case to > specifically test that (since it's fresh in your mind). Great point! I've also seen folks use placeholders in bugs when triaging or fixing them. For example, there are bugzilla keywords that can be used to tag bugs for future test case review, such as 'NeedsTestCase' (see https://bugzilla.redhat.com/describekeywords.cgi). > * finally, it's ok to just start in on some tests and add them over > time, right? We don't want to care if a package has incomplete > coverage right off the bat right? > > > > > it also mentions one big current omission: dependencies. For instance, > > it would be very useful to be able to express 'when yum is updated, we > > should also run the PackageKit test plan' (because it's possible that > > a change in yum could be fine 'within itself', and all the yum test > > cases pass, but could break PackageKit). That's rather complex, > > though, especially with a Wiki-based system. If anyone has any bright > > ideas on how to achieve this, do chip in! Thanks. > > Yeah, tough one. Not sure how best to handle that. Perhaps just a > 'Dependencies:' type header asking you to make sure you test dependent > packages and see their test cases? It occurred to me that transclusion might work in a pinch (http://www.mediawiki.org/wiki/Help:Transclusion). It can be non-obvious, but it might allow us to include PackageKit test cases in the yum test case category by doing {{Category:Package_PackageKit_test_cases}}. I've never tried this with Categories, so not sure if this works. I'll have to test this out. > Thanks again for working on this! Agreed! Great to see this coming together. Thanks, James
signature.asc
Description: This is a digitally signed message part
-- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
