On Sun, 7 May 2006, [UTF-8] Uwe Stöhr wrote: > This email is a proposal for a new test procedure of LyX releases. > > In consequence of this I propose to create a testing procedure > LyX-releases to avoid regressions and new introduced bugs.
This makes a lot of sense to me. What John wrote about a place in the repository for storing test cases also makes sense. In addition I think it will be worth the time to clearly document *how* the testing can be done, so that hopefully non-coders can help with it. Something that would have been helpful when LyX was about to be released would have been a package containing: * Pre-release of LyX * List of test cases, where for each test case you have the following: ** Clear and easy-to-follow instructions on how to run the test ** Prerequisites (eg. if you need any particular external tool) ** Source material - probably just .lyx-files in this case ** Notes on what the result should look like. This would probably quite a few screenshots and/or PDFs. * Finally some way to report the results back I think it's probably unrealistic to create and maintain a big archive with all of this. However, I'm sure few of you will be surprised when I'm now going to suggest how we partially use the wiki for helping with this. So here's one possible approach: * Write a short/simple introduction to the testing and it's purpose. This would go on a wiki page. * The same wiki page (or another) could be used to contain instructions and links on how to obtain the relevant pre-releases of LyX for different platforms that we're interested in users' testing. * On the first wiki page, keep a list of simple test cases (each test case would probably get its own wiki page). * Each test case would contain the instructions for how to do the actual test. This includes links to the test cases inside the repository. At the end of the page we could let users add notes regarding the outcome of the test, including the type of platform that was used. Bug reports should still go to bugzilla of course. * Finally, the first wiki page could include the "results" section from each of the separate test pages, thereby producing a summary of the results. If this makes sense, I'd be happy to set it up. Something like this would also have made it much easier for me to test LyX when it was time for 1.4. Opinions? /Christian PS. I definitely think the test cases should go into the repository, but as a temporary measure we could upload them to the wiki. -- Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr
