Hi Jean, I am sorry for the late repply. I have been busy with LO-3.3-rc2.
Jean-Baptiste Faure píše v Čt 16. 12. 2010 v 06:05 +0100: > Le 13/12/2010 15:04, Petr Mladek a écrit : > > This is why I proposed 7 days instead 5 days. 7 days have the Weekend > > included by definition :-) > Not really if the period start just before the WeekEnd. If a new RC is > released on Friday for QA-test, most of the testers will not be able to > start their tests before Monday. Fair enough. We will try to do not release on Friday. Alternatively, would it help if we pre-announce the upcoming rc2 few days before it is available? It might always be delayed because of technical problems. Though, I think that we could predict it pretty well two days before it is published. > > Hmm, I think that OOo-3.3 release is not a good example: > Yes, from RC cycle point of view, it is an example that LibO should not > follow. > The process is not sufficiently clear: Yeah, we are still setting the processes up. > what QA-team must do when a stopper is found and known? The current solution is described at http://wiki.documentfoundation.org/Release_Criteria#Blocker_Bug_Nomination > If a new RC is build immediately, a second > stopper can be fixed only on this new RC so it is not interesting to > continue to test the old RC. It is more true because the new RC can > introduce regression to the old one. You are right that any fix could bring regression. Though, I think that it always makes sense to continue with testing: + there might be more blocker bugs; they are usually independent; it is better to find all blockers as soon as possible + the fix for the stable branches should be tested and reviewed before they are committed; it should reduce the number of regressions to minimum; the question is if we really need to repeat the whole QA for each rc; I think that it should depend on the number and complexity of the fixes; this information should be mentioned in the announce mail + it takes few days to produce new rc (time needed to fix the bug, do some testing, build in Windows, Linux, MAC; upload to mirrors) > So I think we need a very clear QA test process: known dates for RC, I am not sure if we want a regular schedule, e.g. do the rc release every Thursday and every second Thursday. I think that such approach would wok well for alpha/beta builds. In case of release candidates, I think that it is better to do them according to the current state. I mean to release them when all blockers are fixed or when there are more that 2 weeks without release. Note that we plan to upload also daily builds in the future. They might be used to verify fixes even before the official release. > when it is useful to search blockers and when we can stop to search > stoppers until a new RC is released. I think that it makes sense only when the application does not start at all, so that the blocker bug blocks the testing completely. > > There is an idea of more micro bug fix releases. The published minor > > release should be well usable for 99% of normal users. The micro > > releases could be done every next month or so. They would include just > > safe and reviewed bug fixes and should need not full QA. The second or > > third micro release should be well usable for 99% experienced users. > Ok for me. The only condition I see to make the end-users happy, is that > the status of each version released must be very clear for the end-user. > Individual users or small organizations may want use always the last > micro-release while bigger organizations like administrations will have > a longer cycle of update. Yes, we need to make it clear. > >>> Also I think that it does not make sense that every national team would > >>> do the whole testing. IMHO, 95% of the functionality is language > >>> independent, so most of the testing can be shared and distributed. > >> Right. For OOo, NL QA-team do only Release Sanity Test. But the current > >> test period on OOo 3.3.0 shows that the last blockers have been found by > >> the Community and these blockers could not be found by automatic or TCM > >> tests; they have been found by users who have done tests on their > >> documents they use in the real life. > >> To do that we need time. > > I see. Well, I think that even the released OOo version included pretty > > annoying bugs and some of them would be considered as blockers. Such > > bugs were usually fixed in the micro release x.y.1. > > > > We just need to define the right compromise that would be acceptable for > > all sides developers, testers, and users sure > > Just to get better picture. How much time would you need to finish all > > known manual tests? > For OOo and Release Sanity Test, time needed is about 2 to 4 hours to > complete all the 25 tests in the scenario. Okay, this sounds doable within the week if you can plan it. > But, as said the last blockers are out of the scope of these tests and > have been found by test-cases and files from the real life. I guess that many of these bugs were found by normal users that used OOo in their daily work. I think that it actually supports the idea for more frequent releases and tested/reviewed fixes. The manual testing makes sense and is really helpful, definitely. Though, it could not verify the complete functionality within days or even weeks. Best Regards, Petr _______________________________________________ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
