On Sat, May 11, 2013 at 06:32:21PM +0800, Thomas Goirand wrote: > Hi Lars, > > I do like a lot the idea of running things like piuparts and such at > upload time. > If you have time to work this out with the FTP masters, that would be a very > good idea IMO, and I warmly welcome you to do that. However... > > On 05/10/2013 03:49 AM, Lars Wirzenius wrote: > > Tests for running reference installation might include the following: > > > > * Basic networking setup works: System responds to ping from the outside. > > * Mail server responds appropriate on the SMTP, submission, IMAPS, and POPS > > ports. > > * LAMP server responds on the HTTP and HTTPS ports. > > * A desktop system that automatically logs in a test user has the right > > processes running, and can start some common applications. > > * In each case, it's possible to log in remotely with ssh and run > > "sudo apt-get install hello". > > These wont help. They were not the RC bugs we had during the release > cycle. I believe for example, that Apache, MySQL, and PHP were quite > well maintained and didn't suffer from RC bugs that stayed for a long > time. I didn't see bugs for Exim, Postfix, ssh or Bind either. We had > problems with Dovecot though, but they were well identified, and > having tests against it wouldn't have help in any ways. > > If you want to find out which tests would help, you would have to conduct > a better analysis of the problems we had during the freeze, IMO.
You can't assume that just because something works today, it will work forever. Having such tests in place will help to automatically catch regressions if they arise. If they don't, fine, but you never know. While I agree with you that there are a lot more we can test, I don't think it's useless to include tests for stuff we know is working. Even "trivial" tests have their value with so many moving parts. -- Antonio Terceiro <terce...@debian.org>
signature.asc
Description: Digital signature
