On Wed, 03 Aug 2005 09:51:49 -0400, Peter Hansen <[EMAIL PROTECTED]> wrote: >phil hunt wrote: >> On Tue, 02 Aug 2005 21:26:28 +0200, Christoph Zwerschke <[EMAIL PROTECTED]> >> wrote: >>>According to the "extreme programming" paradigm, testing should be done >>>several times a day. So a requirement for extreme programm is that tests >>>are fast enough. If the testing needs too much time, people are >>>discouraged to test often. >> >> Indeed. Running the tests should ideally take less than a few >> seconds. Any longer, and people won't use them so often. > >That's probably too general a statement to be very useful, and I don't >think it's probably helpful to a newcomer who is seeking guidance.
I did say "ideally"; if the test have to take longer, so be it. >If one makes the XP-style distinction between "unit" and "acceptance" >tests, then the acceptance test *suite* should take no more than some >number of minutes. I believe ten minutes absolute max is a commonly >offered rule of thumb, but I've found even eight minutes a little too >long. Five seems to be just fine, and less if of course wonderful. I think we might be talking at cross purposes here. To me "acceptance test suite" means a test suite that has to be passed each time before a new version of the software is released to the users. I don't see that 10 minutes is a sensible limit here, unless you are releasing more often that once a day. (I once had an acceptance test suite that ran for 12 hours; I used to run it nightly). >If you're doing >test-driven development (TDD), you will want to be able to run unit >tests (though maybe not all of them) sometimes a couple of times per >minute! That's what I often do. >Still, in a large project (and especially one written in Python, with >the overhead of interpreter startup and the cost of executing bytecode) >the suite can get fairly long if you have many hundreds of tests. I'm currently running one with 227 assertions in 38 test functions. It runs in <1 second on an AMD 3000+. -- Email: zen19725 at zen dot co dot uk -- http://mail.python.org/mailman/listinfo/python-list
