On Fri, 06 Jan 2012 10:20:28 -0700, Ian Kelly wrote: > On Fri, Jan 6, 2012 at 10:01 AM, Lie Ryan <lie.1...@gmail.com> wrote: >> That unittest executes its tests in alphabetical order is >> implementation detail for a very good reason, and good unittest >> practice dictates that execution order should never be defined (some >> even argued that the execution order should be randomized). If the test >> runner turns out to execute tests concurrently, that should not cause >> problems for a well-designed test. Displaying the test results in a >> more convenient order for viewing is what you really wanted in 99.99% >> of the cases. > > Randomizing the order is not a bad idea, but you also need to be able to > run the tests in a consistent order, from a specific random seed. In the > real world, test conflicts and dependencies do happen,
In the real world, test conflicts and dependencies are bugs in your test suite that should be fixed, like any other bug in code. The fact that it is test code that is failing is irrelevant. Also in the real world, sometimes it is too hard to fix a bug and you just live with it. > and if we observe > a failure, make a change, rerun the tests and observe success, we need > to be able to be sure that we actually fixed the bug, and that it didn't > pass only because it was run in a different order. Every test should stand alone. You should be able to isolate each and every test and run it without the others. If you can't, your test suite is buggy. (Chances are good that most test suites are buggy. After all, who writes tests for their tests? That's just the start of an infinite regress with rapidly diminishing benefit.) You may not be able to run tests *simultaneously*, due to clashes involving external resources, but you should be able to run them in random order. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
