--- Piers Cawley <[EMAIL PROTECTED]> wrote: > The xUnit style framework does a much better job of enforcing test > isolation than Test::More does (but you have to remember that what > Test::More thinks of as a test, xUnit thinks of as an assertion to be > used *in* a test).
After working with xUnit style testing for almost a year, I definitely agree with this assessment. However, I think each style has its merits. With xUnit frameworks, each test will set up its data in the database, run through its asserts and then rollback the data. The setup and teardown methods ensure that each test is isolated from all other tests and you thus don't have accidental dependencies. The also impose a considerable overhead. At my current position, we've managed to get our test suite (xUnit style) running fast enough that it takes about 50 minutes to run. At my previous job, a comparable size test suite using Test::More took less than 5 minutes to run (I judge "comparable sized" by comparing Test::More tests to xUnit asserts.) Between the two, the 50 minute tests suite is probably more robust, but taking 10 times as long to run means that we've been bitten a few times by programmers who are in a hurry and don't run the full suite for a change that "can't break anything." The other concern I've had with our style of xUnit testing is that we're testing behavior, but not the actual data. With Test::More, we tested against a copy of the live database (when possible -- but this definitely caused some issues) and we sometimes caught data problems that xUnit style testing seems less likely to catch. The reason for this is quite simple: when you setup the data, you're setting up your *idea* of what the data should be. This might not match what your data actually is. Am I missing something fundamental? The long test times and the lack of real data testing are two weaknesses that I would like to correct in our current system, but I see how beneficial xUnit testing is, so I'm concerned about introducing "fixes" that weaken our testing strategy. How do others deal with this? Cheers, Ovid ===== Silence is Evil http://users.easystreet.com/ovid/philosophy/indexdecency.htm Ovid http://www.perlmonks.org/index.pl?node_id=17000 Web Programming with Perl http://users.easystreet.com/ovid/cgi_course/
