On 10/28/05, Ovid <[EMAIL PROTECTED]> wrote: > The code is designed well enough that adding new features is quick and > easy. Unfortunately, whenever I need to change my code I fire up a Web > server and view the results in the browser and then write the tests > after I've written the code (this is closely related to When > test-driven development just won't do). This is because XML and XHTML > are just text. I need to see the output. I've been finding more and > more that small changes in my code are making huge changes in the > output and trying to continuously update the tests to exactly match the > XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious > productivity drop.
I'm in a similar boat, and I do the same thing. It's pretty easy to the idiot tests -- XML well-formed-ness and XHTML validity -- but testing the content of the output is obviously more difficult. At the very least, you need a mockup of your expectations, and creating that mockup by hand is *much* more error-prone and tedious than grabbing the initial output. Especially so when things like whitespace sensitivity enter into the discussion. While this isn't test-first development, it works well with the test-early-test-often mindset. Instead of testing your expectations (which are quickly met or dashed by visual inspection) you refocus on regression testing. So while you're not proving that your initial output is exactly as you expect, you *are* proving that your tests are the canary in the coal mine when you mess up. The other legs of this three-legged stool are version control and brute force. The project I'm working on at the moment involves parsing queries and converting them into SQL (involving a bizarre and baroque set of business rules). The only way I can keep my sanity here is to md5^Wcheck each and every input and every output to make sure that nothing changes. The only way for these tests to be meaningful is to be rigorous -- identify all possible things to test, and generate all possible combinations and permutations. When you break a corner case, you'll know. (HOP really helps here -- iterators, odometers and such really help when writing a few dozen lines of Perl that generate a few thousand inputs.) Keep in mind that all of your sources and tests are in subversion, so when things break, the regression two-step is a quick 'diff -b' and 'diff' to see if the changes are meaningful and/or expected. When those checks fail, there's a bug. (Alternatively, you may want to run a constant pre-processor on your test data to normalize it, but that's both trickier and riskier.) Of course, be ready to throw out previous test results when the expectations change radically. ;-) HTH, Z.
