Hi, On Thu, Jun 09, 2005 at 04:59:31AM +0100, Scott James Remnant wrote: > [...] > My main concerns so far with test suites is that the suite be: > > 1) easy for users and casual developers to run, and be able to quickly > identify failed tests and use them to fix the code. > > 2) sufficiently easy to use to encourage developers to write tests with > every code modification they make, and even with every bug fix. > > 3) not require exotic or unusual support platforms or languages. > > I've not yet come across an off-the-shelf suite that manages this for C; > perhaps I'm a bit spoiled by pyunit and doctest, but I want something > approaching that level of usability.
I guess you're talking here mostly about unit tests, what about functional tests? > DejaGNU in particular doesn't seem to manage to satisfy any of these > three concerns, the output of the suite is hard to use, the tests and > especially C unit tests are hard to write and it requires TCL which is > so widely incompatible with itself and no longer anywhere approaching a > usual support platform. My idea is _not_ using DejaGNU "directly", but using it as a "base" to build our testing framework. I really _don't know_ if DejaGNU gives us enough to base the framework on it, or if it would be better to simply start fresh. Have you seen my current work? To add a dpkg-parsechangelog test, for example, you simply drop two files in testsuite/dpkg-parsechangelog: xxx.input and xxx.expected, where xxx is usually a number. The problem is, of course, adding new "types" of tests, in which case you do have to program in Tcl :-( About the C unit tests, currently (with my toy testsuite) you can simply drop some *.c file with your new tests, and add it to the PROGRAMS variable in the Makefile. Moreover, it's trivial, with a simple naming convention, to remove the second step. The actual unit tests could be simpler to write (they currently use the "check" library), but I guess we can do some simple code-generation to remove the repetitive chunks. Do you have an idea of how would you like to make them? pyunit-like, other? The output of the DejaGNU suite is too verbose for me, but perhaps not particularly difficult to parse. I _thought_ that using DejaGNU for this would let us use some testing front-end written by some other person, _but_ I really don't know how many people use DejaGNU _and_ has written extensions, and I don't know if it's better using DejaGNU and being able to use third-party extensions, or not using it and being able to program in some non-Tcl language :-) About the Tcl incompatibilities, I really don't know what you mean, my experience with Tcl is almost limited to this experiment... care to explain? BTW, what platforms/languages do you consider non-exotic and non-unusual? Ruby, Haskell, Python, Shell, Perl, OCaml? > That being said, I did have some interesting discussions with tridge at > LCA about how they test the samba suite. Any pointers or ideas? -- Esteban Manchado Vel�zquez <[EMAIL PROTECTED]> EuropeSwPatentFree - http://EuropeSwPatentFree.hispalinux.es Help spread it through the Net in signatures, webpages, whatever! -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
