On Tue, Jul 22, 2008 at 10:21 PM, Alexander Leidinger <[EMAIL PROTECTED]> wrote: > Quoting "Garrett Cooper" <[EMAIL PROTECTED]> (from Tue, 22 Jul 2008 > 08:31:12 -0700): > >> On Tue, Jul 22, 2008 at 2:10 AM, Alexander Leidinger >> <[EMAIL PROTECTED]> wrote: >>> >>> Quoting "Garrett Cooper" <[EMAIL PROTECTED]> (from Mon, 21 Jul 2008 >>> 23:43:11 -0700): > >>>> I'm also working on getting tst_res (1.) dually licensed from >>>> the LTP folks with a BSD / GPL license to allow for better inclusion >>>> into FreeBSD's infrastructure, to ensure that we have a deterministic >>>> means of dealing with testcases and reporting. >>>> Just wondering what, if any, interest would be in adopting and >>>> applying this work to FreeBSD for the good of the community. >>> >>> If this is what is used in the LTP to run some tests, I have to say it >>> does >>> not look mature. I've run the LTP a lot in FreeBSD (native linux >>> binaries, >>> http://wiki.FreeBSD.org/linux-kernel/ltp ... LTP not run and updated the >>> page since a long time) to test the linux compatibility layer, and the >>> are >>> several places where tests fail and no output is produced or even the >>> summary said the test passed. I even opened bug reports at the LTP page >>> on >>> SF, but it seems nobody was interested in those reports. >> >> tst_res consists of the functions: >> >> tst_res >> tst_resm >> tst_brk >> tst_brkm >> tst_brkloop >> tst_brkloopm >> tst_exit >> >> which (thanks to some feedback) has finally made it up onto its own >> online manpage: >> >> http://ltp.sourceforge.net/ltpmantemp.php?file=man3/tst_res.html >> >> This is the simple output and behavior modification API's that produce >> messages like (according to the sample on the manpage): >> >> tsttcs01 1 PASS : Able to create MAXUP processes >> tsttcs01 2 FAIL : Too many processes (MAXUP+1) created >> tsttcs01 3 BROK : tabinfo(PROCTAB, &tbs) failed; errno = >> 13: Permission denied >> tsttcs01 4-10 BROK : Remaining cases broken >> tsttcs01 0 WARN : cleanup(): kill(0, SIGALRM) failed; errno >> = 3: No such process > > And this is where I've seen problems while testing the linux compatibility > layer in FreeBSD. Some tests said PASS when in reality they didn't pass but > died. I don't know if this is fixed now, it's been a while when I reported > this.
Yeah, and I've seen tests hang on LTP, but that's because people don't monitor test execution but just claim that since it works, it's fine. I try to keep abreast of these things because test quality's important; otherwise your results are worth no more than a $2 bill! >>> There are also some other ideas, like using the protocol the perl test >>> suites use, to be able to use existing perl stuff to generate reports and >>> overviews out of the generated logs (AFAIR this was the idea behind some >>> changes to the existing regression tests a long time ago). >> >> I have yet to really explore Test::Harness, but while I like Perl and >> I'm sure that Test::Harness is as good as people laud it to be, I find >> dealing with Perl to be cumbersome as they tend to break some stuff >> between releases =(... > > I don't know if I was clear, the tests themself are written in C (or > whatever) and not in perl, the goal as I remember it was to be able to use > the already existing perl tools where we don't care about the development, > as someone else will fix broken things because it is used for perl itself. > > Bye, > Alexander. Hmmm... I've also been forced into working with Python nose lately, and while I might not be a Python officiando, I do find the framework to be expeditious and simple to use. Just another thought, because Python seems to be a better base framework... -Garrett _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
