On Wed, Sep 13, 2006 at 11:15:04AM -0700, Gary Kline wrote: > On Wed, Sep 13, 2006 at 04:46:05PM +0200, Pawel Jakub Dawidek wrote: > > On Sat, Sep 09, 2006 at 12:38:13PM -0500, Karl Denninger wrote: > > > This is not cool folks. > > > > I'm really sorry for the breakage. I'm trying to treat -STABLE very > > gently, unfortunately this time I made a mistake. > > > > The change was committed to HEAD at 9 August. The change fixed one bug, > > but introduced another, which I didn't expected. The change seemed to be > > trivial and I only tested that it fixes the bug I was tracking down, I > > haven't looked for regressions. > > > > Well, after this lengthy discussion, I've switched to -RELEASE. > -STABLE just ain't... We all realize that none of us would > put out a buggy release--not even -CURRENT. But let me ask > the next obvious question. How difficult would it be to > build a regression test, or suite of tests? Obviously, this > could be done over months -> years. (In my last lifetime > as a hacker I was in the kernel test group [a BSD-4.4 based > release on new architecture]. ) It's a bit hard to believe > that with all the genius in this effort, that no regression > testing is done.
I'm trying to implement regression tests to the code I add. You can find
them in /usr/src/tools/regression/:
geom_concat 2 files, 2 tests
geom_eli 15 files, 5818 tests
geom_gate 3 files, 6 tests
geom_mirror 7 files, 27 tests
geom_nop 2 files, 2 tests
geom_raid3 12 files, 13 tests
geom_shsec 2 files, 6 tests
geom_stripe 2 files, 2 tests
ipsec 1 file, 306 tests
redzone9 1 file, 6 tests
usr.bin/pkill 27 files, 49 tests
As I said already, I mistakenly thought the change was trivial and the
only thing I tested was if it fixes a bug I was tracking down back then.
We dicuss from time to time that we should have service simlar to
tinderbox, which will run regression tests regularly and report
regressions to the mailing lists - the more we automate the smaller
chance for a human mistake like mine. Unfortunately this is not yet
done.
--
Pawel Jakub Dawidek http://www.wheel.pl
[EMAIL PROTECTED] http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
pgp2bvrpeTG1I.pgp
Description: PGP signature
