On Sat, Jul 27, 2013 at 5:26 AM, Glyph <gl...@twistedmatrix.com> wrote:
> > On Jul 26, 2013, at 7:12 AM, exar...@twistedmatrix.com wrote: > > To address this problem, I suggest you get into the habit of watching your > unit tests fail in the expected way before you make the necessary > implementation changes to make them pass. > > This is only one of an unlimited number of ways your unit tests can be > buggy. It might be tempting to try to fix the test runner to prevent you > from ever falling into this trap again - and who knows, it might even be a > good idea. > However, if you run your tests and see them fail in the way you expected > them to fail before you write the code that makes them pass, then you will > be sure to avoid the many, many, many *other* pitfalls that have nothing to > do with accidentally returning the wrong object. > > This is just one of the attractions of test-driven development for me. > > > On a more serious note than our previous digression, perhaps *this* is the > thing we should be modifying Trial to support. > > The vast majority of Twisted committers do development this way - or at > least aspire to, most of the time - but to someone new to automated > testing, it's not entirely clear *how* you're supposed to use something > like Trial, or how important it is that you see the tests fail first. > > Perhaps if trial had a bit more of a memory of things that happened > between test runs it would be useful. For example, a mode where you could > tell it what you're working on, and you could just re-run the same thing > and you'd only get a 'success' when you went back and forth between red and > green. > > Here's a silly little narrative about how one might use such a thing: > > $ tribulation begin myproject > Beginning a time of turmoil for python package 'myproject', in > './myproject/'. > myproject.test_1 > Case1 > test_1 ... > [OK] > > > ------------------------------------------------------------------------------- > Ran 2 tests in 0.033s > > PROCEED (successes=1) - All tests passing, an auspicious beginning. Now > write a failing test. > $ tribulation continue > myproject.test_1 > Case1 > test_1 ... > [OK] > myproject.test_2 > Case2 > test_2 ... > [OK] > > > ------------------------------------------------------------------------------- > Ran 2 tests in 0.033s > > *AGAIN* (successes=2) - a test should have failed. > # oops, 'test_2' was just 'pass'... let me fix that > $ tribulation continue > $ tribulation begin myproject > Beginning a time of turmoil for python package 'myproject', in > './myproject/'. > myproject.test_1 > Case1 > test_1 ... > [OK] > myproject.test_2 > Case2 > test_2 ... > [FAIL] > > > ------------------------------------------------------------------------------- > Ran 2 tests in 0.450s > > PROCEED (successes=2) - we are working on myproject.Case2.test_2 now. > $ tribulation continue > myproject.test_2 > Case2 > test_2 ... > [FAIL] > > > ------------------------------------------------------------------------------- > Ran 1 tests in 0.020s > *AGAIN* (successes=2) - you should have made the test pass. > $ tribulation continue > myproject.test_2 > Case2 > test_2 ... > [OK] > > > ------------------------------------------------------------------------------- > Ran 1 tests in 0.01s > PROCEED (successes=1) - myproject.Case2.test_2 works now, let's make sure > nothing else broke. > $ tribulation continue > myproject.test_1 > Case1 > test_1 ... > [OK] > myproject.test_2 > Case2 > test_2 ... > [OK] > > > ------------------------------------------------------------------------------- > Ran 2 tests in 0.033s > PROCEED (successes=2) - no regressions, find the next thing to work on > $ tribulation conclude > You have received one billion points, congratulations you have defeated > software. > > > Does this seem like it might be a useful feature for someone to work on? > Not shown here is the part that when you do introduce a regression, it > runs just the tests that failed until you fix all of them, then goes back > up the suite until it reaches the top and you move on to the next thing... > > I like the idea. testrepository <http://testrepository.readthedocs.org/en/latest/> stores tests results in a database and already has support for running just the tests that failed, etc. It just runs whatever executable you give it, provided that executable outputs subunit. Not on a computer with it installed (and can't find a good webpage), but here's the basic gist: $ testr init # initialize the database $ testr run # runs everything ... # test output goes here $ testr run # runs everything, shows delta of tests failed & time taken $ testr run --failing # just run what failed It doesn't have the TDD straightjacket you describe, but that would be a fun thing to make. jml
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
