One simple thing I asked about the other day was running multiple instances of autotest to do different things. Currently I'd like to run one for my specs and one for my features, but you could easily extend this idea. Creating several profiles that run at the same time, with the long running ones having a low priority would give a range of feedback that eventually would be fairly complete (on a big project it might fully catch up overnight, or at the weekend) whilst providing sufficient feedback to be able to iterate quickly with reasonable confidence
2009/4/14 Stephen Eley <sfe...@gmail.com> > On Mon, Apr 13, 2009 at 7:46 AM, aslak hellesoy > <aslak.helle...@gmail.com> wrote: > > > > So if someone develops a better AutoTest with a plugin architecture, and > > that doesn't have to run as a long/lived process, then I'd be very > > interested in writing the neural network part - possibly backed by FANN > > (http://leenissen.dk/fann/) > > In the immortal words of Socrates, "That rocks." > > The nice thing about separating concerns like this -- the reason why > design patterns appeal so much to me -- is that pieces can rather > easily be built up incrementally. As Matt said, thinking 'neural > networks' with this is way beyond the level of anything I'd had in my > head. But it's a damn cool idea. > > I can think of a few ways to handle this particular chunk, with levels > of complexity that would scale up: > > 1.) DUMB REACTION: Just re-run the tests that failed in this run > cycle. Then, periodically, run tests that succeeded in the background > as a regression check. (This isn't much beyond what Autotest does > now.) > > 2.) OBSERVERS: Allow handlers to register themselves against certain > files, so that when a file is changed, that handlers gets run. > Multiple handlers can observe any given file, and a handler can > declare multiple rules, including directories or pattern matches. > (Again, Autotest has something _sort of_ like this, but not nearly as > flexible.) > > 3.) PERSISTENCE: Track the history of tests and the times they were > created, edited, last run, and last failed. Also track file > modification times. When a file changes, run the tests first that are > either new or have failed since the last time the file was changed. > Then run the tests that went from failing to passing in that time. > (This could certainly be improved -- I haven't sat down to figure out > the actual rule set -- but you get the gist. Know when things changed > and set priorities accordingly.) > > 4.) INTELLIGENCE: Aslak's neural network. Let the system figure out > which tests matter to which files, and run what it thinks it ought to > run. Maybe use code coverage analysis. It can 'learn' and improve > when the full suite is run and it discovers new failures. > > In all four of these cases, I still think it's imperative to run the > full suite. None of these methods are foolproof, and code is tricky > and makes weird things happen in weird crevices. That's _why_ testing > must be done. But running the suite doesn't have to be a 'blocking' > activity, like it is with Autotest now. It can happen in bits and > pieces, when nothing else is going on, and it can be configured to > only grab your attention when something failed unexpectedly. > > (That's one of the prime reasons for the 'multiple output views' > feature, by the way. When I said output views I wasn't just thinking > of the console or a window. I'm also thinking Dashboard widgets, or > gauges in the toolbar, or RSS feeds, or dynamic wallpaper, or whatever > else anyone can think of. Stuff that stays out of your way until it > either needs you or you choose to look at it.) > > Still making sense? This is starting to sound pretty big and pretty > complex -- but I don't think it strictly needs to be. #1 and #2 above > are pretty easy. The others don't have to be built before releasing > something. And, of course, you wouldn't have to pick just one > selection module. You could run or disable all of these depending on > your project's needs, or come up with your own system involving Tarot > cards and Linux running on a dead badger.(*) > > I just want to build a core that detects changes in stuff, tells other > stuff about it, and passes on what that stuff says about it to a third > set of stuff. The rest is implementation-specific details. >8-> > > (* http://www.strangehorizons.com/2004/20040405/badger.shtml ) > > > -- > Have Fun, > Steve Eley (sfe...@gmail.com) > ESCAPE POD - The Science Fiction Podcast Magazine > http://www.escapepod.org > _______________________________________________ > rspec-users mailing list > rspec-users@rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
_______________________________________________ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
