On Wed, Sep 7, 2011 at 9:14 PM, Konstantin Boudnik <c...@apache.org> wrote: > On Mon, Aug 22, 2011 at 09:58AM, Eli Collins wrote: >> On Sun, Aug 21, 2011 at 9:34 PM, Konstantin Boudnik <c...@apache.org> wrote: >> > System tests (Herriot controlled) tests were a part of nightly testing of >> > every >> > build for at least 2 of .2xx release. I really can not comment on .203 and >> > after. >> >> Owen - are you running the system tests on the 20x release candidates? >> Do we know if the 20x release pass the system tests? >> >> > A normal procedure was to build a normal bits and run the tests; build >> > instrumented bits, deploy them to a 10 nodes cluster, and run system tests. >> > The current state of the code is that system tests require source code >> > workspace to be executed from. I have done some initial work to do >> > workspace >> > independent testing but I don't know if it has been included to the public >> > releases of .203+ - I haven't really checked. >> > >> > At any rate, running system tests are an easy task and the wiki page is >> > explaining how to do it. >> >> Running the system tests is actually not easy, those wiki instructions >> are out of date, require all kinds of manual steps, and some of the >> tests fail when just run from a local build (ie they require 3 DNs so >> you have to setup a cluster). > > I will try once again: system tests are ALWAYS require cluster. This is why > they are called 'system' in the first place. The execution model, however, > require source code workspace as well so you can say 'ant test-system' there: > ant is used as the driver. > > I am going to revisit the wiki to make sure it is up-to-date. But I don't > think it has been out-dated as you say.
Its the system tests that need to be fixed, eg they're referencing scripts in bin that now live in sbin etc. You'll discover this stuff if you try to run them. >> How hard would it be to copy your 22 system test Jenkins job to adapt >> it to use a 20x build? Seems like the test bits should mostly be the >> same. > > Adapting something is usually a non-effort conditional to the HW and time > resources available. > Will you volunteer to maintain the system tests? Currently they're not running as part of Hudson and are bit-rotting. Thanks, Eli
