On Thu, Mar 08, 2012 at 09:17:42AM -0300, Ademar Reis wrote: > On Thu, Mar 08, 2012 at 11:54:31AM +0000, Stefan Hajnoczi wrote: > > On Thu, Mar 8, 2012 at 11:44 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > > > On Thu, Mar 8, 2012 at 4:00 AM, Lucas Meneghel Rodrigues > > > <l...@redhat.com> wrote: > > >> One of our main goals is to provide useful tools for the qemu community, > > >> since we have a good number of tests and libraries written to perform > > >> integration/QA testing for that tool, being successfuly used by a number > > >> of > > >> QA teams that work on qemu. Also, we recently provided a subset of that > > >> infrastructure to test libvirt, one of our virtualization projects of > > >> interest. > > > > > > Thanks for sharing. > > > > One thing I should have added is that my message is about what it > > would take for me to use autotest and contribute tests. But now I > > realize that you might be going for a different model: > > > > If you're aiming for a different model where autotest integrates > > external test suites (i.e. tests wouldn't be written in autotest.git, > > instead autotest.git would contain snapshots of external test suites), > > then this proposal seems fine. Upstream projects like QEMU would > > develop their own test suite and it would be dropped into autotest or > > a specific autotest instance. > > > > Yes, that's the idea. We want autotest to be the framework, not > (just a) collection of tests. We also want each development team > to implement and maintain their own set of tests, using (or not) > the goodies from autotest at their discretion. > > In summary, autotest is (or is going to be) a framework that > provides: > > - A test runner, with grid/cluster support and advanced > instrumentation > - A devel library and set of utilities for test writers > - A set of pre-built images (JeOS – Just Enough OS) for > test writers > > (attached is a picture showing what we want to achieve)
Facepalm right after pressing 'y'. > > If a project has an internal library or set of utilities that can > be of general use, they can be submitted to autotest.git for > inclusion, thus reaching a broader audience. > > A short summary of the plans: > > - Tests can live anywhere and each devel team implements and > maintains their own set of tests > - Usage of the autotest library by test writers is optional > - Tests are scripts returning 0 or error (any language) > - Tests can be run individually or in sets > - Tests should run fast, our target is seconds or a few minutes > - The test runner is smart and “just works” by default > - Trivial standard output (FAIL, PASS, SKIPPED) > - Collect logs, OS data and other stuff (e.g. --record-video!) > - Skip unsupported tests based on the environment they're run > - Multiplex configurations / platforms when on the grid > - Support to run tests “in the cloud” > > There are also lots of small changes and usability improvements > in the pipeline (and feedback right now is very valuable). > > Thanks, > - Ademar > > -- > Ademar de Souza Reis Jr. > Red Hat > > ^[:wq! -- Ademar de Souza Reis Jr. Red Hat ^[:wq!
<<attachment: autotest-arch.jpg>>
