On Fri, Mar 9, 2012 at 2:00 PM, Ademar Reis <ar...@redhat.com> wrote: > On Fri, Mar 09, 2012 at 09:41:05AM +0000, Stefan Hajnoczi wrote: >> On Thu, Mar 8, 2012 at 11:51 PM, Ademar Reis <ar...@redhat.com> wrote: >> > On Thu, Mar 08, 2012 at 05:21:44PM -0600, Anthony Liguori wrote: >> >> On 03/08/2012 04:24 PM, Ademar Reis wrote: >> >> >On Thu, Mar 08, 2012 at 03:24:15PM -0600, Anthony Liguori wrote: >> >> >>On 03/08/2012 03:02 PM, Ademar Reis wrote: >> >> >>>On Thu, Mar 08, 2012 at 01:16:58PM -0600, Anthony Liguori wrote: >> >> >>>>On 03/08/2012 11:59 AM, Ademar Reis wrote: >> >> >>> - QE will be alienated from the qemu test effort. There will be >> >> >>> no integration between the QE efforts and the maintenance of >> >> >>> the qemu developer-level tests. >> >> >> >> >> >>I think we're a pretty friendly and open community :-) There is no >> >> >>reason that QE should be "alienated" unless folks are choosing not >> >> >>to participate upstream. >> >> > >> >> >For the exact same reasons you as a developer don't want to >> >> >implement tests inside autotest, QE won't want to implement tests >> >> >for qemu.git. It's out of their comfort zone, just put yourself >> >> >on their shoes. >> >> >> >> This is a really, really poor argument and I hope I don't need to go >> >> into details of why. If the primary reason for libautotest is so >> >> the people writing tests for QEMU can avoid actually working with >> >> the developers of QEMU... we've got a problem. >> > >> > No, one of the benefits of having libautotest is to *collaborate* >> > with QE. I'll explain again: >> > >> > - As a qemu developer, I don't want to spend my time learning and >> > getting involved in autotest, which is a complex QE project >> > (I heard this numerous times). >> > >> > - As a Quality Engineer, I don't want to invest my time learning >> > and getting involved into upstream qemu to test HEAD. >> >> I think this is the key point of the whole discussion - most of the >> other topics have been distractions. Both communities do testing but >> we test different things and have different priorities. >> >> For me this has been the big realization from this discussion. I felt >> kvm-autotest and qemu should share tests. I was pushing for that but >> after following this thread I don't think it makes sense, here's why: >> >> The Quality Engineer you describe is not a QEMU upstream QE, instead >> the QE has a broader and more downstream focus. (This is why >> comparisons with WebKit or other upstream projects doing testing are >> not valid comparisons.) > > Lucas, Cleber and the others red-hatters should remembers this > from my internal presentation, it was the first point I made: > QE and Developers have very different goals and interests. > > Which is why we're pushing all these changes in autotest. We see > opportunities for collaboration, but we do realize the difference. > > And look: Lucas and Cleber are not QE, they're developers working > on the autotest framework/library/whatever. We'll need similar > positions inside qemu as the test infra-structure grows.
I don't understand this last paragraph. If qemu.git upstream was doing full-scale QE it would work fine because the differences that I've described and you also have pointed out would be absent. >> I think what's much more valuable is for qemu.git tests to be easily >> hooked into autotest. That way you get access to the testing that the >> qemu community is doing for free. > > We get this by default in our plans: any application that can be > run and returns 0/error can be run as a test inside autotest. Yes, and I think that part of the plan makes sense. > What we're asking is the *possibility* of a developer using > libautotest when writting a complex test. > > - We're not asking everybody to use autotest > - We're not saying autotest is better for everything > - We're not requiring you to install autotest on your machine > (even though the process will be trivial) > > - We're asking that, if a developer is going to write a test > together with his patch to submit to qemu.git, he should be > allowed to use libautotest at his own discretion... > - Ditto for using the test-runner: even if tests are simple > scripts, there will be benefits in using our optional > test-runner during development. > > This is how I see the interaction with autotest-based tests: > > $ make check-full > tests.d/my-test-using-qemu-tests.sh -- PASS > tests.d/trivial-test.sh -- PASS > tests.d/my-test-using-libautotest -- SKIP > ... > > $ yum install libautotest > # (or whatever the bootstrap procedure is) > $ make check-full > tests.d/my-test-using-qemu-tests.sh -- PASS > tests.d/trivial-test.sh -- PASS > tests.d/my-test-using-libautotest -- PASS > ... > > But for some reason that I still don't understand, the simple > acceptance of the optional dependency of libautotest in qemu is > seen as a bad thing. I don't mind a libvirttest but I've expressed concerns that that if qemu.git has it's own test library then the incentive to use or contribute to an external library is not there. That's why I suggested a less ambitious approach. Stefan
