On Wed, Oct 16, 2013 at 11:26:11PM +0200, Paolo Bonzini wrote: > Il 16/10/2013 20:37, Michael S. Tsirkin ha scritto: > > Gleb, Paolo, what do you think? OK to merge kvm unit test > > into qemu? It depends on qemu anyway, in-tree will make it easier. > > Maybe someone's looking at this already? > > I think merging KVM unit tests doesn't make much sense because, with > some small exceptions, it is mostly a test or a benchmark for KVM.
But why keep them separate? They need qemu to work, don't they? What I wanted to use from kvm unit test is the infrastructure to generating the kernel binary for qemu. > What > may make sense is to have a quick way to run autotest on a QEMU tree, > with a subset of testcases that doesn't take too much time (let's say <4 > hours) That's not really reasonable for make check though. > and is more or less guaranteed to pass. That's still the main challenge. > KVM unit tests are run > by autotest, that should be enough. > > I agree with Anthony that device model code should be tested by qtest. > I'm not sure this extends to firmware interfaces, though, for two reasons: > > (1) any testcase you could have written would have likely not shown the > kind of problem that Igor and Gerd found in your previous versions. > Black box unit testing can only do so much for something as complex as a > DSDT, while black box integration testing works well. > > (2) IMO qtest's main advantage is that, at least in principle, the same > testcases could run on all the rarely-used almost-unmaintained targets > (the endianness-test already does that for example). This does not > apply to most firmware interfaces, though. > > > By the way, this advantage of qtest is also being mostly negated by the > immaturity (or sheer absence) of infrastructure. > Looking at bugs that were reported, at least these two from Igor are > probably best handled with integration tests (like autotest or Anthony's > qemu-test): > > * WS2008R2x64 BSODs with ACPI error on boot when 64bit PCI hole is > present, but it boots fine with upstream QEMU > > * hotadd CPU to guest, reboot guest, only initial CPUs are visible to guest > > > qtest could at best host some sanity checks on the ACPI tables, which > would catch the MCFG problems that Gerd reported on v5. Depends on how deep the test understands ACPI - the signature was wrong I think. Note I was testing this too - comparing tables between revisions. I just didn't notice that list of tables to test included was generated by me on piix, so MCFG wasn't tested. > Gerd also reported some segfaults, not sure how they escaped mst's > testing so I cannot judge what kind of testing could have exposed them > preemptively. > > Paolo Mostly forgot to commit mistakes. I since added a script that checks my tree is clean before build.
