On 12/29/2011 06:26 PM, Anthony Liguori wrote: > On 12/28/2011 11:21 AM, Avi Kivity wrote: >> On 12/28/2011 06:42 PM, Anthony Liguori wrote: >>>> In fact using linux as a guest negates that. First of all, which >>>> linux >>>> version? if it's fixed, you'll eventually miss functionality and >>>> need to >>>> migrate. If it keeps changing, so does your test, and it will keep >>>> breaking. >>> >>> >>> The kernel is a git submodule so it's a very specific version. Yes, >>> we may need to bump the version down the road and obviously, if we >>> have to change any tests in the process, we can. >> >> Having a full linux source as part of the build process detracts >> somewhat from the advantages here. > > Except that binaries can be made available. The initramfs is > currently 512k while the kernel is about 5MB. > > OTOH, a Fedora install runs in the multiple GB. Having > kernels/initramfs for the dozen target types that QEMU supports is > reasonable. Having Fedora installs for all of them probably isn't for > most people.
Then use DSL for the ping test. > >>> But there's certain things that I still consider to be unit testing >>> (like basic networking tests) that I don't want to have to write with >>> qtest. I'm not up for writing a TCP/IP stack in Python... >> >> A ping test is not a unit test. >> >> The ping test is already covered by kvm-autotest; just set up a config >> that runs just that; after the initial run you'll have a guest installed >> so it'll be quick. If we have a DSL guest it'll even be very quick. > > Achieving this with kvm-autotest is not easy. I doubt that DSL has an > automated install mechanism and the step file based guest installation > is fincky at best. > > When we had the kvm-autotest day, none of the folks that tried to set > it up who had never set it up before had something working by the end > of the day. And that was with lmr available on IRC to answer questions. > > There is a huge setup cost with kvm-autotest. That needs to be fixed, not workarounded. I previously advised Lucas not to spend too much time on this, since the audience is limited and it would be hard to amortize the effort. Now I see that this was wrong. > >> To test the various NIC emulations, you don't need a full TCP stack, >> just like you didn't need to write an NTP implementation for qtest's rtc >> test. Instead you just poke values until it sends out a packet. If you >> want to test virtio-net with both direct and indirect buffers, you can >> only do that with a unit test, you can't do it using a full linux guest >> since it has its own ideas of when to use indirects and when to avoid >> them (for example, it may choose to always avoid them). > > I am not advocating that we don't write proper unit tests... Right, and qtest is dangerously close to making writing tests fun. But qemu-test is a wierd cross between qtest and kvm-autotest. > >>> Acceptance testing is, "does Windows boot", "can I create three >>> virtio-serial devices". >>> >>> Obviously, part of acceptance testing is, "does this set of functional >>> tests pass". >> >> Seems like a very blurry line. Especially as the functional test is >> weaker than either qtest and kvm-autotest. I now have to agree with the >> others that it's duplicate functionality. Does it really matter whether >> you're creating an image by compiling Linux and assembling and >> initramfs, or by downloading Fedora.iso and installing it? > > Yes. One of them is built entirely from source and the other is doing > something meant to be done on bare metal. > > How do you test ARM guests with kvm-autotest? There is no such thing > as an ISO installer for ARM. Are we going to download random images > from the internet? Do exactly what you plan to do with qemu-test. > > Even if there is an ISO, installing a guest with TCG will take many > hours. My local qemu-test repo can now fully bootstrap an initramfs > for non-x86 targets (which involves building gcc and uClibc). Start > to finish it takes about 30 minutes regardless of the target (since > the build runs natively). You can skip the install test and only do the boot test. Seed your disk with images from qemu.org or by running the install. kvm-autotest does this. > >> Would you add live migration testing to qemu-test? If yes, you're >> duplicating some more. If not, you're not doing functional or coverage >> tests for that functionality. > > I would add live migration schema testing, absolutely. I wouldn't add > an acceptance test though. > > An acceptance test with live migration would be creating a guest, > running it, and then seeing it live migration worked. Then it's duplicate functionality. It's already been done, but the setup procedure is hard, so let's do it again. > >>> I don't want to write a TCP/IP stack. We aren't just grabbing a >>> random distro kernel. We're building one from scratch configured in a >>> specific way. >> >> How does that help? > > Not sure I understand the question. > In what way is your specifically configured kernel's TCP stack better than the random distro's kernel's? -- error compiling committee.c: too many arguments to function
