On Thu, Feb 20, 2014 at 02:45:03PM -0500, Sean Dague wrote: > > So I'm one of the first people to utter "if it isn't tested, it's > probably broken", however I also think we need to be realistic about the > fact that if you did out the permutations of dependencies and config > options, we'd have as many test matrix scenarios as grains of sand on > the planet. > > I do think in some ways this is unique to OpenStack, in that our > automated testing is head and shoulders above any other Open Source > project out there, and most proprietary software systems I've seen. > > So this is about being pragmatic. In our dependency testing we are > actually testing with most recent versions of everything. So I would > think that even with libvirt, we should err in that direction.
I'm very much against that, because IME, time & time again across all open source projects I've worked on, people silently introduce use of features/apis that only exist in newer versions without anyone ever noticing until it is too late. > That being said, we also need to be a little bit careful about taking > such a hard line about "supported vs. not" based on only what's in the > gate. Because if we did the following things would be listed as > unsupported (in increasing level of ridiculousness): > > * Live migration > * Using qpid or zmq > * Running on anything other than Ubuntu 12.04 > * Running on multiple nodes > > Supported to me means we think it should work, and if it doesn't, it's a > high priority bug that will get fixed quickly. Testing is our sanity > check. But it can't be considered that it will catch everything, at > least not before the heat death of the universe. I agree we should be pragmatic here to some extent. We shouldn't aim to test every single intermediate version, or every possible permutation of versions - just a representative sample. Testing both lowest and highest versions is a reasonable sample set IMHO. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
