On Thu, Jul 27, 2017 at 05:19:57PM +0800, Fam Zheng wrote: > On Thu, 07/27 10:14, Markus Armbruster wrote: > > This brings some advantages of "verify output with diff" to tests that > > verify with code. Improvement if it simplifies the verification code. > > > > I'd still prefer *no* verification code (by delegating the job to diff) > > for tests where I can get away wit it. > > Python based iotests can be (re)done in such a way that they print actual logs > (interactions with qtest/monitor, stdout/stderr of QEMU, etc) instead of the > current dot dot dot summary, then we automatically have diff based > verification, > no?
The python test 149 that I wrote does exactly that. There's no reason why the others couldn't do the same. > One thing I feel painful with bash iotests is how harder it is to write > complicated test scenarios such as migration, incremental backup, etc. Yes, shell is an awful language if you need non-trivial control logic or data structures > On the other hand the iotests are more difficult to debug when things go wrong > because it eats the output which, if done with shell, should be very easy to > get. Even if the python tests are not doing verify-by-diff, it should be fairly easy to wire up an env variable IOTESTS_DEBUG=1 which would force them to propagate stdout/err of all commands run (or at least save it to a log file somewhere). Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
