Today, live migration only works when using shared storage that is fully cache coherent using raw images.
The failure case with weak coherent (i.e. NFS) is subtle but nontheless still exists. NFS only guarantees close-to-open coherence and when performing a live migration, we do an open on the source and an open on the destination. We fsync() on the source before launching the destination but since we have two simultaneous opens, we're not guaranteed coherence. This is not necessarily a problem except that we are a bit gratituous in reading from the disk before launching a guest. This means that as things stand today, we're guaranteed to read the first 64k of the disk and as such, if a client writes to that region during live migration, corruption will result. The second failure condition has to do with image files (such as qcow2). Today, we aggressively cache metadata in all image formats and that cache is definitely not coherent even with fully coherent shared storage. In all image formats, we prefetch at least the L1 table in open() which means that if there is a write operation that causes a modification to an L1 table, corruption will ensue. This series attempts to address both of these issue. Technically, if a NFS client aggressively prefetches this solution is not enough but in practice, Linux doesn't do that. I need some help with the qcow2 metadata invalidation. We need to delay the loading of the l1 and the reference count table but we only do this synchronously today. I think we can just do this on demand but I'd still like a second opinion.
