On 09/12/2010 05:46 AM, Avi Kivity wrote:
On 09/11/2010 05:04 PM, Anthony Liguori wrote:
Today, live migration only works when using shared storage that is fully
cache coherent using raw images.
The failure case with weak coherent (i.e. NFS) is subtle but
nontheless still
exists. NFS only guarantees close-to-open coherence and when
performing a live
migration, we do an open on the source and an open on the
destination. We
fsync() on the source before launching the destination but since we
have two
simultaneous opens, we're not guaranteed coherence.
This is not necessarily a problem except that we are a bit gratituous
in reading
from the disk before launching a guest. This means that as things
stand today,
we're guaranteed to read the first 64k of the disk and as such, if a
client
writes to that region during live migration, corruption will result.
The second failure condition has to do with image files (such as
qcow2). Today,
we aggressively cache metadata in all image formats and that cache is
definitely
not coherent even with fully coherent shared storage.
In all image formats, we prefetch at least the L1 table in open()
which means
that if there is a write operation that causes a modification to an
L1 table,
corruption will ensue.
This series attempts to address both of these issue. Technically, if
a NFS
client aggressively prefetches this solution is not enough but in
practice,
Linux doesn't do that.
I think it is unlikely that it will, but I prefer to be on the right
side of the standards.
I've been asking around about this and one thing that was suggested was
acquiring a file lock as NFS requires that a lock acquisition drops any
client cache for a file. I need to understand this a bit more so it's
step #2.
Why not delay image open until after migration completes? I know
your concern about the image not being there, but we can verify that
with access(). If the image is deleted between access() and open()
then the user has much bigger problems.
3/3 would still be needed because if we delay the open we obviously can
do a read until an open.
So it's only really a choice between invalidate_cache and delaying
open. It's a far less invasive change to just do invalidate_cache
though and it has some nice properties.
Regards,
Anthony Liguori
Note that on NFS, removing (and I think chmoding) a file after it is
opened will cause subsequent data access to fail, unlike posix.
