> -----Original Message-----
> From: Zhao Yan [mailto:yan.y.z...@intel.com]
> Sent: Thursday, February 21, 2019 8:25 AM
> To: Gonglei (Arei) <arei.gong...@huawei.com>
> Cc: alex.william...@redhat.com; qemu-devel@nongnu.org;
> intel-gvt-...@lists.freedesktop.org; zhengxiao...@alibaba-inc.com;
> yi.l....@intel.com; eskul...@redhat.com; ziye.y...@intel.com;
> coh...@redhat.com; shuangtai....@alibaba-inc.com; dgilb...@redhat.com;
> zhi.a.w...@intel.com; mlevi...@redhat.com; pa...@linux.ibm.com;
> a...@ozlabs.ru; eau...@redhat.com; fel...@nutanix.com;
> jonathan.dav...@nutanix.com; changpeng....@intel.com; ken....@amd.com;
> kwankh...@nvidia.com; kevin.t...@intel.com; c...@nvidia.com;
> k...@vger.kernel.org
> Subject: Re: [PATCH 0/5] QEMU VFIO live migration
>
> On Wed, Feb 20, 2019 at 11:56:01AM +0000, Gonglei (Arei) wrote:
> > Hi yan,
> >
> > Thanks for your work.
> >
> > I have some suggestions or questions:
> >
> > 1) Would you add msix mode support,? if not, pls add a check in
> vfio_pci_save_config(), likes Nvidia's solution.
> ok.
>
> > 2) We should start vfio devices before vcpu resumes, so we can't rely on vm
> start change handler completely.
> vfio devices is by default set to running state.
> In the target machine, its state transition flow is running->stop->running.
That's confusing. We should start vfio devices after vfio_load_state, otherwise
how can you keep the devices' information are the same between source side
and destination side?
> so, maybe you can ignore the stop notification in kernel?
> > 3) We'd better support live migration rollback since have many failure
> scenarios,
> > register a migration notifier is a good choice.
> I think this patchset can also handle the failure case well.
> if migration failure or cancelling happens,
> in cleanup handler, LOGGING state is cleared. device state(running or
> stopped) keeps as it is).
IIRC there're many failure paths don't calling cleanup handler.
> then,
> if vm switches back to running, device state will be set to running;
> if vm stayes at stopped state, device state is also stopped (it has no
> meaning to let it in running state).
> Do you think so ?
>
IF the underlying state machine is complicated,
We should tell the canceling state to vendor driver proactively.
> > 4) Four memory region for live migration is too complicated IMHO.
> one big region requires the sub-regions well padded.
> like for the first control fields, they have to be padded to 4K.
> the same for other data fields.
> Otherwise, mmap simply fails, because the start-offset and size for mmap
> both need to be PAGE aligned.
>
But if we don't need use mmap for control filed and device state, they are
small basically.
The performance is enough using pread/pwrite.
> Also, 4 regions is clearer in my view :)
>
> > 5) About log sync, why not register log_global_start/stop in
> vfio_memory_listener?
> >
> >
> seems log_global_start/stop cannot be iterately called in pre-copy phase?
> for dirty pages in system memory, it's better to transfer dirty data
> iteratively to reduce down time, right?
>
We just need invoking only once for start and stop logging. Why we need to call
them literately? See memory_listener of vhost.
Regards,
-Gonglei
