On 08/05/19 06:39, Peter Xu wrote: >> The disadvantage of this is that you won't clear in the kernel those >> dirty bits that come from other sources (e.g. vhost or >> address_space_map). This can lead to double-copying of pages. >> >> Migration already makes a local copy in rb->bmap, and >> memory_region_snapshot_and_clear_dirty can also do the clear. Would it >> be possible to invoke the clear using rb->bmap instead of the KVMSlot's >> new bitmap? > > Actually that's what I did in the first version before I post the work > but I noticed that there seems to have a race condition with the > design. The problem is we have multiple copies of the same dirty > bitmap from KVM and the race can happen with those multiple users > (bitmaps of the users can be a merged version containing KVM and other > sources like vhost, address_space_map, etc. but let's just make it > simpler to not have them yet).
I see now. And in fact the same double-copying inefficiency happens already without this series, so you are improving the situation anyway. Have you done any kind of benchmarking already? Thanks, Paolo > (1) page P is written by the guest (let's version its data as P1), > its dirty bit D is set in KVM > > (2) QEMU sync dirty log, fetch D for page P (which is set). D is > not cleared in KVM due to MANUAL_PROTECT cap. > > (3) QEMU copies the D bit to all users of dirty bmap (MIGRATION and > VGA as example). > > (4) MIGRATION code collects bit D in migration bitmap, clear it, > send CLEAR to KVM to clear the bit on remote (then D in KVM is > cleared too), send the page P with content P1 to destination. > CAUTION: when reach here VGA bitmap still has the D bit set. > > (5) page P is written by the guest again (let's version its data as > P2), bit D set again in KVM. > > (6) VGA code collectes bit D (note! we haven't synced the log again > so this is the _old_ dirty bit came from step 3 above) in VGA > bitmap, clear it, send CLEAR to KVM to clear the bit on remote. > Then D bit in KVM is cleared again. Until here, D bit in all > bitmaps are cleared (MIGRATION, VGA, KVM). > > (7) page P is never written again until migration completes (no > matter whether we sync again D bit will be cleared). > > (8) On destination VM page P will contain content P1 rather than P2, > this is data loss...
