* Chunguang Li (lichungu...@hust.edu.cn) wrote: > > > > > -----原始邮件----- > > 发件人: "Dr. David Alan Gilbert" <dgilb...@redhat.com> > > 发送时间: 2016年9月26日 星期一 > > 收件人: "Chunguang Li" <lichungu...@hust.edu.cn> > > 抄送: qemu-devel@nongnu.org, amit.s...@redhat.com, pbonz...@redhat.com, > > stefa...@redhat.com, quint...@redhat.com > > 主题: Re: [Qemu-devel] Migration dirty bitmap: should only mark pages as > > dirty after they have been sent > > > > * Chunguang Li (lichungu...@hust.edu.cn) wrote: > > > Hi all! > > > I have some confusion about the dirty bitmap during migration. I have > > > digged into the code. I figure out that every now and then during > > > migration, the dirty bitmap will be grabbed from the kernel space through > > > ioctl(KVM_GET_DIRTY_LOG), and then be used to update qemu's dirty bitmap. > > > However I think this mechanism leads to resendness of some NON-dirty > > > pages. > > > > > > Take the first iteration of precopy for instance, during which all the > > > pages will be sent. Before that during the migration setup, the > > > ioctl(KVM_GET_DIRTY_LOG) is called once, so the kernel begins to produce > > > the dirty bitmap from this moment. When the pages "that haven't been > > > sent" are written, the kernel space marks them as dirty. However I don't > > > think this is correct, because these pages will be sent during this and > > > the next iterations with the same content (if they are not written again > > > after they are sent). It only makes sense to mark the pages which have > > > already been sent during one iteration as dirty when they are written. > > > > > > > > > Am I right about this consideration? If I am right, is there some advice > > > to improve this? > > > > I think you're right that this can happen; to clarify I think the > > case you're talking about is: > > > > Iteration 1 > > sync bitmap > > start sending pages > > page 'n' is modified - but hasn't been sent yet > > page 'n' gets sent > > Iteration 2 > > sync bitmap > > 'page n is shown as modified' > > send page 'n' again > > > > Yes，this is right the case I am talking about. > > > So you're right that is wasteful; I guess it's more wasteful > > on big VMs with slow networks where the length of each iteration > > is large. > > I think this is "very" wasteful. Assume the workload writes the pages dirty > randomly within the guest address space, and the transfer speed is constant. > Intuitively, I think nearly half of the dirty pages produced in Iteration 1 > is not really dirty. This means the time of Iteration 2 is double of that to > send only really dirty pages.
Yes, it's probably pretty bad; and we really need to do something like split the sync into smaller chunks; there are other suggestions for how to improve it (e.g. there's the page-modification-logging changes). However, I don't think you usually get really random writes, if you do precopy rarely converges at all, because even without your observation it changes lots and lots of pages. Dave > Thanks, > > Chunguang > > > > > Fixing it is not easy, because you have to be really careful > > never to miss a page modification, even if the page is sent > > about the same time it's dirtied. > > > > One way would be to sync the dirty log from the kernel > > in smaller chunks. > > > > Dave > > > > > > > > > > Thanks, > > > Chunguang Li > > -- > > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK > > > -- > Chunguang Li, Ph.D. Candidate > Wuhan National Laboratory for Optoelectronics (WNLO) > Huazhong University of Science & Technology (HUST) > Wuhan, Hubei Prov., China > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
