* Alexey Perevalov (a.pereva...@samsung.com) wrote: > Hi, David, Andrea and Mike
Hi Alexey, > The problem I want to discuss it's 1G hugepage based VM and post copy live > migration. > > I would like to know your opinion on following approach of avoiding such > problem: > Once we have mmap'ed area through 1G hugetlbfs, remap physical pages > with /dev/mem. It will be 2 types of vmas mapped to the same PFN. > Register userfaultfd for newly obtained virtual > addresses, it could reduce granularity of pages and reduce downtime per > one 1G page. So registering userfaultfd for 2Mb, when the real hugepage > was 1G, I think, could help. > > Current postcopy implementation in QEMU allows to make live migration > from 1G based hugepage VM to 2Mb based hugepages VM (sanity checks prevent > it). > > Also I checked, it's possible to remap through /dev/mem and get PFN > based vmas, register userfaultfd (with allowance in vma_can_userfault) > and finally make UFFDIO_COPY with allowing PFN based vmas in __mcopy_atomic. > > But there are a lot of drawback of such approach: > First of all it's /dev/mem interface. Need to provide full access > (kernel w/o CONFIG_STRICT_DEVMEM) and need to disable PAT. > The second drawback, maybe I just didn't find possibility to remap > hugepages again, but mmap of /dev/mem character driver maps 4Kb pages. > I don't know how THP could help here, but madvise with MADV_HUGEPAGE > didn't. So 4Kb is not exactly what needed, due to overhead of > encapsulation summary downtime is worse than in other cases. > It would be great to have interface to obtain new virtual address based > on existing PFN, but for hugepages. Yes, and I think as well on some architectures there can be cache problems from mapping the same page in two addresses unless we're careful. I think to do this we'd basically need the kernel to set up something similar to what you're saying, but without the mess of having to go via /dev/mem. Ideally it would all happen magically when I mark a hugetlb page as userfault and start UFFDIO_COPYing in 4kb pages; but I can imagine perhaps some more syscalls needed to tell it to do it. I've no idea how hard that is to do though. Dave > Honestly, I can't find another use case for this feature. > > > -- > > BR > Alexey -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
