On Fri, Nov 06, 2015 at 03:48:11PM +0000, Dr. David Alan Gilbert wrote: > * Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote: > > > > Where we have iterable, but non-postcopiable devices (e.g. htab > > > or block migration), complete them before forming the 'package' > > > but with the CPUs stopped. This stops them filling up the package. > > > > That helps and the migration suceeds now when I switch to postcopy > > immediately after starting the migration. > > Excellent. > > > However after postcopy migration, when I attempt to start an incoming > > instance again to migrate the guest back, I see this failure: > > > > qemu-system-ppc64: cannot set up guest memory 'ppc_spapr.ram': Cannot > > allocate memory > > > > The same doesn't happen with normal migration. > > Huh that's fun; that's the original source guest that's running out of RAM? > It's original QEMU should be gone by that point.
Yes, the original source QEMU is gone, but there is not enough memory left in the host to start another incoming QEMU instance because... At the beginning ----------------- $ grep -i mem /proc/meminfo MemTotal: 132816832 kB MemFree: 128781632 kB MemAvailable: 131668224 kB After starting the guest (-m 64G,slots=32,maxmem=128G) ------------------------ $ grep -i mem /proc/meminfo MemTotal: 132816832 kB MemFree: 124866880 kB MemAvailable: 127753728 kB After starting the destination instance (incoming) ------------------------------------------------- $ grep -i mem /proc/meminfo MemTotal: 132816832 kB MemFree: 122514880 kB MemAvailable: 125401920 kB After postcopy migration completes ---------------------------------- $ grep -i mem /proc/meminfo MemTotal: 132816832 kB MemFree: 55150592 kB MemAvailable: 58037888 kB After terminating the source instance ------------------------------------- $ grep -i mem /proc/meminfo MemTotal: 132816832 kB MemFree: 59432448 kB MemAvailable: 62319872 kB So as you can see, postcopy migration will result in guest claiming its entire RAM memory from host. This doesn't happen during normal migration. The above results are with the additional patch you sent in this thread and I was switching to postcopy migration immediately after starting the migration. Without this patch, when I delay the switching to postcopy migration a bit, I see the free memory as below when postcopy migration completes. $ grep -i mem /proc/meminfo MemTotal: 132816832 kB MemFree: 85018176 kB MemAvailable: 87937024 kB Regards, Bharata.
