* Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote:
> On Fri, Nov 06, 2015 at 03:48:11PM +0000, Dr. David Alan Gilbert wrote:
> > * Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote:
> >
> > > > Where we have iterable, but non-postcopiable devices (e.g. htab
> > > > or block migration), complete them before forming the 'package'
> > > > but with the CPUs stopped. This stops them filling up the package.
> > >
> > > That helps and the migration suceeds now when I switch to postcopy
> > > immediately after starting the migration.
> >
> > Excellent.
> >
> > > However after postcopy migration, when I attempt to start an incoming
> > > instance again to migrate the guest back, I see this failure:
> > >
> > > qemu-system-ppc64: cannot set up guest memory 'ppc_spapr.ram': Cannot
> > > allocate memory
> > >
> > > The same doesn't happen with normal migration.
> >
> > Huh that's fun; that's the original source guest that's running out of RAM?
> > It's original QEMU should be gone by that point.
>
> Yes, the original source QEMU is gone, but there is not enough memory left
> in the host to start another incoming QEMU instance because...
Ah, so this is ping-pong on one host?
> At the beginning
> -----------------
> $ grep -i mem /proc/meminfo
> MemTotal: 132816832 kB
> MemFree: 128781632 kB
> MemAvailable: 131668224 kB
>
> After starting the guest (-m 64G,slots=32,maxmem=128G)
> ------------------------
> $ grep -i mem /proc/meminfo
> MemTotal: 132816832 kB
> MemFree: 124866880 kB
> MemAvailable: 127753728 kB
>
> After starting the destination instance (incoming)
> -------------------------------------------------
> $ grep -i mem /proc/meminfo
> MemTotal: 132816832 kB
> MemFree: 122514880 kB
> MemAvailable: 125401920 kB
>
> After postcopy migration completes
> ----------------------------------
> $ grep -i mem /proc/meminfo
> MemTotal: 132816832 kB
> MemFree: 55150592 kB
> MemAvailable: 58037888 kB
>
> After terminating the source instance
> -------------------------------------
> $ grep -i mem /proc/meminfo
> MemTotal: 132816832 kB
> MemFree: 59432448 kB
> MemAvailable: 62319872 kB
>
> So as you can see, postcopy migration will result in guest claiming
> its entire RAM memory from host. This doesn't happen during normal migration.
I'll try and see if I can replicate this.
Can you :
1) show me the command line you're using
2) Show me /proc/pid/smaps for the destination qemu
3) Turn on the trace postcopy_place_page_zero
the theory is that most of your pages should end up as zero pages
and not be allocated.
Dave
> The above results are with the additional patch you sent in this thread
> and I was switching to postcopy migration immediately after starting the
> migration. Without this patch, when I delay the switching to postcopy
> migration a bit, I see the free memory as below when postcopy migration
> completes.
>
> $ grep -i mem /proc/meminfo
> MemTotal: 132816832 kB
> MemFree: 85018176 kB
> MemAvailable: 87937024 kB
>
> Regards,
> Bharata.
>
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
