Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-07 Thread Dr. David Alan Gilbert
* Alexey Kardashevskiy (a...@ozlabs.ru) wrote: > On 02/06/2014 10:24 PM, Dr. David Alan Gilbert wrote: > > * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: > >> On 02/06/2014 03:45 AM, Paolo Bonzini wrote: > >>> Il 05/02/2014 17:42, Dr. David Alan Gilbert ha scritto: > Because: > * the

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-06 Thread Alexey Kardashevskiy
On 02/07/2014 10:49 AM, Paolo Bonzini wrote: > Il 06/02/2014 04:10, Alexey Kardashevskiy ha scritto: Ok, I thought Alexey was saying we are not redirtying that handful of pages. >> >> Every iteration we read the dirty map from KVM and send all dirty pages >> across the stream. > > But w

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-06 Thread Alexey Kardashevskiy
On 02/06/2014 10:24 PM, Dr. David Alan Gilbert wrote: > * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: >> On 02/06/2014 03:45 AM, Paolo Bonzini wrote: >>> Il 05/02/2014 17:42, Dr. David Alan Gilbert ha scritto: Because: * the code is still running and keeps redirtying a small handful

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-06 Thread Paolo Bonzini
Il 06/02/2014 04:10, Alexey Kardashevskiy ha scritto: >> > Ok, I thought Alexey was saying we are not redirtying that handful of >> > pages. > > Every iteration we read the dirty map from KVM and send all dirty pages > across the stream. But we never finish because qemu_savevm_state_pending is o

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-06 Thread Dr. David Alan Gilbert
* Alexey Kardashevskiy (a...@ozlabs.ru) wrote: > On 02/06/2014 03:45 AM, Paolo Bonzini wrote: > > Il 05/02/2014 17:42, Dr. David Alan Gilbert ha scritto: > >> Because: > >> * the code is still running and keeps redirtying a small handful of > >> pages > >> * but because we've underestimated

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-05 Thread Alexey Kardashevskiy
On 02/06/2014 03:45 AM, Paolo Bonzini wrote: > Il 05/02/2014 17:42, Dr. David Alan Gilbert ha scritto: >> Because: >> * the code is still running and keeps redirtying a small handful of >> pages >> * but because we've underestimated our available bandwidth we never stop >> it and just

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-05 Thread Paolo Bonzini
Il 05/02/2014 17:42, Dr. David Alan Gilbert ha scritto: Because: * the code is still running and keeps redirtying a small handful of pages * but because we've underestimated our available bandwidth we never stop it and just throw those pages across immediately Ok, I thought Alexey

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-05 Thread Dr. David Alan Gilbert
* Paolo Bonzini (pbonz...@redhat.com) wrote: > Il 05/02/2014 10:09, Dr. David Alan Gilbert ha scritto: > >I think the case Alexey is hitting is: > > 1 A few dirtied pages > > 2 but because of the hpratio most of the data is actually zero > > - indeed most of the target-page sized chunks are

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-05 Thread Paolo Bonzini
Il 05/02/2014 10:09, Dr. David Alan Gilbert ha scritto: I think the case Alexey is hitting is: 1 A few dirtied pages 2 but because of the hpratio most of the data is actually zero - indeed most of the target-page sized chunks are zero 3 Thus the data compresses very heavily 4 Whe

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-05 Thread Dr. David Alan Gilbert
* Paolo Bonzini (pbonz...@redhat.com) wrote: > Il 04/02/2014 23:17, Alexey Kardashevskiy ha scritto: > > Well, it will fix it in my particular case but in a long run this does > > not > > feel like a fix - there should be a way for migration_thread() to know > > that > > ram_s

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Paolo Bonzini
Il 04/02/2014 23:17, Alexey Kardashevskiy ha scritto: >> Well, it will fix it in my particular case but in a long run this does not >> feel like a fix - there should be a way for migration_thread() to know that >> ram_save_iterate() sent all dirty pages it had to send, no? > > No, because new pa

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Alexey Kardashevskiy
On 02/05/2014 01:00 AM, Paolo Bonzini wrote: > Il 04/02/2014 13:16, Alexey Kardashevskiy ha scritto: >> On 02/04/2014 11:07 PM, Paolo Bonzini wrote: >>> Il 04/02/2014 12:59, Alexey Kardashevskiy ha scritto: >> With the default throttling of 32 MiB/s, bandwidth must be something >> like

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Paolo Bonzini
Il 04/02/2014 13:16, Alexey Kardashevskiy ha scritto: On 02/04/2014 11:07 PM, Paolo Bonzini wrote: Il 04/02/2014 12:59, Alexey Kardashevskiy ha scritto: With the default throttling of 32 MiB/s, bandwidth must be something like 33000 (expressed in bytes/ms) with the default settings, and then m

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Alexey Kardashevskiy
On 02/04/2014 11:07 PM, Paolo Bonzini wrote: > Il 04/02/2014 12:59, Alexey Kardashevskiy ha scritto: >>> > With the default throttling of 32 MiB/s, bandwidth must be something like >>> > 33000 (expressed in bytes/ms) with the default settings, and then >>> max_size >>> > should be 33000*3*10^9 / 10

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Paolo Bonzini
Il 04/02/2014 12:59, Alexey Kardashevskiy ha scritto: > With the default throttling of 32 MiB/s, bandwidth must be something like > 33000 (expressed in bytes/ms) with the default settings, and then max_size > should be 33000*3*10^9 / 10^6 = 600. Where is my computation wrong? migrate_max_d

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Alexey Kardashevskiy
On 02/04/2014 09:46 PM, Paolo Bonzini wrote: > Il 04/02/2014 08:15, Alexey Kardashevskiy ha scritto: >> So. migration_thread() gets dirty pages number, tries to send them in a >> loop but every iteration resets the number of pages to 96 and we start >> again. After several tries we cross BUFFER_DEL

Re: [Qemu-devel] migration: broken ram_save_pending

2014-02-04 Thread Paolo Bonzini
Il 04/02/2014 08:15, Alexey Kardashevskiy ha scritto: So. migration_thread() gets dirty pages number, tries to send them in a loop but every iteration resets the number of pages to 96 and we start again. After several tries we cross BUFFER_DELAY timeout and calculate new @max_size and if the host