> From: Qemu-devel [mailto:qemu-devel- > bounces+liang.z.li=intel....@nongnu.org] On Behalf Of Juan Quintela > Sent: Wednesday, May 04, 2016 7:20 PM > To: QEMU Developer > Subject: [Qemu-devel] Migration ToDo list (a.k.a. Rant) > > > Hi > > I am lots of times asked about what is the ToDo list for migration, that was > on > my head, and random notes over my desk, so, trying some organization (Yes, > I would put this in the wiki). > >
Is it proper to add: 'speed up live migration by skipping free pages' ? Liang > - migration thread on reception > would make trivial to do other things while receiving, and would make > postcopy easier also (I was going to put much easier, but postcopy is > never easy). > > - migration capabilities and parameters > this is a mess. Not, is worse than that. I don't know who is to > blame here, but something needs to be done: > > void qmp_migrate_set_parameters(bool has_compress_level, > int64_t compress_level, > bool has_compress_threads, > int64_t compress_threads, > bool has_decompress_threads, > int64_t decompress_threads, > bool has_x_cpu_throttle_initial, > int64_t x_cpu_throttle_initial, > bool has_x_cpu_throttle_increment, > int64_t x_cpu_throttle_increment, > bool has_multifd_threads, > int64_t multifd_threads, > Error **errp) > > > > Can we move this to an array of structs, please, pretty please? > I think that for this one, the blame is on qmp > > but we can continue: > > migrate > migrate_cancel > migrate_incoming > migrate_start_postcopy > > Not a lot to do until here > > migrate_set_capability > Minor nickpit, if it only allow booleans, "migrate_set_capability > x-multifd", > should be an equivalent of "migrate_set_capability x-multifd on" > > migrate_set_cache_size > migrate_set_downtime > migrate_set_speed > This three should be claimed obsolete, deprecated, whatever, and > make it on top of next one > > migrate_set_parameter > > Now to read the migration information: > > migrate_capabilities > good > migrate_parameters > good > migrate_cache_size > good, but we are missing migrate_speed and migrate_downtime, see > why I want it be inside migrate_set_parameters > > migrate > now, this is ..... weird? We put here lots of information, and > this is basically the only way to put information out. To make > things more interesting, the values change meaning during > migration, and the fields it shows change also over time. > > - info migrate > This deserves its own item. Lets see a typical output > > (qemu)info migrate > > capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: > off > compress: off events: off postcopy-ram: off x-multifd: on > > Aha, we have the capabilities, but not the parameters. This is > historical, I know, but don't belong here. > > Migration status: completed > ok > total time: 1621 milliseconds > ok > downtime: 208 milliseconds > ok > setup: 9 milliseconds > ok > > transferred ram: 609708 kbytes > kilo bytes, not pages > > throughput: 27.64 mbps > but we measure bandwidth is megabytes by second > previous one was kylobytes > > remaining ram: 0 kbytes > > total ram: 2106180 kbytes > this amount don't change. I can understand why it was here. > > duplicate: 452528 pages > name is historical. It really means pages filled with the same > characeter. Althought in practical effects it means zero pages > > skipped: 0 pages > Even I don't remember what this means. > normal: 151064 pages > This is normal pages that we have sent, i.e. pages that are not zero > pages nor skipped pages. Notice that we have put here pages, not > bytes, not kilobytes, but pages. > > normal bytes: 604256 kbytes > Don't worry, we put for you the same number as kilobytes. > > dirty sync count: 11 > Number of iterations over the full ram. Yes, I know, we are very, > very bad at naming. > > And we still have more optional information that appears if we are doing > block migration, xbzrle, compression, rdma, etc, etc. > > We need to decide some units also internal. Some things are in bytes, some > are in kilobytes, some are in pages. Some are in host pages, or guest pages, > or who knows :-( > > > - Block migration (the migration/block.c one). This is the bastard > child of migration. Much less tested, we should make a decision > about letting it live or deprecating it. Things needed from memory: > - functions should return the same values than ram.c > some functions don't have "exact" values, and return 1 when there > are more than one block dirty, etc, etc > - if we continue maintaing it, allowing it to have _some_ shared > devices and some non shared ones, insntead of everything? > > - RDMA: Another step child > > This is really, really weird. We don't use the normal infrastructure > for RDMA, we use the ram_control_* stuff. We should really move to > use the normal stuff here. > > - autoconverge code: This could be used outside of migration (i.e. just > to slow down a guess). We should really do some measurement here to > see how useful it is for migration. If the guest is using lots of > memory dirtying, we end having to throttle the guest 90% or so :-( > > - xbzrle. We only have one cache, we should decide how to work with > this for multithread/compression. > > - When we do migration, we have spaguetti code to decide if: > * it is a zero page > * it is a duplicated page > * it is a xbzrle page > * it is a compressed page > And as the code is written, it is not trivial to add new "options". I > think that we should "re-think" what combinations are allowed an which > ones make nosense. > > - savevm and migration: they use two different paths for not really good > reason. We should really abstract this to a single code path. > We always forget the savevm one when we do changes. > > - error handling. Every function should return an error. Every > function should return an error. > > - qemu_get_buffer() don't give one error if there is nothing to read, > sniff. > > - Multipage support: Welcome to the XXI century. Now almost all > architectures have HugePages. And other have different sized pages > (in PPC is not strange that page size of host and guest differ). We > have work to do here. For starters, sending Huge pages as one chunk > will make TransparentHugePages happier. > > - Bitmaps. Related with previous one. We should really be better about > walking them and about synchronising them between qemu/kernel. > > - COLO: We need to integrate it. > > I will continue the rant at some other point O:-) Just now I need to left for > the bar. > > Thanks for your attention, Juan. > > PD. I just looked while I wrote this to the channel code from Daniel, a step > on the right direction. >
