From: "Dr. David Alan Gilbert" <dgilb...@redhat.com> This is the 8th cut of my version of postcopy.
The userfaultfd linux kernel code is now in the upstream kernel tree, and so 4.3-rc3 can be used without modification. This qemu series can be found at: https://github.com/orbitfp7/qemu.git on the wp3-postcopy-v8 tag Testing status: * Tested heavily on x86 * Smoke tested on aarch64 (so it does work on different page sizes) * Power is unhappy for me (but gets further than the htab problem v7 used to have) (I get a kvm run failed) Note that patches: 3 Init page size in qtest 10 Use RAMBlock rather than Memory Region 14,15 (splitting ram find and save block) 36 Split out end of migration code have also been posted seperately during the last month and can be taken separately from this series. This work has been partially funded by the EU Orbit project: see http://www.orbitproject.eu/about/ v8 Huge page changes The precopy phase is now allowed to keep transparent-huge-pages enabled, although these maybe split in the discard phase. A change to the discard process now causes discards for unsent (as well as redirtied) pages; the combination of these changes means that the destination starts running with many of the precopy pages as huge pages, resulting in a significant performance benefit. This change adds one more state ('discard') which is entered by the destination on reception of the 1st discard command. Add global_state_store for postcopy Moved postcopy_state back out of MigrationIncomingState During the end of the main migration thread reading state the postcopy state is read to see who should cleanup; in failure conditions there was a race between the state being read and the freeing of the MIS Stop calling the iterate method for non-postcopiable devices during postcopy Helps fix Power (thanks for Bharata for helping debug this) Review comment fixes rework of migration command parsing rework of postcopy_chunk_hostpages rework of discard code & protocol into start/length rather than start/end rename qemu_get_buffer_less_copy -> qemu_get_buffer_in_place split 'Postcopy end in migration thread' patch into two split of MIG_RP_MSG_REQ_PAGES into subtype with name Added comments documenting postcopy_state's use lots of other minor fixups Notes: I kept the mlock support (users are saying they wanted migration/postcopy with mlock) I'm keeping the x- for now, until the libvirt interface gets finalised. There are two checkpatch errors, that I don't think are right to change: a) a 'typedef enum' it wants split - that's the way we do all our enums and would force a dummy name for the enum. b) A complaint about postcopy_ram_discard_version = 0 that's a global static; I could get rid of it by making my version 1, but it doesn't seem right to omit the '= 0 ' for a version constant. Dave Dr. David Alan Gilbert (54): Add postcopy documentation Provide runtime Target page information Init page sizes in qtest Move configuration section writing qemu_ram_block_from_host Rename mis->file to from_src_file Add qemu_get_buffer_in_place to avoid copies some of the time Add wrapper for setting blocking status on a QEMUFile Add QEMU_MADV_NOHUGEPAGE migration/ram.c: Use RAMBlock rather than MemoryRegion ram_debug_dump_bitmap: Dump a migration bitmap as text migrate_init: Call from savevm Move dirty page search state into separate structure ram_find_and_save_block: Split out the finding Rename save_live_complete to save_live_complete_precopy Return path: Open a return path on QEMUFile for sockets Return path: socket_writev_buffer: Block even on non-blocking fd's Migration commands Return path: Control commands Return path: Send responses from destination to source Return path: Source handling of return path Rework loadvm path for subloops Add migration-capability boolean for postcopy-ram. Add wrappers and handlers for sending/receiving the postcopy-ram migration messages. MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Modify save_live_pending for postcopy postcopy: OS support test migrate_start_postcopy: Command to trigger transition to postcopy MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Avoid sending vmdescription during postcopy Add qemu_savevm_state_complete_postcopy Postcopy: Maintain sentmap and calculate discard postcopy: Incoming initialisation postcopy: ram_enable_notify to switch on userfault Postcopy: Postcopy startup in migration thread Split out end of migration code from migration_thread Postcopy: End of iteration Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Page request: Process incoming page request Page request: Consume pages off the post-copy queue postcopy_ram.c: place_page and helpers Postcopy: Use helpers to map pages during migration Don't sync dirty bitmaps in postcopy Don't iterate on precopy-only devices during postcopy Host page!=target page: Cleanup bitmaps postcopy: Check order of received target pages Round up RAMBlock sizes to host page sizes Postcopy; Handle userfault requests Start up a postcopy/listener thread ready for incoming page data postcopy: Wire up loadvm_postcopy_handle_ commands Postcopy: Mark nohugepage before discard End of migration for postcopy Disable mlock around incoming postcopy Inhibit ballooning during postcopy balloon.c | 11 + docs/migration.txt | 191 ++++++++ exec.c | 72 ++- hmp-commands.hx | 15 + hmp.c | 7 + hmp.h | 1 + hw/ppc/spapr.c | 2 +- hw/virtio/virtio-balloon.c | 4 +- include/exec/cpu-common.h | 3 + include/exec/ram_addr.h | 2 - include/migration/migration.h | 128 ++++- include/migration/postcopy-ram.h | 99 ++++ include/migration/qemu-file.h | 10 + include/migration/vmstate.h | 8 +- include/qemu/osdep.h | 9 + include/qemu/typedefs.h | 3 + include/sysemu/balloon.h | 2 + include/sysemu/sysemu.h | 46 +- migration/Makefile.objs | 2 +- migration/block.c | 9 +- migration/migration.c | 753 +++++++++++++++++++++++++++-- migration/postcopy-ram.c | 763 ++++++++++++++++++++++++++++++ migration/qemu-file-unix.c | 111 ++++- migration/qemu-file.c | 74 +++ migration/ram.c | 995 +++++++++++++++++++++++++++++++++++---- migration/savevm.c | 825 ++++++++++++++++++++++++++++---- qapi-schema.json | 18 +- qmp-commands.hx | 19 + qtest.c | 1 + trace-events | 81 +++- 30 files changed, 3996 insertions(+), 268 deletions(-) create mode 100644 include/migration/postcopy-ram.h create mode 100644 migration/postcopy-ram.c -- 2.5.0
