From: "Dr. David Alan Gilbert" <dgilb...@redhat.com> This is the 9th cut of my version of postcopy.
The userfaultfd linux kernel code is now in the upstream kernel tree, and so 4.3 can be used without modification. This qemu series can be found at: https://github.com/orbitfp7/qemu.git on the wp3-postcopy-v9 tag Testing status: * Tested heavily on x86 * Smoke tested on aarch64 (so it does work on different page sizes) This work has been partially funded by the EU Orbit project: see http://www.orbitproject.eu/about/ v9 Almost all of the changes are changes from review comments, and most are minor. The following are new patches, mostly split out from earlier patches (The exception being adding userfaultfd.h header back in - which was in v5 but we took out, but is needed again due to 1842bdfd) 04/56 Move page_size_init earlier 08/56 qemu_ram_block_by_name 12/56 Factor out host_from_stream_offset call and check 15/56 Add Linux userfaultfd.h header back 21/56 migration_is_setup_or_active 30/56 migration_completion: Take current state 34/56 Maintain unsentmap The previous patches 03,10,13,36/54 went in upstream already. Fix for assert using hotplug (Thanks Bharata for spotting that) Fix for migrate_cancel a second time Rework for migration_bitmap_rcu after Denis's deadlock fix Rework ram_load into a separate postcopy loop The 'sentmap' is now an 'unsentmap' - this saves a complement step at the end The unsentmap creation is now split into a separate patch The unsentmap is now stored in the RCU structure (although it can't really resize during the migrate) Fix for block migration still not a suggested combination. Minor: trace_savevm_send_open_return_path added split migration_is_active function into separate patch and made migration_is_setup_or_active move file reads into migrate_handle_advise and migrate_handle_packaged use of MIN in send-packaged-chunk migration_thread_started -> migration_thread_running updated qemu_get_buffer_in_place with Juan's version (and size_t'ified it) split the host_from_stream_offset change into a separate patch 'ram_load: Factor out host_from_stream_offset call and check' split out ram_find_block_by_id into a separate patch and now called qemu_ram_block_by_name postcopy_discard_send_range etc now take start/length rather than start/end (also added another trace) split ram_save_host_page into ram_save_host_page/ram_save_target_page split host page cleanup function into a core that handles both passes Dave Dr. David Alan Gilbert (56): Add postcopy documentation Provide runtime Target page information Move configuration section writing Move page_size_init earlier qemu_ram_block_from_host qemu_ram_block_by_name Rename mis->file to from_src_file Add qemu_get_buffer_in_place to avoid copies some of the time Add wrapper for setting blocking status on a QEMUFile Add QEMU_MADV_NOHUGEPAGE ram_debug_dump_bitmap: Dump a migration bitmap as text ram_load: Factor out host_from_stream_offset call and check migrate_init: Call from savevm Rename save_live_complete to save_live_complete_precopy Add Linux userfaultfd.h header Return path: Open a return path on QEMUFile for sockets Return path: socket_writev_buffer: Block even on non-blocking fd's Migration commands Return path: Control commands Return path: Send responses from destination to source migration_is_setup_or_active Return path: Source handling of return path Rework loadvm path for subloops Add migration-capability boolean for postcopy-ram. Add wrappers and handlers for sending/receiving the postcopy-ram migration messages. MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Modify save_live_pending for postcopy postcopy: OS support test migrate_start_postcopy: Command to trigger transition to postcopy migration_completion: Take current state MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Avoid sending vmdescription during postcopy Add qemu_savevm_state_complete_postcopy Postcopy: Maintain unsentmap Postcopy: Calculate discard postcopy: Incoming initialisation postcopy: ram_enable_notify to switch on userfault Postcopy: Postcopy startup in migration thread Postcopy: End of iteration Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Page request: Process incoming page request Page request: Consume pages off the post-copy queue postcopy_ram.c: place_page and helpers Postcopy: Use helpers to map pages during migration postcopy: Check order of received target pages Don't sync dirty bitmaps in postcopy Don't iterate on precopy-only devices during postcopy Host page!=target page: Cleanup bitmaps Round up RAMBlock sizes to host page sizes Postcopy; Handle userfault requests Start up a postcopy/listener thread ready for incoming page data postcopy: Wire up loadvm_postcopy_handle_ commands Postcopy: Mark nohugepage before discard End of migration for postcopy Disable mlock around incoming postcopy Inhibit ballooning during postcopy balloon.c | 11 + docs/migration.txt | 191 ++++++++ exec.c | 92 +++- hmp-commands.hx | 15 + hmp.c | 7 + hmp.h | 1 + hw/ppc/spapr.c | 2 +- hw/virtio/virtio-balloon.c | 4 +- include/exec/cpu-common.h | 4 + include/exec/exec-all.h | 1 - include/exec/ram_addr.h | 2 - include/migration/migration.h | 121 ++++- include/migration/postcopy-ram.h | 99 ++++ include/migration/qemu-file.h | 10 + include/migration/vmstate.h | 8 +- include/qemu-common.h | 1 + include/qemu/osdep.h | 9 + include/qemu/typedefs.h | 3 + include/sysemu/balloon.h | 2 + include/sysemu/sysemu.h | 46 +- kvm-all.c | 1 - linux-headers/linux/userfaultfd.h | 167 +++++++ migration/Makefile.objs | 2 +- migration/block.c | 9 +- migration/migration.c | 723 +++++++++++++++++++++++++-- migration/postcopy-ram.c | 760 +++++++++++++++++++++++++++++ migration/qemu-file-unix.c | 111 ++++- migration/qemu-file.c | 64 +++ migration/ram.c | 997 +++++++++++++++++++++++++++++++++++--- migration/savevm.c | 823 +++++++++++++++++++++++++++---- qapi-schema.json | 18 +- qmp-commands.hx | 19 + qtest.c | 1 - trace-events | 84 +++- vl.c | 1 + 35 files changed, 4159 insertions(+), 250 deletions(-) create mode 100644 include/migration/postcopy-ram.h create mode 100644 linux-headers/linux/userfaultfd.h create mode 100644 migration/postcopy-ram.c -- 2.5.0
