Hi David, When i migrated VM in postcopy way when configuring VM with '-realtime mlock=on' option, It failed, and reports "postcopy_ram_hosttest: remap_anon_pages not available: File exists" in destination,
Is it a bug of userfaultfd API? cc: Andrea reproduce Steps: Source: qemu-postcopy/qemu # x86_64-softmmu/qemu-system-x86_64 -msg timestamp=on \ -machine pc-i440fx-2.2,accel=kvm -m 1024 -realtime mlock=on -smp 4 \ -hda /mnt/sdb/pure_IMG/redhat/redhat-6.4-httpd.img -vnc :11 -monitor stdio Destination: qemu-postcopy/qemu # x86_64-softmmu/qemu-system-x86_64 -msg timestamp=on \ -machine pc-i440fx-2.2,accel=kvm -m 1024 -realtime mlock=on -smp 4 \ -hda /mnt/sdb/pure_IMG/redhat/redhat-6.4-httpd.img -vnc :12 -monitor stdio \ -incoming unix:/mnt/migrate.sock (1) migrate_set_capability x-postcopy-ram on (2) migrate -d unix:/mnt/migrate.sock In Destination, it fails, reports: savevm@2040988668 qemu_loadvm_state_main QEMU_VM_COMMAND ret: 0 savevm@2040988668 qemu_loadvm_state loop: section_type=6 savevm@2040988668 loadvm_postcopy_ram_handle_advise postcopy_ram_hosttest: remap_anon_pages not available: File exists savevm@2040988668 qemu_loadvm_state_main QEMU_VM_COMMAND ret: -1 And one more thing, i want to know: ;) Why we must start precopy first before start postcopy? Can we do postcopy at the beginning of migration? Thanks, zhanghailiang On 2014/10/4 1:47, Dr. David Alan Gilbert (git) wrote:
From: "Dr. David Alan Gilbert" <dgilb...@redhat.com> Hi, This is the 4th cut of my version of postcopy; it is designed for use with the Linux kernel additions just posted by Andrea Arcangeli here: http://marc.info/?l=linux-kernel&m=141235633015100&w=2 (Note: This is a new version compared to my previous postcopy patchset; you'll need to update the kernel to the new version.) Other than the new kernel ABI (which is only a small change to the userspace side); the major changes are; a) Code for host page size != target page size b) Support for migration over fd From Cristian Klein; this is for libvirt support which Cristian recently posted to the libvirt list. c) It's now build bisectable and builds on 32bit Testing wise; I've now done many thousand of postcopy migrations without failure (both of idle and busy guests); so it seems pretty solid. Must-TODO's: 1) A partially repeatable migration_cancel failure 2) virt_test's migrate.with_reboot test is failing 3) The ACPI fix in 2.1 that allowed migrating RAMBlocks to be larger than the source feels like it needs looking at for postcopy. 4) Paolo's comments with respect to the wakeup_request/is_running code in the migration thread 5) xbzrle needs disabling once in postcopy Later-TODO's: 1) Control the rate of background page transfers during postcopy to reduce their impact on the latency of postcopy requests. 2) Work with RDMA 3) Could destination RP be made blocking (as per discussion with Paolo; I'm still worried that that changes too many assumptions) V4: Initial support for host page size != target page size - tested heavily on hps==tps - only partially tested on hps!=tps systems - This involved quite a bit of rework around the discard code Updated to new kernel userfault ABI - It won't work with the previous version Fix mis-optimisation of postcopy request for wrong RAMBlock request for block A offset n un-needed fault for block B/m (already received - no req sent) request for block B/l - wrongly sent as request for A/l Fix thinko in discard bitmap processing (missed last word of bitmap) Symptom: remap failures near the top of RAM if postcopy started late Fix bug that caused kernel page acknowledgments to be misaligned May have meant the guest was paused for longer than required Fix potential for crashing cleaning up failed RP Fixes in docs (from Yang) Handle migration by fd as sockets if they are sockets Build tested on 32bit Fully build bisectable (x86-64) Dave Cristian Klein (1): Handle bi-directional communication for fd migration Dr. David Alan Gilbert (46): QEMUSizedBuffer based QEMUFile Tests: QEMUSizedBuffer/QEMUBuffer Start documenting how postcopy works. qemu_ram_foreach_block: pass up error value, and down the ramblock name improve DPRINTF macros, add to savevm Add qemu_get_counted_string to read a string prefixed by a count byte Create MigrationIncomingState socket shutdown Provide runtime Target page information Return path: Open a return path on QEMUFile for sockets Return path: socket_writev_buffer: Block even on non-blocking fd's Migration commands Return path: Control commands Return path: Send responses from destination to source Return path: Source handling of return path qemu_loadvm errors and debug ram_debug_dump_bitmap: Dump a migration bitmap as text Rework loadvm path for subloops Add migration-capability boolean for postcopy-ram. Add wrappers and handlers for sending/receiving the postcopy-ram migration messages. QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream migrate_init: Call from savevm Allow savevm handlers to state whether they could go into postcopy postcopy: OS support test migrate_start_postcopy: Command to trigger transition to postcopy MIG_STATE_POSTCOPY_ACTIVE: Add new migration state qemu_savevm_state_complete: Postcopy changes Postcopy page-map-incoming (PMI) structure Postcopy: Maintain sentmap and calculate discard postcopy: Incoming initialisation postcopy: ram_enable_notify to switch on userfault Postcopy: Postcopy startup in migration thread Postcopy: Create a fault handler thread before marking the ram as userfault Page request: Add MIG_RPCOMM_REQPAGES reverse command Page request: Process incoming page request Page request: Consume pages off the post-copy queue Add assertion to check migration_dirty_pages postcopy_ram.c: place_page and helpers Postcopy: Use helpers to map pages during migration qemu_ram_block_from_host Don't sync dirty bitmaps in postcopy Host page!=target page: Cleanup bitmaps Postcopy; Handle userfault requests Start up a postcopy/listener thread ready for incoming page data postcopy: Wire up loadvm_postcopy_ram_handle_{run,end} commands End of migration for postcopy Makefile.objs | 2 +- arch_init.c | 739 +++++++++++++++++++++++++-- docs/migration.txt | 189 +++++++ exec.c | 76 ++- hmp-commands.hx | 15 + hmp.c | 7 + hmp.h | 1 + include/exec/cpu-common.h | 8 +- include/migration/migration.h | 130 +++++ include/migration/postcopy-ram.h | 106 ++++ include/migration/qemu-file.h | 47 ++ include/migration/vmstate.h | 2 +- include/qemu/sockets.h | 1 + include/qemu/typedefs.h | 9 +- include/sysemu/sysemu.h | 43 +- migration-fd.c | 24 +- migration-rdma.c | 4 +- migration.c | 693 +++++++++++++++++++++++++- postcopy-ram.c | 1016 ++++++++++++++++++++++++++++++++++++++ qapi-schema.json | 14 +- qemu-file.c | 598 +++++++++++++++++++++- qmp-commands.hx | 19 + savevm.c | 881 +++++++++++++++++++++++++++++++-- tests/Makefile | 2 +- tests/test-vmstate.c | 74 +-- util/qemu-sockets.c | 28 ++ 26 files changed, 4550 insertions(+), 178 deletions(-) create mode 100644 include/migration/postcopy-ram.h create mode 100644 postcopy-ram.c
