On 04/19/2016 04:20 PM, Liang Li wrote:
Current QEMU live migration implementation marks all guest's RAM pages
as dirtied in the ram bulk stage, all these pages will be processed
and it consumes quite a lot of CPU cycles and network bandwidth.
From guest's point of view, it doesn't care about the content in free
page. We can make use of this fact and skip processing the free
pages, it can save a lot CPU cycles and reduce the network traffic
significantly while speed up the live migration process obviously.
This patch set is the QEMU side implementation.
The virtio-balloon is extended so that QEMU can get the free pages
information from the guest.
After getting the free page bitmap, QEMU can use it to filter out
guest's free pages. This make the live migration process much more
efficient.
In order to skip more free pages, we add an interface to let the user
decide whether dropping the cache in guest during live migration.
Wouldn't that potentially break things like CoLo? Which side is the slow
one? We already do condense zero pages into small metadata in the
migration stream. Is writing them out the slow part? If so, just don't
write them in you're on a clean state.
Alex
Performance data
================
Test environment:
CPU: Intel (R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
Host RAM: 64GB
Host Linux Kernel: 4.4.0 Host OS: CentOS 7.1
Guest Linux Kernel: 4.5.rc6 Guest OS: CentOS 6.6
Network: Intel X710 with 10 Gigabit connection
Guest RAM: 8GB
Case 1: Idle guest just boots:
==================================================
| original | skip free page
--------------------------------------------------
total time(ms) | 1505 | 573
--------------------------------------------------
transferred ram(KB) | 399792 | 379057
==================================================
Case 2: The guest has ever run some memory consuming workload, the
workload is terminated before live migration.
==================================================
| original | skip free page
--------------------------------------------------
total time(ms) | 10641 | 597
--------------------------------------------------
transferred ram(KB) | 8350829 | 389900
==================================================
Case 3: The guest has ever built the linux kernel, the building is
completed before live migration. The page cache is dropped
during live migration in this case.
==================================================
| original | skip free page
--------------------------------------------------
total time(ms) | 2636 | 914
--------------------------------------------------
transferred ram(KB) | 1316747 | 421980
==================================================
Liang Li (5):
bitmap: Add a new bitmap_move function
kvm: Add two new arch specific functions
virtio-balloon: Add a new feature to balloon device
migration: filter out free pages during live migration
migration: Add the interface for cache drop control
balloon.c | 29 +++++++-
hmp.c | 8 +++
hw/virtio/virtio-balloon.c | 92 +++++++++++++++++++++++-
include/hw/virtio/virtio-balloon.h | 30 +++++++-
include/migration/migration.h | 1 +
include/qemu/bitmap.h | 13 ++++
include/standard-headers/linux/virtio_balloon.h | 1 +
include/sysemu/balloon.h | 12 +++-
include/sysemu/kvm.h | 2 +
migration/migration.c | 31 +++++++-
migration/ram.c | 95 +++++++++++++++++++++++++
qapi-schema.json | 25 ++++++-
target-arm/kvm.c | 14 ++++
target-i386/kvm.c | 35 +++++++++
target-mips/kvm.c | 14 ++++
target-ppc/kvm.c | 14 ++++
target-s390x/kvm.c | 14 ++++
17 files changed, 421 insertions(+), 9 deletions(-)
