Signed-off-by: Orit Wasserman <owass...@redhat.com> --- docs/xbzrle.txt | 114 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 114 insertions(+), 0 deletions(-) create mode 100644 docs/xbzrle.txt
diff --git a/docs/xbzrle.txt b/docs/xbzrle.txt new file mode 100644 index 0000000..16950d5 --- /dev/null +++ b/docs/xbzrle.txt @@ -0,0 +1,114 @@ +XBZRLE (Xor Based Zero Run Length Encoding) +=========================================== + +Using XBZRLE (Xor Based Zero Run Length Encoding) allows for the reduction of VM +downtime and the total live-migration time of Virtual machines. +It is particularly useful for virtual machines running memory write intensive +workloads that are typical of large enterprise applications such as SAP ERP +Systems, and generally speaking for any application that uses a sparse memory +update pattern. + +Instead of sending the changed guest memory page this solution will send a +compressed version of the updates, thus reducing the amount of data sent during +live migration. +In order to be able to calculate the update, the previous memory pages needed to +be stored. Those pages are stored in a dedicated cache (hash table) and are +accessed by their address. +The larger the cache size the better the chances are that the page has already +been stored in the cache. +A small cache size will result in high cache miss rate. +Cache size can be changed before and during migration. + +Format +======= + +The compression format uses the zero value, where zero represents an unchanged +value. +The page data delta is represented by zero and non zero runs. +A zero run is represented by it's length (in bytes). +A non zero run is represented by it's length (in bytes) and the data. +The run length is encoded using ULEB128 (http://en.wikipedia.org/wiki/LEB128) + +page = zrun nzrun + | zrun nzrun page + +zrun = length + +nzrun = length byte... + +length = uleb128 encoded integer + +On the sender side XBZRLE is used as a compact delta encoding of page updates, +retrieving the old page content from the cache (default size of 512 MB). The +receiving side uses the existing page's content and XBZRLE to decode the new +page's content. + +This is a more compact way to store the deltas than the previous version. + +This work was originally based on research results published +VEE 2011: Evaluation of Delta Compression Techniques for Efficient Live +Migration of Large Virtual Machines by Benoit, Svard, Tordsson and Elmroth. +Additionally the delta encoder XBRLE was improved further using the XBZRLE +instead. + +XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making it +ideal for in-line, real-time encoding such as is needed for live-migration. + +Migration Capabilities +====================== +In order to use XBZRLE the destination QEMU version should be able to +decode the new format. +Adding a new migration capabilities command that will allow external management +to query for it support. +A typical use for the destination + {qemu} info migrate_capabilities + {qemu} xbzrle, ... + +In order to enable capabilities for future live migration, +a new command migrate_set_parameter is introduced: + {qemu} migrate_set_parameter xbzrle + +Usage +====== + +1. Activate xbzrle +2. Set the XBZRLE cache size - the cache size is in MBytes and should be a +power of 2. The cache default value is 64MBytes. +3. start outgoing migration + +A typical usage scenario: + {qemu} migrate_set_parameter xbzrle + {qemu} migrate_set_cachesize 256m + {qemu} migrate -d tcp:destination.host:4444 + {qemu} info migrate + ... + transferred ram-duplicate: A kbytes + transferred ram-normal: B kbytes + transferred ram-xbrle: C kbytes + overflow ram-xbrle: D pages + cache-miss ram-xbrle: E pages + +cache-miss: the number of cache misses to date - high cache-miss rate +indicates that the cache size is set too low. +overflow: the number of overflows in the decoding which where the delta could +not be compressed. This can happen if the changes in the pages are too large +or there are many short changes for example change every second byte (half a +page). + +Testing: Testing indicated that live migration with XBZRLE was completed in 110 +seconds, whereas without it would not be able to complete. + +A simple synthetic memory r/w load generator: +.. include <stdlib.h> +.. include <stdio.h> +.. int main() +.. { +.. char *buf = (char *) calloc(4096, 4096); +.. while (1) { +.. int i; +.. for (i = 0; i < 4096 * 4; i++) { +.. buf[i * 4096 / 4]++; +.. } +.. printf("."); +.. } +.. } -- 1.7.7.6
