Hi, I am writing to submit a code change aimed at enhancing live migration acceleration by leveraging the compression capability of the Intel In-Memory Analytics Accelerator (IAA).
The implementation of the IAA (de)compression code is based on Intel Query Processing Library (QPL), an open-source software project designed for IAA high-level software programming. https://github.com/intel/qpl In the last version, there was some discussion about whether to introduce a new compression algorithm for IAA. Because the compression algorithm of IAA hardware is based on deflate, and QPL already supports Zlib, so in this version, I implemented IAA as an accelerator for the Zlib compression method. However, due to some reasons, QPL is currently not compatible with the existing Zlib method that Zlib compressed data can be decompressed by QPl and vice versa. I have some concerns about the existing Zlib compression 1. Will you consider supporting one channel to support multi-stream compression? Of course, this may lead to a reduction in compression ratio, but it will allow the hardware to process each stream concurrently. We can have each stream process multiple pages, reducing the loss of compression ratio. For example, 128 pages are divided into 16 streams for independent compression. I will provide the a early performance data in the next version(v4). 2. Will you consider using QPL/IAA as an independent compression algorithm instead of an accelerator? In this way, we can better utilize hardware performance and some features, such as IAA's canned mode, which can be dynamically generated by some statistics of data. A huffman table to improve the compression ratio. Test condition: 1. Host CPUs are based on Sapphire Rapids, and frequency locked to 3.4G 2. VM type, 16 vCPU and 64G memory 3. The Idle workload means no workload is running in the VM 4. The Redis workload means YCSB workloadb + Redis Server are running in the VM, about 20G or more memory will be used. 5. Source side migartion configuration commands a. migrate_set_capability multifd on b. migrate_set_parameter multifd-channels 2/4/8 c. migrate_set_parameter downtime-limit 300 d. migrate_set_parameter multifd-compression zlib e. migrate_set_parameter multifd-compression-accel none/qpl f. migrate_set_parameter max-bandwidth 100G 6. Desitination side migration configuration commands a. migrate_set_capability multifd on b. migrate_set_parameter multifd-channels 2/4/8 c. migrate_set_parameter multifd-compression zlib d. migrate_set_parameter multifd-compression-accel none/qpl e. migrate_set_parameter max-bandwidth 100G Early migration result, each result is the average of three tests +--------+-------------+--------+--------+---------+----+-----+ | | The number |total |downtime|network |pages per | | | of channels |time(ms)|(ms) |bandwidth|second | | | and mode | | |(mbps) | | | +-------------+-----------------+---------+----------+ | | 2 chl, Zlib | 20647 | 22 | 195 | 137767 | | +-------------+--------+--------+---------+----------+ | Idle | 2 chl, IAA | 17022 | 36 | 286 | 460289 | |workload+-------------+--------+--------+---------+----------+ | | 4 chl, Zlib | 18835 | 29 | 241 | 299028 | | +-------------+--------+--------+---------+----------+ | | 4 chl, IAA | 16280 | 32 | 298 | 652456 | | +-------------+--------+--------+---------+----------+ | | 8 chl, Zlib | 17379 | 32 | 275 | 470591 | | +-------------+--------+--------+---------+----------+ | | 8 chl, IAA | 15551 | 46 | 313 | 1315784 | +--------+-------------+--------+--------+---------+----------+ +--------+-------------+--------+--------+---------+----+-----+ | | The number |total |downtime|network |pages per | | | of channels |time(ms)|(ms) |bandwidth|second | | | and mode | | |(mbps) | | | +-------------+-----------------+---------+----------+ | | 2 chl, Zlib | 100% failure, timeout is 120s | | +-------------+--------+--------+---------+----------+ | Redis | 2 chl, IAA | 62737 | 115 | 4547 | 387911 | |workload+-------------+--------+--------+---------+----------+ | | 4 chl, Zlib | 30% failure, timeout is 120s | | +-------------+--------+--------+---------+----------+ | | 4 chl, IAA | 54645 | 177 | 5382 | 656865 | | +-------------+--------+--------+---------+----------+ | | 8 chl, Zlib | 93488 | 74 | 1264 | 129486 | | +-------------+--------+--------+---------+----------+ | | 8 chl, IAA | 24367 | 303 | 6901 | 964380 | +--------+-------------+--------+--------+---------+----------+ v2: - add support for multifd compression accelerator - add support for the QPL accelerator in the multifd compression accelerator - fixed the issue that QPL was compiled into the migration module by default v3: - use Meson instead of pkg-config to resolve QPL build dependency issue - fix coding style - fix a CI issue for get_multifd_ops function in multifd.c file Yuan Liu (4): migration: Introduce multifd-compression-accel parameter multifd: Implement multifd compression accelerator configure: add qpl option multifd: Introduce QPL compression accelerator hw/core/qdev-properties-system.c | 11 + include/hw/qdev-properties-system.h | 4 + meson.build | 18 ++ meson_options.txt | 2 + migration/meson.build | 1 + migration/migration-hmp-cmds.c | 10 + migration/multifd-qpl.c | 323 ++++++++++++++++++++++++++++ migration/multifd.c | 40 +++- migration/multifd.h | 8 + migration/options.c | 28 +++ migration/options.h | 1 + qapi/migration.json | 31 ++- scripts/meson-buildoptions.sh | 3 + 13 files changed, 477 insertions(+), 3 deletions(-) create mode 100644 migration/multifd-qpl.c -- 2.39.3
