Thanks for your review. I will add more trace_ calls in the next version(V2) and modify its according to your suggestions.
-----Original Message----- From: Dr. David Alan Gilbert [mailto:dgilb...@redhat.com] Sent: Thursday, January 16, 2020 3:57 AM To: fengzhimin <fengzhim...@huawei.com> Cc: quint...@redhat.com; arm...@redhat.com; ebl...@redhat.com; qemu-devel@nongnu.org; Zhanghailiang <zhang.zhanghaili...@huawei.com>; jemmy858...@gmail.com Subject: Re: [PATCH RFC 00/12] *** mulitple RDMA channels for migration *** * Zhimin Feng (fengzhim...@huawei.com) wrote: > From: fengzhimin <fengzhim...@huawei.com> > > Currently there is a single channel for RDMA migration, this causes > the problem that the network bandwidth is not fully utilized for > 25Gigabit NIC. Inspired by the Multifd, we use two RDMA channels to > send RAM pages, which we call MultiRDMA. > > We compare the migration performance of MultiRDMA with origin RDMA > migration. The VM specifications for migration are as follows: > - VM use 4k page; > - the number of VCPU is 4; > - the total memory is 16Gigabit; > - use 'mempress' tool to pressurize VM(mempress 8000 500); > - use 25Gigabit network card to migrate; > > For origin RDMA and MultiRDMA migration, the total migration times of > VM are as follows: > +++++++++++++++++++++++++++++++++++++++++++++++++ > | | NOT rdma-pin-all | rdma-pin-all | > +++++++++++++++++++++++++++++++++++++++++++++++++ > | origin RDMA | 18 s | 23 s | > ------------------------------------------------- > | MultiRDMA | 13 s | 18 s | > +++++++++++++++++++++++++++++++++++++++++++++++++ Very nice. > For NOT rdma-pin-all migration, the multiRDMA can improve the total > migration time by about 27.8%. > For rdma-pin-all migration, the multiRDMA can improve the total > migration time by about 21.7%. > > Test the multiRDMA migration like this: > 'virsh migrate --live --rdma-parallel --migrateuri rdma://hostname > domain qemu+tcp://hostname/system' It will take me a while to finish the review; but another general suggestion is add more trace_ calls; it will make it easier to diagnose problems later. Dave > > fengzhimin (12): > migration: Add multiRDMA capability support > migration: Export the 'migration_incoming_setup' function > and add the 'migrate_use_rdma_pin_all' function > migration: Create the multi-rdma-channels parameter > migration/rdma: Create multiRDMA migration threads > migration/rdma: Create the multiRDMA channels > migration/rdma: Transmit initial package > migration/rdma: Be sure all channels are created > migration/rdma: register memory for multiRDMA channels > migration/rdma: Wait for all multiRDMA to complete registration > migration/rdma: use multiRDMA to send RAM block for rdma-pin-all mode > migration/rdma: use multiRDMA to send RAM block for NOT rdma-pin-all > mode > migration/rdma: only register the virt-ram block for MultiRDMA > > migration/migration.c | 55 +- > migration/migration.h | 6 + > migration/rdma.c | 1320 +++++++++++++++++++++++++++++++++++++---- > monitor/hmp-cmds.c | 7 + > qapi/migration.json | 27 +- > 5 files changed, 1285 insertions(+), 130 deletions(-) > > -- > 2.19.1 > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
