On 02/18/2013 03:24 AM, Orit Wasserman wrote:
Hi Michael, The guest device state is quite small (~100K probably
less) especially when compared to the guest memory and we already are
pinning the guest memory for RDMA any way I was actually wondering
about the memory pinning, do we pin all guest memory pages as
migration starts or on demand?
The patch supports both methods. There's a function called
"rdma_server_prepare()" which optionally pins all the memory in advanced.
We prefer on-demand pinning, of course, because we want to preserve the
ability to do ballooning and the occasional madvise() calls.
The patch defaults to pinning everything right now for performance
evaluation.......... later I'll make sure to switch that off when we've
coalesced on a solution for the actual RDMA transfer itself.
For the guest memory pages, sending the pages directly without
QemuFile (that does buffering) is better, I would suggest implementing
an QemuRDMAFile for this. It will have a new API for the memory pages
(zero copy) so instead of using qemu_put_buffer we will call
qemu_rdma_buffer or it can reimplement qemu_put_buffer (you need to
add offset). As for device state which is sent in the last phase and
is small you can modify the current implementation. (well Paolo sent
patches that are changing this but I think buffering is still an
option) The current migration implementation copies the device state
into a buffer and than send the data from the buffer
(QemuBufferedFile). You only need to pin this buffer, and RDMA it
after all the device state was written into it. Regards, Orit
I like it .......... can you help me understand - how much different is
this design than the "QEMUFileOps" design that Paulo suggested?
Or it basically the same? ....reimplementing
qemu_put_buffer/get_buffer() for RDMA purposes......
- Michael
