Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-23 Thread Michael Galaxy
Hi All, This is just a heads up: I will be changing employment soon, so my Akamai email address will cease to operate this week. My personal email: mich...@flatgalaxy.com. I'll re-subscribe later once I have come back online to work soon. Thanks! - Michael On 10/7/24 08:45, Michael Galaxy

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-08 Thread Zhu Yanjun
在 2024/10/8 2:15, Leon Romanovsky 写道: On Mon, Oct 07, 2024 at 08:45:07AM -0500, Michael Galaxy wrote: Hi, On 10/7/24 03:47, Yu Zhang wrote: !---| This Message Is From an External Sender This message came from outside your

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-07 Thread Leon Romanovsky
On Mon, Oct 07, 2024 at 08:45:07AM -0500, Michael Galaxy wrote: > Hi, > > On 10/7/24 03:47, Yu Zhang wrote: > > !---| > >This Message Is From an External Sender > >This message came from outside your organization. > > |---

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-07 Thread Michael Galaxy
Hi, On 10/7/24 03:47, Yu Zhang wrote: !---| This Message Is From an External Sender This message came from outside your organization. |---! Sure, as we talked at

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-07 Thread Yu Zhang
Sure, as we talked at the KVM Forum, a possible approach is to set up two VMs on a physical host, configure the SoftRoCE, and run the migration test in two nested VMs to ensure that the migration data traffic goes through the emulated RDMA hardware. I will continue with this and let you know. On

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-04 Thread Michael Galaxy
On 10/3/24 16:43, Peter Xu wrote: !---| This Message Is From an External Sender This message came from outside your organization. |---! On Thu, Oct 03, 2024 at

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-03 Thread Peter Xu
On Thu, Oct 03, 2024 at 04:26:27PM -0500, Michael Galaxy wrote: > What about the testing solution that I mentioned? > > Does that satisfy your concerns? Or is there still a gap here that needs to > be met? I think such testing framework would be helpful, especially if we can kick it off in CI whe

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-10-03 Thread Michael Galaxy
On 9/30/24 14:47, Peter Xu wrote: !---| This Message Is From an External Sender This message came from outside your organization. |---! On Mon, Sep 30, 2024 at 07

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-30 Thread Peter Xu
On Mon, Sep 30, 2024 at 07:20:56PM +, Sean Hefty wrote: > > > I'm sure rsocket has its place with much smaller transfer sizes, but > > > this is very different. > > > > Is it possible to make rsocket be friendly with large buffers (>4GB) like > > the VM > > use case? > > If you can perform l

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-30 Thread Sean Hefty
> > I'm sure rsocket has its place with much smaller transfer sizes, but > > this is very different. > > Is it possible to make rsocket be friendly with large buffers (>4GB) like the > VM > use case? If you can perform large VM migrations using streaming sockets, rsockets is likely usable, but

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-30 Thread Peter Xu
On Sat, Sep 28, 2024 at 12:52:08PM -0500, Michael Galaxy wrote: > On 9/27/24 16:45, Sean Hefty wrote: > > !---| > >This Message Is From an External Sender > >This message came from outside your organization. > > |--

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-30 Thread Yu Zhang
Hello Michael, That's true. To my understanding, to ease the maintenance, Gonglei's team has taken efforts to refactorize the RDMA migration code by using rsocket. However, due to a certain limitation in rsocket, it turned out that only small VM (in terms of core number and memory) can be migrated

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-30 Thread Michael Galaxy
On 9/29/24 17:26, Michael S. Tsirkin wrote: !---| This Message Is From an External Sender This message came from outside your organization. |---! On Sun, Sep 29

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-29 Thread Michael S. Tsirkin
On Sun, Sep 29, 2024 at 03:26:58PM -0500, Michael Galaxy wrote: > > On 9/29/24 13:14, Michael S. Tsirkin wrote: > > !---| > >This Message Is From an External Sender > >This message came from outside your organization. > > |---

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-29 Thread Michael Galaxy
On 9/29/24 13:14, Michael S. Tsirkin wrote: !---| This Message Is From an External Sender This message came from outside your organization. |---! On Sat, Sep 28

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-29 Thread Michael S. Tsirkin
On Sat, Sep 28, 2024 at 12:52:08PM -0500, Michael Galaxy wrote: > A bounce buffer defeats the entire purpose of using RDMA in these cases. > When using RDMA for very large transfers like this, the goal here is to map > the entire memory region at once and avoid all CPU interactions (except for > me

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-28 Thread Michael Galaxy
On 9/27/24 16:45, Sean Hefty wrote: !---| This Message Is From an External Sender This message came from outside your organization. |---! I have met with the tea

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-27 Thread Sean Hefty
> > > I have met with the team from IONOS about their testing on actual IB > > > hardware here at KVM Forum today and the requirements are starting > > > to make more sense to me. I didn't say much in our previous thread > > > because I misunderstood the requirements, so let me try to explain > > >

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-27 Thread Michael Galaxy
...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou ; linux-r...@vger.kernel.org; lixiao (H) ; jinpu.w...@ionos.com; Wangjialin Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API Hi All, I have met with the team from IONOS about their testing on actual IB hardware here at KVM

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-25 Thread Peter Xu
) ; qemu-devel@nongnu.org; > > yu.zh...@ionos.com; elmar.ger...@ionos.com; zhengchuan > > ; berra...@redhat.com; arm...@redhat.com; > > lizhij...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou > > ; linux-r...@vger.kernel.org; lixiao (H) > > ; jinpu.w...@ionos.co

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-22 Thread Gonglei (Arei)
ra...@redhat.com; arm...@redhat.com; > lizhij...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou > ; linux-r...@vger.kernel.org; lixiao (H) > ; jinpu.w...@ionos.com; Wangjialin > > Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API > > Hi All, > >

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-09-22 Thread Michael Galaxy
Hi All, I have met with the team from IONOS about their testing on actual IB hardware here at KVM Forum today and the requirements are starting to make more sense to me. I didn't say much in our previous thread because I misunderstood the requirements, so let me try to explain and see if we'r

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-08-27 Thread Michael S. Tsirkin
On Tue, Aug 27, 2024 at 04:15:42PM -0400, Peter Xu wrote: > On Tue, Jun 04, 2024 at 08:14:06PM +0800, Gonglei wrote: > > From: Jialin Wang > > > > Hi, > > > > This patch series attempts to refactor RDMA live migration by > > introducing a new QIOChannelRDMA class based on the rsocket API. > > >

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-08-27 Thread Peter Xu
On Tue, Jun 04, 2024 at 08:14:06PM +0800, Gonglei wrote: > From: Jialin Wang > > Hi, > > This patch series attempts to refactor RDMA live migration by > introducing a new QIOChannelRDMA class based on the rsocket API. > > The /usr/include/rdma/rsocket.h provides a higher level rsocket API > tha

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-10 Thread Peter Xu
On Fri, Jun 07, 2024 at 08:49:01AM +, Gonglei (Arei) wrote: > Actually we tried this solution, but it didn't work. Pls see patch 3/6 > > Known limitations: > For a blocking rsocket fd, if we use io_create_watch to wait for > POLLIN or POLLOUT events, since the rsocket fd is blocking, we >

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-10 Thread Peter Xu
at.com; yu.zh...@ionos.com; > > mgal...@akamai.com; elmar.ger...@ionos.com; zhengchuan > > ; berra...@redhat.com; arm...@redhat.com; > > lizhij...@fujitsu.com; pbonz...@redhat.com; m...@redhat.com; Xiexiangyou > > ; linux-r...@vger.kernel.org; lixiao (H) > > ; Wangjialin

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-07 Thread Yu Zhang
.@akamai.com; elmar.ger...@ionos.com; zhengchuan > > ; berra...@redhat.com; arm...@redhat.com; > > lizhij...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou > > ; linux-r...@vger.kernel.org; lixiao (H) > > ; jinpu.w...@ionos.com; Wangjialin > > > > Subject: Re: [PATCH 0/6] re

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-07 Thread Daniel P . Berrangé
On Tue, Jun 04, 2024 at 03:32:19PM -0400, Peter Xu wrote: > Hi, Lei, Jialin, > > Thanks a lot for working on this! > > I think we'll need to wait a bit on feedbacks from Jinpu and his team on > RDMA side, also Daniel for iochannels. Also, please remember to copy > Fabiano Rosas in any relevant f

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-07 Thread Gonglei (Arei)
..@redhat.com; lizhij...@fujitsu.com; > pbonz...@redhat.com; m...@redhat.com; Xiexiangyou > ; linux-r...@vger.kernel.org; lixiao (H) > ; jinpu.w...@ionos.com; Wangjialin > ; Fabiano Rosas > Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API > > On Wed,

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-07 Thread Gonglei (Arei)
ra...@redhat.com; arm...@redhat.com; > lizhij...@fujitsu.com; pbonz...@redhat.com; m...@redhat.com; Xiexiangyou > ; linux-r...@vger.kernel.org; lixiao (H) > ; Wangjialin > Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API > > Hi Gonglei, hi folks on the li

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-06 Thread Jinpu Wang
Hi Gonglei, hi folks on the list, On Tue, Jun 4, 2024 at 2:14 PM Gonglei wrote: > > From: Jialin Wang > > Hi, > > This patch series attempts to refactor RDMA live migration by > introducing a new QIOChannelRDMA class based on the rsocket API. > > The /usr/include/rdma/rsocket.h provides a higher

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-06 Thread Zhijian Li (Fujitsu)
you >>> ; linux-r...@vger.kernel.org; lixiao (H) >>> ; jinpu.w...@ionos.com; Wangjialin >>> >>> Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API >>> >>> On Tue, Jun 04, 2024 at 08:14:06PM +0800, Gonglei wrote: >>>> F

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-06 Thread Leon Romanovsky
et...@redhat.com; yu.zh...@ionos.com; > > mgal...@akamai.com; elmar.ger...@ionos.com; zhengchuan > > ; berra...@redhat.com; arm...@redhat.com; > > lizhij...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou > > ; linux-r...@vger.kernel.org; lixiao (H) > > ; jinpu.w...@ionos.co

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-05 Thread Peter Xu
gjialin > > ; Fabiano Rosas > > Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API > > > > Hi, Lei, Jialin, > > > > Thanks a lot for working on this! > > > > I think we'll need to wait a bit on feedbacks from Jinpu

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-05 Thread Michael S. Tsirkin
et...@redhat.com; yu.zh...@ionos.com; > > mgal...@akamai.com; elmar.ger...@ionos.com; zhengchuan > > ; berra...@redhat.com; arm...@redhat.com; > > lizhij...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou > > ; linux-r...@vger.kernel.org; lixiao (H) > > ; jinpu.w...@ionos.co

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-05 Thread Gonglei (Arei)
om; arm...@redhat.com; lizhij...@fujitsu.com; > pbonz...@redhat.com; m...@redhat.com; Xiexiangyou > ; linux-r...@vger.kernel.org; lixiao (H) > ; jinpu.w...@ionos.com; Wangjialin > ; Fabiano Rosas > Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API > > Hi,

RE: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-05 Thread Gonglei (Arei)
; berra...@redhat.com; arm...@redhat.com; > lizhij...@fujitsu.com; pbonz...@redhat.com; Xiexiangyou > ; linux-r...@vger.kernel.org; lixiao (H) > ; jinpu.w...@ionos.com; Wangjialin > > Subject: Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API > > On Tue, Jun 0

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-05 Thread Michael S. Tsirkin
On Tue, Jun 04, 2024 at 08:14:06PM +0800, Gonglei wrote: > From: Jialin Wang > > Hi, > > This patch series attempts to refactor RDMA live migration by > introducing a new QIOChannelRDMA class based on the rsocket API. > > The /usr/include/rdma/rsocket.h provides a higher level rsocket API > tha

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-04 Thread Peter Xu
Hi, Lei, Jialin, Thanks a lot for working on this! I think we'll need to wait a bit on feedbacks from Jinpu and his team on RDMA side, also Daniel for iochannels. Also, please remember to copy Fabiano Rosas in any relevant future posts. We'd also like to know whether he has any comments too. I